Visual Servoing

TeachFeature

Primitive Description and Usage

• Description: This primitive collects image feature data from AI multiple times and outputs their average value.

• Example Usage: Move the robot to the target position and use this primitive to record the target image features. You can save the values of the output parameters in the variables, and use them later in other primitives under the Visual Servoing category.

Primitive Input Parameters

Input Parameter	Description	Type	Unit	Default Value & Range
objName*	Object name used by AI	OBJNAME	none
objIndex	Object index used by AI	INT	none	0 ∈ [0 … 1]
collectTimes	Number of times AI collects image feature samples	INT	none	30 ∈ [10 … 200]

*Parameters marked with an asterisk must be assigned a value prior to executing the primitive.

Primitive State Parameters

State Parameter	Description	Type	Unit
terminated	The termination flag of the primitive. It is set to true if the primitive is terminated.	BOOL	none
timePeriod	The time spent on running the current primitive.	DOUBLE	s

Primitive Output Parameters

Output Parameter	Description	Type	Unit
tcpPoseOut	The TCP pose when the primitive is terminated. It is represented in the world coordinate system.	COORD	m-deg
taughtFeaturePoint	Position of the taught feature points in the image coordinate system. Its value can be used in input parameter [targetFeaturePoint] in primitive [AlignPoint] and input parameter [targetFeaturePattern] in primitive [AlignPattern].	ARRAY_VEC_2d	none
taughtFeaturePointNoise	Noise of the taught feature points. Its value can be used in input parameter [visualDetectNoise] in primitive [AlignPoint] and [AlignPattern].	ARRAY_VEC_2d	none
taughtMatchPoint	Position of the taught feature points of the aligned object in the image coordinate system. Its value can be used in input parameter [alignObjPoint] in primitive [AlignPoint].	ARRAY_VEC_2d	none
taughtFeatureLine	Position of the points on the taught feature lines in the image coordinate system. Its value can be used in the input parameter [targetFeatureLine] in primitive [AlignLine].	ARRAY_VEC_2d	none

Default Transition Condition

State Parameter	Condition	Value
terminated	=	1

AlignPoint

Primitive Description and Usage

• Description: This primitive uses Image-Based Visual Servoing (IBVS) to move the robot continuously based on real-time visual feedback to align the feature points on the target object with the user-defined feature points.

• Example Usage: Use this primitive to control the robot to grasp an object whose position is uncertain or changing. This primitive can be used to align two objects, such as aligning a PFC cable with a socket.

Primitive Input Parameters

Input Parameter	Description	Type	Unit	Default Value & Range
targetFeaturePoint*	Position of the target feature points in the image coordinate system. Each point is represented by two pixel values (X, Y).	ARRAY_VEC_2d	none
targetDepth*	Depth of target feature points in the image coordinate system	ARRAY_DOUBLE	m	[0.01 … 1.2]
motionAxis	Axes that are allowed to move in the image coordinate system	VEC_6i	none	1 1 1 0 0 1 ∈ [0 0 0 0 0 0 … 1 1 1 1 1 1]
maxVel	Maximum linear velocity of robot movement	DOUBLE	m/s	0.05 ∈ [0.001 … 0.5]
maxAngVel	Maximum angular velocity of robot movement	DOUBLE	deg/s	20.0 ∈ [1.0 … 100]
velScale	Velocity scale of visual servoing. Caution: Setting this value too large may cause robot vibration.	DOUBLE	none	10 ∈ [1.0 … 100.0]
imageConvToler	Convergence tolerance of visual servoing. When the error between the captured image features and the target features is within tolerance, it will be determined that they are aligned.	DOUBLE	none	2.0 ∈ [0.2 … 20]
objName*	Object name used by AI	OBJNAME	none
objIndex	Object index used by AI	INT	none	0 ∈ [0 … 1]
enableObjAlign	Flag to enable the object alignment function. This function can be used to align the feature points of two objects.	BOOL	none	0 ∈ [0 / 1]
alignObjPoint	Position of feature points of aligned object in the image coordinate system. Each point is represented by two pixel values (X, Y). The aligned object is in the robot hand and is moved by the robot.	ARRAY_VEC_2d	none
optVelScale	Increase the velocity scale of visual servoing in the Z and Rz directions. It can help optimize the robot’s Cartesian motion trajectory.	VEC_2d	none	5 5 ∈ [1.0 1.0 … 100 100]
visualDetectNoise	Noise of the visual detection result of each feature point	ARRAY_VEC_2d	none	[0 0 … 20 20]

*Parameters marked with an asterisk must be assigned a value prior to executing the primitive.

Primitive State Parameters

State Parameter	Description	Type	Unit
terminated	The termination flag of the primitive. It is set to true if the primitive is terminated.	BOOL	none
timePeriod	The time spent on running the current primitive.	DOUBLE	s
featureAligned	Flag to indicate if the maximum pixel error between target feature points and detected feature points is within tolerance	BOOL	none

Primitive Output Parameters

Output Parameter	Description	Type	Unit
tcpPoseOut	The TCP pose when the primitive is terminated. It is represented in the world coordinate system.	COORD	m-deg

Default Transition Condition

State Parameter	Condition	Value
reachedTarget	=	1

AlignPattern

Primitive Description and Usage

• Description: This primitive uses Image-Based Visual Servoing (IBVS) to move the robot continuously based on real-time visual feedback to align the feature patterns on the target object with the user-defined feature patterns.

• Example Usage: When the pattern features (such as rectangles) on the object are obvious, use this primitive to control the robot to align the object to the target position via visual servoing. It is recommended to use this primitive in applications that require more than 4 degrees of freedom alignment.

Primitive Input Parameters

Input Parameter	Description	Type	Unit	Default Value & Range
targetFeaturePattern*	Position of points on the target feature pattern in the image coordinate system. A pattern consists of several points. Each point is represented by two pixel values (X, Y).	ARRAY_VEC_2d	none
targetDepth*	Depth of target feature points in the image coordinate system	ARRAY_DOUBLE	m	[0.01 … 1.2]
motionAxis	Axes that are allowed to move in the image coordinate system	VEC_6i	none	1 1 1 0 0 1 ∈ [0 0 0 0 0 0 … 1 1 1 1 1 1]
maxVel	Maximum linear velocity of robot movement	DOUBLE	m/s	0.05 ∈ [0.001 … 0.5]
maxAngVel	Maximum angular velocity of robot movement	DOUBLE	deg/s	20.0 ∈ [1.0 … 100]
rectangleFeature	Flag to indicate if the feature pattern is rectangle	BOOL	none	1 ∈ [0 / 1]
velScale	Velocity scale of each axis in the image coordinate system for visual servoing. Caution: Setting this value too large may cause robot vibration.	VEC_6d	none	15 15 15 3 3 15 ∈ [0.0 0.0 0.0 0.0 0.0 0.0 … 100 100 100 100 100 100]
imageConvToler	Convergence tolerance of visual servoing. When the error between the captured image features and the target features is within tolerance, it will be determined that they are aligned.	DOUBLE	none	5.0 ∈ [1 … 200]
objName*	Object name used by AI	OBJNAME	none
objIndex	Object index used by AI	INT	none	0 ∈ [0 … 1]

*Parameters marked with an asterisk must be assigned a value prior to executing the primitive.

Primitive State Parameters

State Parameter	Description	Type	Unit
terminated	The termination flag of the primitive. It is set to true if the primitive is terminated.	BOOL	none
timePeriod	The time spent on running the current primitive.	DOUBLE	s
featureAligned	Flag to indicate if the maximum pixel error between target feature points and detected feature points is within tolerance	BOOL	none

Primitive Output Parameters

Output Parameter	Description	Type	Unit
tcpPoseOut	The TCP pose when the primitive is terminated. It is represented in the world coordinate system.	COORD	m-deg

Default Transition Condition

State Parameter	Condition	Value
reachedTarget	=	1