Skip to main content

estimate_pose_multi_view

Introduction​

The estimate_pose_multi_view skill can be used to estimate the pose of objects in the scene given at least 2 cameras (and a maximum of 4 cameras). We recommend using 3 cameras in a triangular configuration for the best estimation performance. The skill uses the input images obtained from the cameras to estimate the 6 DOF pose of every instance of the desired object in the scene. The CAD model of the object to be estimated is defined at training time, which is why it is important that the object matches the CAD model provided.

Prerequisites​

Required assets:

  • Cameras : The skill can be run with a minimum of 2 cameras and a maximum of 4. If less than 4 cameras are available, the remaining slots can be filled with an already passed camera. The cameras should be placed at similar distances to the objects to be detected. The distances and angles of all cameras w.r.t the objects in the region of interest should fall within the pose range specified at training time.
  • Pose estimator : the pose estimator for the object to be detected should be trained in advance following the ML Multi-view pose estimator training process.

The cameras must be extrinsically calibrated w.r.t each other. This means that, if the solution is running in real world, the relative position and orientation of the cameras should match (up to the calibration accuracy) the real world position. Follow camera-to-camera calibration process for more information on how to calibrate multiple cameras.

Usage example​

  • Set the Cameras that need to be used for pose estimation. The skill provides up to 4 possible slots. However, if less than 4 cameras are available, the remaining slots can be filled with an already passed in camera.
  • Set the Pose estimator to be used for the object that needs to be detected. The Pose estimator should be of type ML Multi-view pose estimator.

(Optional parameters):

  • Set the Min num instances parameter to define the minimum number of instances that the pose estimator should detect in the scene. The skill will fail if the not enough parts are detected in the scene. Default: 0.
  • Set Update object pose to update the pose of an object in the belief world with the first returned pose of the pose estimator.
  • Set Visibility score params to additionally calculate the visibility score of each detected part in the scene. Visibility score defines how visibile the part is and it can be used for grasp planning to tell the robot which part to pick up first.

When the skill is run, the skill dialog will display two images:

  • A raw image from the first camera
  • An annotated image where every detected object instance is highlighted.

Error handling​

Only one camera is passed​

The skill will fail if the same camera is passed 4 times in the camera slots. This skill can only be run with at least 2 different cameras. For single-view pose estimation follow estimate_pose skill guide.

Wrong pose estimator is passed​

The skill will fail if the pose estimator used is not a ML Multi-view pose estimator. To train a ML Multi-view pose estimator follow the ML Multi-view pose estimator training process.

Not enough instances detected​

The skill will fail if the instances of an object detected in a scene are less than Min num instances.