Page Comparison

...

• Aruco tag mounted off of the robot faceplate

• Static Aruco tag placed in the scene which is easily visible to the sensor

• Calibration preset with accurate TCP located at the center of the Aruco tag. More details in the cal.json file linked below

• cal.json file which has all the camera serial_numbers and it’s corresponding robot target positions for calibration which have the Aruco tag on the robot in the sensor field of view

Setup and Running Calibration Process

To easily generate the cal.json file and save it in /etc/rapidsense/cal.json, there is a calibration_generator.py script which prompts the user for all the necessary information. After the cal.json file has been generated, run the calibration_service.py script and go to localhost:9000/calibration to calibrate all the cameras after rapidsense_app and proxy are running. This script will pull in information from the cal.json file and make the robots move to the cal_targets sequentially, extrinsically calibrating all corresponding cameras. The robot will then return to the home location and all the cameras will save the pose of the static marker placed in the scene. After a successful calibration, all the values will be automatically updated in the cal.json file.

Architecture

The diagram below gives a quick overview of the architecture of the calibration sequence:

...

Calibration ASCII Interface

'topic': 'Calibrate', 'data': { 'serial_number' : serial_number, 'tcp_pose' : tcp_pose }

The calibration ASCII command { 'topic': 'Calibrate', 'data': { 'serial_number' : serial_number, 'tcp_pose' : tcp_pose } } is used where the serial_number is the camera to be calibrated and the TCP_pose is for the associated robot with the tag mounted on it. These values are sent over the proxy to rapidsense where aruco Aruco detection algorithm finds the markers which gives us the aruco_to_camera transformation. The TCP_pose is requested when in the calibration preset so we get the world_to_aruco transform.

Using these two transforms, we can calculate the world_to_camera transform as:

World_to_camera = World_to_aruco * aruco_to_cameraallows for calibration of the sensors to occur.

After successful calibration, we get the pose of the static marker is found and save it saved in the file cal.json. We send out a status = 0 to appliance and also update the GetRapidSenseStatus GetRapidSenseStatus is set as true for the is_calibrated for the corresponding sensor. The algorithm also periodically checks the pose of the static marker for every camera. Assuming the cameras have been bumped, we will get a different pose for the static marker than previously stored during calibration. Thus indicating that the calibration is off. Subsequently we will throw a warn the user and update the GetRapidSenseStatus

Calibration Checking Functionality

During the calibration process, the static Aruco tag in the scene is found and the image is cached to compare against during system running in order to detect when the sensors may have been shifted out of calibration. When it is detected that a sensor may have been shifted out of calibration, GetRapidSenseStatus is set to false for Is_calibrated for that particular sensor .

Further details of components can be found in the pages below:

...

in order to warn the user that the calibration needs to be adjusted and run again.

Demo Video of Calibration

...