The system is divided into four different subsystems according to the functional architecture: Robot simulation, Geometry calibration, sensor noise calibration and color calibration.
Generally speaking, when we push the start button, the whole system will begin by checking the safety situation and present accuracy, if the system needs calibration, we will carry out the four steps of calibration procedure successively.
1. Model simulation
In the model simulation, we are working on two model, Robot model and path optimization.
1.1 Robot model simulation
In the Robot model simulation, we would simulate the working environment of the robot and the robot movement. The most important concern here is using simulation system to test the different robot movement and make sure it was safe and valid. It could save us tons of time to work on the real robot.
1.2 Path Planning
In the Path Planning, we tried to simulate the view of the field of the hundred of cameras in the environment and based on the requirement of geometry calibration to optimize the path. Generally speaking, we wanted to generate the path with high coverage for each camera and reduce the duplicate pose to make the path become more efficient.
2.Geometric calibration subsystem
The task of geometric calibration can be divided into 3 parts: the manipulation and control part, the image capturing part and the data processing and calibration part.
2.1 Manipulation and control
From the access terminal, we give instructions to control the ABB Robotic Arm which has the calibration target and sensors on it. The ABB robotic arm will then move the calibration target in a certain pattern according to the motion planning algorithm, and the sensors on it will collect position data. The system will send back the encoded positional data back to the computer and used for the following calibration procedure.
2.2 Image capturing
Also from the access terminal, we give instructions to the triggering mechanism. Under that mechanism, the 12 MP @ 73 fps canon cameras will be triggered to capture images. For the discrete trigger, the ABB robotic arm will move in a discrete pattern with pause, and the cameras will be triggered to capture images during the pause. Then this will be comparatively slower, but we will be able to get accurate images for the calibration process. When finishing capturing, all the images will be sent back to the computer and used for the following calibration procedure.
2.3 Data processing and calibration
During this part, we will use the captured image data and the position data of the calibration target to do the calibration calculation. We will call the geometric calibration algorithm to do the calibration work for multi cameras, and then the parameters of cameras and the positional and rotational data will be stored in a certain unit.
3.Sensor noise Calibration
In the sensor noise calibration, we would focus on four type of noise: shot noise, read noise, pattern noise, and thermal noise. In order to minimize these four types of noise influence in the camera for better camera calibration, we would design a noise collection and remove process. With that process, we could systematically record the noise parameter for each camera and conducted the calibration to remove them.
4.Color calibration
Color calibration is to measure and adjust the color response of a device (input or output) to a known state. We used an X-Rite ColorChecker Classic Card as our ground truths. We would try to find the fittest model that could best convert the image color to ground truth color.
5. Virtual Image Generation
This is a virtual image data set generation system to aide in validation of the geometric calibration algorithm for a 3D calibration target. The pipeline loads in camera data (intrinsic & extrinsic), object data (calibration target), required configuration and the number of the cameras and the render settings. It spews out Images and other mesh data using the cycles render engine and bpy module.