Technical Specs

For stereo matching, the position and orientation of the left and right cameras relative to each other has to be
known with very high accuracy. This is achieved by calibration. The rc_visard’s cameras are pre-calibrated
during production. However, if the rc_visard has been decalibrated, during transport for example, then the user
has to recalibrate the stereo camera.
The following rc_visard software components are required to compute 3D information:
• Stereo camera: This component is responsible for capturing synchronized stereo image pairs and transform-
ing them into images approaching those taken by an ideal stereo camera (rectification) (Section 6.1).
• Stereo matching: This component computes disparities for the rectified stereo camera pair using
SGM (Section 6.2).
• Camera calibration: This component enables the user to recalibrate the rc_visard’s stereo camera (Section
6.6).
5.2 Sensor dynamics
In addition to providing 3D information about the scene, the rc_visard can also estimate its egomotion or dy-
namic state in real time. This comprises its current pose, i.e., its position and orientation relative to a reference
coordinate system or reference frame, as well as its velocity and acceleration. Measurements from stereo visual
odometry (SVO) and the integrated Inertial Measurement Unit (IMU) are fused to compute this information. This
combination is called a Visual Inertial Navigation System (VINS).
Visual odometry observes the motion of characteristic points in the camera images to estimate the camera motion.
Object points are projected on different pixels in the camera image depending on the camera’s viewing position.
Each point’s 3D coordinates can also be computed using stereo matching between the point positions in the left
and right camera images. Thus, for two different viewing positions A and B, two sets of corresponding 3D points
are computed. Assuming a static environment, the motion that transforms one set of points into the other is the
camera’s motion. The principle is illustrated for a simplified 2D case in Fig. 5.2.1.
View A
View B
Pose A
Pose B
Observed motion
3D positions
view A
3D positions
view B
Computed camera
motion
Fig. 5.2.1: Simplified sketch of the stereo visual odometry principle for 2D motions: Camera motion is computed
from the observed motion of characteristic image points.
Since visual odometry relies on image-data quality, motion estimates deteriorate when the images are blurred or
are poorly illuminated. Furthermore, visual odometry’s frame rate is too low for control applications. That’s
why the rc_visard has an integrated Inertial Measurement Unit (IMU), which measures the accelerations and
angular velocities that occur when the rc_visard moves. It also measures acceleration due to gravity, which
5.2. Sensor dynamics 25