Technical Specs

– Visual odometry (rc
_
stereovisodo, Section 6.4) estimates the motion of the rc_visard device
based on the motion of characteristic visual features in the left camera images.
– Stereo INS (rc
_
stereo
_
ins, Section 6.5) combines visual odometry measurements with read-
ings from the on-board Inertial Measurement Unit (IMU) to provide accurate and high-
frequency state estimates in real time.
• Camera calibration (rc
_
cameracalib, Section 6.6) automatically checks and performs the self-
calibration of the rc_visard’s stereo camera in case it has been decalibrated. It furthermore enables
the user to check and perform recalibration manually via the WEB GUI (Section 4.5).
• Hand-eye calibration (rc
_
hand
_
eye
_
calibration, Section 6.7) enables the user to calibrate the
rc_visard with respect to a robot, either via the Web GUI or the REST-API.
6.1 Stereo camera
The stereo camera component contains functionality for acquiring stereo image pairs and performing planar recti-
fication needed to use the stereo camera as a measurement device.
6.1.1 Image acquisition
Acquiring stereo image pairs is the first step toward stereo vision. Since both cameras are equipped with global
shutters and their chips are hardware-synchronized, all pixels of both camera images are always exposed at the
exactly same time. GPIO out 1 (Section 3.5) signals the respective exposure time. Additionally, the time in the
middle of the image exposure is attached to the images as a timestamp. This timestamp becomes important for
dynamic applications in which the rc_visard or the scene moves.
Exposure time can be set manually to a fixed value. This is useful in an environment where lighting is controlled
so that it is always at the same intensity. The camera is set to auto exposure by default. In this mode, the rc_visard
chooses the exposure time automatically, up to a user defined maximum. The permitted maximum is meant to
limit the motion blur that occurs when taking images while the rc_visard or the scene is moving. The maximum
exposure time thus depends on the application. If the maximum exposure time is reached, the auto-exposure
algorithm uses the gain to increase image brightness. However, larger gain factors also amplify image noise.
Thus, the maximum exposure time trades motion blur off against image noise under weak-light conditions.
6.1.2 Planar rectification
Camera parameters such as focal length, lens distortion, and the relationship of the cameras to each other must be
exactly known to use the stereo camera as a measuring instrument. The parameters are determined by calibration
(see Camera calibration, Section 6.6). The rc_visard is already calibrated at production time and normally requires
no recalibration. The camera parameters describe with great precision all of the stereo-camera system’s geometric
properties, but the resulting model is complex and difficult to use.
Rectification is the process of remapping the images according to an ideal stereo-camera model. Lens distortion
is removed and the images are aligned so that an object point is always projected onto the same image row in
both images. The cameras’ optical axes become exactly parallel. This means that points at infinite distance are
projected onto the same image column in both images. The closer an object point is, the larger is the difference
between its image columns in the right and left images. This difference is called disparity.
Mathematically, the object point 𝑃 = (𝑃
𝑥
, 𝑃
𝑦
, 𝑃
𝑧
) is projected onto image point 𝑝
𝑙
= (𝑝
𝑙𝑥
, 𝑝
𝑙𝑦
, 1) in the left
rectified image and onto 𝑝
𝑟
= (𝑝
𝑟𝑥
, 𝑝
𝑟𝑦
, 1) in the right rectified image by
𝐴 =
⎛
⎝
𝑓 0
𝑤
2
0 𝑓
ℎ
2
0 0 1
⎞
⎠
, 𝑇
𝑠
=
⎛
⎝
𝑡
0
0
⎞
⎠
,
𝑠
1
𝑝
𝑙
= 𝐴𝑃,
𝑠
2
𝑝
𝑟
= 𝐴(𝑃 − 𝑇
𝑠
).
6.1. Stereo camera 28