Operator`s manual

Signametrics 60
6 Calibration
Each SMU2055 DMM uses its own SM60CAL.DAT calibration record to ensure the accuracy of its
functions and ranges. The SM60CAL.DAT file is a text file that contains the DMM identification
number, calibration date, and calibration constants for all DMM ranges. When the DMM is installed this
file is generated from an internally stored record. Once extracted, the DMM reads it from a file rather than
from its on-board record, since it is faster to read from a file. For most functions, the calibration constants
are scale factor and offset terms that solve an "y = mx + b" equation for each range. An input "x" is
corrected using a scale factor term "m" and an offset term "b"; this gives the desired DMM reading, "y".
Keep in mind that for ranges and functions that are unavailable for a particular product in the SM2060
family. The following calibration record is for the SMU2055 and it contains some placeholders for ranges
that are not available with the product. An example SM60CAL.DAT follows:
card_id 8123 type 2055 calibration_date 06/15/2008
ad #A/D compensation
2.0 10 0.99995
vdc #VDC 240mV, 2.4V,24V, 240V, 330V ranges, offset and gain parameters
-386.0 0.99961
-37.0 0.999991
-83.0 0.999795
-8.8 1.00015
0 1.0 ;Place holder
vac #VAC 1st line - DC offset. Than offset, gain and freq each range240mV to 330V
0 ;Place holder
0.84 1.015461 23
0.0043 1.0256 23
0.0 1.02205 0
0.0 1.031386 0
0 1.0 0 ;Place holder
idc # IDC 240nA to 2.5A, 8 ranges, offset and gain
0 1 ;Place holder
0 1 ;Place holder
0 1 ;Place holder
0 1 ;Place holder
-1450.0 1.00103 ;2.4mA range
-176.0 1.00602
-1450.0 1.00482
-176.0 1.00001 ;2.4A range
iac # IAC 2.4mA to 2.5A ranges, offset and gain
1.6 1.02402
0.0 1.03357
1.69 1.00513
0.0 1.0142
2w-ohm #Ohms 24, 240, 2.4k,24k,240k,2.4M,24M,240Meg ranges, offset and gain
0 1 ;Place holder
1256.0 1.002307 ;240 Ohms
110.0 1.002665
0.0 1.006304
0.0 1.003066
0.0 1.001848
0.0 0.995664 ;24 MOhms
0 1 ;Place holder
…
The first column under any function, e.g.,"vdc", is the offset term "b", expressed as a value proportional
to analog-to-digital (a/d) counts. The second column is the scale factor term "m". Within each function,
the "b" and "m" terms are listed with the lowest range at the beginning. For example, under "2
w-ohm"
above, "
1.27e+4 1.002259" represents the offset term for the 24  range, and "1.002259" is the scale
factor for this range.
For the ACV function, the first line in the calibration record is the DC offset value. The rest of the lines
contain the RMS offset, gain correction factor, and a third column that represents a digital code from 0 to
31 that controls the high frequency performance of each AC function. A large value, e.g., 31, implies
high attenuation.