Operator`s manual

Signametrics 180
6.10 Calibration
Each SMU2060/64 DMM uses its own SM60CAL.DAT calibration record to ensure the accuracy of its
functions and ranges. The SM60CAL.DAT file is a text file that contains the DMM identification
number, calibration date, and calibration constants for all DMM ranges. When the DMM is installed this
file is generated from an internally stored record. Once extracted, the DMM reads it from a file rather than
from its on-board record, since it is faster to read from a file. For most functions, the calibration constants
are scale factor and offset terms that solve an "y = mx + b" equation for each range. An input "x" is
corrected using a scale factor term "m" and an offset term "b"; this gives the desired DMM reading, "y".
Keep in mind that for ranges and functions that are unavailable for a particular product in the SMU2060
family, the calibration record contains a placeholder. An example SM60CAL.DAT is shown:
card_id 10123 type 2064 calibration_date 06/15/1999
ad #A/D compensation
72.0 20.0 0.99995
vdc #VDC 240mV, 2.4V,24V, 240V, 330V ranges, offset and gain parameters
-386.0 0.99961
-37.0 0.999991
-83.0 0.999795
-8.8 1.00015
44.5 1.000001
vac #VAC 1st line - DC offset. Than offset, gain and freq each range240mV to 330V
5.303
0.84 1.015461 23
0.0043 1.0256 23
0.0 1.02205 0
0.0 1.031386 0
1.2 0.994999 2
idc # IDC 240nA to 2.5A, 8 ranges, offset and gain
-22.3 1.000030
33.4 0.999939
32.0 0.993499
-54.3 1.000102
-1450.0 1.00103
-176.0 1.00602
-1450.0 1.00482
-176.0 1.00001
iac # IAC 2.4mA to 2.5A ranges, offset and gain
1.6 1.02402
0.0 1.03357
1.69 1.00513
0.0 1.0142
2w-ohm #Ohms 24, 240, 2.4k,24k,240k,2.4M,24M,240Meg ranges, offset and gain
1.27e+4 1.002259
1256.0 1.002307
110.0 1.002665
0.0 1.006304
0.0 1.003066
0.0 1.001848
0.0 0.995664
0.0 1.00030
…
The first column under any function, e.g.,"vdc", is the offset term "b", expressed as a value proportional
to analog-to-digital (a/d) counts. The second column is the scale factor term "m". Within each function,
the "b" and "m" terms are listed with the lowest range at the beginning. For example, under "2
w-ohm"
above, "
1.27e+4 1.002259" represents the offset term for the 33  range, and "1.002259" is the scale
factor for this range. This record must be for the SMU2064 since the SMU2060 does not have the 33
Ohms range, and therefore these values will be set to 0.0 and 1.0.
For the ACV function, the first line in the calibration record is the DC offset value. The rest of the lines
contain the RMS offset, gain correction factor, and a third column that represents a digital code from 0 to