Technical data

6 Specifications and Characteristics
90 U3402A User’s and Service Guide
DC Specifications
Table 6-1 DC resolution, full scale reading, and accuracy [±(% of reading + count)]
Function Rate Range Resolution
Maximum
reading
Accuracy
(One year; 23ºC
± 5ºC)
Typical input
impedance
[1]
Burden voltage
[4]
Voltage
Slow
120.000 mV
1.20000 V
12.0000 V
120.000 V
1000.00 V
0.001 mV
0.00001 V
0.0001 V
0.001 V
0.01 V
119.999
1.19999
11.9999
119.999
1000.00
[3]
±0.012% + 8
[2]
±0.012% + 5
±0.012% + 5
±0.012% + 5
±0.012% + 5
10.0 MW
10.0 MW
11.1 MW
10.1 MW
10.0 MW
—
Med
400.00 mV
4.0000 V
40.000 V
400.00 V
1000.0 V
0.01 mV
0.0001 V
0.001 V
0.01 V
0.1 V
399.99
3.9999
39.999
399.99
1000.0
[3]
±0.012% + 5
±0.012% + 5
±0.012% + 5
±0.012% + 5
±0.012% + 5
10.0 MW
11.1 MW
10.1 MW
10.0 MW
10.0 MW
—
Fast
400.0 mV
4.000 V
40.00 V
400.0 V
1000 V
0.1 mV
0.001 V
0.01 V
0.1 V
1 V
399.9
3.999
39.99
399.9
1000
[3]
±0.012% + 2
±0.012% + 2
±0.012% + 2
±0.012% + 2
±0.012% + 2
10.0 MW
11.1 MW
10.1 MW
10.0 MW
10.0 MW
—