Owner manual
Publication 1762-UM003A-EN-P - February 2003
Module Data, Status, and Channel Configuration 3-21
Table 3.9 Effective Resolution and Range for 10 Hz Filter Frequency
Input
Type
Raw/Proportional Data
Over Full Input Range
Engineering Units x 1
Over Full Range
Engineering Units x 10
Over Full Range
Scaled for PID Over Full
Range
Percent of Full Scale
0 to 100%
Decimal
Range
Resolution
Decimal
Range
Resolution
Decimal
Range
Resolution
Decimal
Range
Resolution
Decimal
Range
Resolution
°C °F °C °F °C °F °C °F °C °F
100Ω
Pt 385
±32767
0.112°C/
7 counts
0.202°F/
7 counts
-2000
to
+8500
0.200°C/
2 counts
0.360°F/
2 counts
-200
to
+850
1.0°C/ 1
count
1.8°F/
1 count
0 to 16383
0.128°C/
2 counts
0.231°F/
2 counts
0 to 10000
0.210°C/ 2
counts
0.378°F/
2 counts
200Ω
Pt 385
0.112°C/
7 counts
0.202°F/
7 counts
-2000
to
+8500
0.200°C/
2 counts
0.360°F/
2 counts
-200
to
+850
1.0°C/ 1
count
1.8°F/
1 count
0.128°C/
2 counts
0.231°F/
2 counts
0.210°C/ 2
counts
0.378°F/
2 counts
500Ω
Pt 385
0.096°C/
6 counts
0.173°F/
6 counts
-2000
to
+8500
0.100°C/
1 count
0.180°F/
1 count
-200
to
+850
1.0°C/ 1
count
1.8°F/
1 count
0.128°C/
2 counts
0.231°F/
2 counts
0.105°C/ 1
count
0.189°F/
1 count
1000Ω
Pt 385
0.096°C/
6 counts
0.173°F/
6 counts
-2000
to
+8500
0.100°C/
1 count
0.180°F/
1 count
-200
to
+850
1.0°C/ 1
count
1.8°F/
1 count
0.128°C/
2 counts
0.231°F/
2 counts
0.105°C/ 1
count
0.189°F/
1 count
100Ω
Pt
3916
0.114°C/
9 counts
0.205°F/
9 counts
-2000
to
+6300
0.200°C/
2 counts
0.360°F/
2 counts
-200
to
+630
1.0°C/ 1
count
1.8°F/
1 count
0.152°C/
3 counts
0.274°F/
3 counts
0.166°C/ 2
counts
0.299°F/
2 counts
200Ω
Pt
3916
0.063°C/
5 counts
0.114°F/
5 counts
-2000
to
+6300
0.100°C/
1 count
0.180°F/
1 count
-200
to
+630
1.0°C/ 1
count
1.8°F/
1 count
0.101°C/
2 counts
0.182°F/
2 counts
0.083°C/ 1
count
0.149°F/
1 count
500Ω
Pt
3916
0.101°C/
8 counts
0.182°F/
8 counts
-2000
to
+6300
0.100°C/
1 count
0.180°F/
1 count
-200
to
+630
1.0°C/ 1
count
1.8°F/
1 count
0.101°C/
2 counts
0.182°F/
2 counts
0.166°C/ 2
counts
0.299°F/
2 counts
1000Ω
Pt
3916
0.101°C/
8 counts
0.182°F/
8 counts
-2000
to
+6300
0.100°C/
1 count
0.180°F/
1 count
-200
to
+630
1.0°C/ 1
count
1.8°F/
1 count
0.101°C/
2 counts
0.182°F/
2 counts
0.166°C/ 2
counts
0.299°F/
2 counts
10Ω
Cu 426
7.690°C/
1400
counts
13.843°F/
1400
counts
-1000
to
+2600
7.7°C/ 77
counts
13.8°F/
77 counts
-100
to
+260
8.0°C/ 8
counts
14.4°F/
8 counts
7.690°C/
350
counts
13.843°F/
350 counts
7.703°C/
214
counts
13.866°F/
214
counts
120Ω
Ni 618
0.055°C/
10 counts
0.099°F/
10 counts
-1000
to
+2600
0.100°C/
1 count
0.180°F/
1 count
-100
to
+260
1.0°C/ 1
count
1.8°F/
1 count
0.066°C/
3 counts
0.119°F/
3 counts
0.072°C/ 2
counts
0.130°F/
2 counts
120Ω
Ni 672
0.042°C/
8 counts
0.075°F/
8 counts
-800 to
+2600
0.100°C/
1 count
0.180°F/
1 count
-80 to
+260
1.0°C/ 1
count
1.8°F/
1 count
0.042°C/
2 counts
0.075°F/
2 counts
0.068°C/ 2
counts
0.122°F/
2 counts
604Ω
NiFe
518
0.060°C/
11 counts
0.091°F/
11 counts
-1000
to
+2000
0.120°C/
1 count
0.180°F/
1 count
-100
to
+200
1.0°C/ 1
count
1.8°F/
1 count
0.066°C/
3 counts
0.099°F/
3 counts
0.072°C/ 2
counts
0.108°F/
2 counts
150Ω 0.039Ω / 17 counts 0 to
+15000
0.040Ω / 4 counts 0 to
+1500
0.100Ω / 1 count 0.046Ω / 5 counts 0.045Ω / 3 counts
500Ω 0.046Ω / 6 counts 0 to
+5000
0.100Ω / 1 count 0 to
+500
1.00Ω / 1 count 0.061Ω / 2 counts 0.050Ω / 1 count
1000Ω 0.092Ω / 6 counts 0 to
+10000
0.100Ω / 1 count 0 to
+1000
1.00Ω / 1 count 0.122Ω / 2 counts 0.100Ω / 1 count
3000Ω 0.320Ω / 7 counts 0 to
+30000
0.400Ω / 4 counts 0 to
+3000
1.00Ω / 1 count 0.366Ω / 2 counts 0.600Ω / 2 counts