Datasheet

Section 20 A/D Converter
Rev. 4.00 Sep 27, 2006 page 632 of 1130
REJ09B0327-0400
AN0 to AN7
10 kΩ
20 pF
To A/D converter
Note: Numeric values are reference values.
Figure 20.8 Analog Input Pin Equivalent Circuit
A/D Conversion Precision Definitions
The A/D conversion precision in this LSI is defined as follows.
• Resolution
The number of A/D converter digital output codes
• Offset error
The deviation of the analog input voltage value from the ideal A/D conversion characteristic
when the digital output changes from the minimum voltage value B'0000000000 (H'000) to
B'0000000001 (H'001) (see figure 20.10).
• Full-scale error
The deviation of the analog input voltage value from the ideal A/D conversion characteristic
when the digital output changes from B'1111111110 (H'3FE) to B'111111111 (H'3FF) (see
figure 20.11).
• Quantization error
The deviation inherent in the A/D converter, given by 1/2 LSB (see figure 20.9).
• Nonlinearity error
The error with respect to the ideal A/D conversion characteristic between the zero voltage and
the full-scale voltage. Does not include the offset error, full-scale error, or quantization error.
• Absolute precision
The deviation between the digital value and the analog input value. Includes the offset error,
full-scale error, quantization error, and nonlinearity error.