HP-UX Floating-Point Guide

Appendix C 221

Floating-Point Problem Checklist

Incorrect or Imprecise Results

If your application produces results that are either clearly incorrect or

much less precise than you expected, consider the following possibilities:

• Did your application encounter an exception condition, either in a

basic operation or in a math library function? Exceptions such as

overﬂows, or invalid arguments to library functions, can introduce a

dramatic amount of error into a computation. On HP-UX systems,

exceptions by default do not prevent an application from continuing;

you must detect exceptions explicitly.

For information on how exceptions can affect application results, see

“How Exceptions and Library Errors Affect Application Results” on

page 74. For information on math library error handling, see “Math

Library Basics” on page 101. For information on how to detect and

trap exceptions, see Chapter 6.

• Did you forget to include math.h in a C program that calls math

library functions? Or did you compile your C program with -Aa (strict

ANSI) but call a math function not speciﬁed by the ANSI C standard?

If so, the C compiler generates code that expects the default C

function return value of type int, and the function returns an

incorrect result.

For more information, see “The C Math Library (libm)” on page 114.

• Does your application convert values properly from decimal (ASCII)

to binary and vice versa? Check the points at which your program

assigns constant values to variables, reads ﬁle or keyboard input into

variables, and writes variables to ﬁles or to the screen. Make sure

that the format speciﬁcations in your program allow the right amount

of precision in the values being read and written.

For more information on conversions between decimal and binary, see

“Conversions Between Decimal and Binary Floating-Point Format” on

page 54 and “Conversions Between Binary and Decimal” on page 76.

• Does your program test ﬂoating-point values for equality? Because of

the inherent inexactness of ﬂoating-point representations, values

that should be equal from a purely algebraic perspective in fact rarely

are.