Previous: , Up: FP Math Caution   [Contents][Index]


16.4.5 Setting the Rounding Mode

The ROUNDMODE variable provides program-level control over the rounding mode. The correspondence between ROUNDMODE and the IEEE rounding modes is shown in Table 16.5.

Rounding modeIEEE nameROUNDMODE
Round to nearest, ties to evenroundTiesToEven"N" or "n"
Round toward positive infinityroundTowardPositive"U" or "u"
Round toward negative infinityroundTowardNegative"D" or "d"
Round toward zeroroundTowardZero"Z" or "z"
Round away from zero"A" or "a"

Table 16.5: gawk rounding modes

ROUNDMODE has the default value "N", which selects the IEEE 754 rounding mode roundTiesToEven. In Table 16.5, the value "A" selects rounding away from zero. This is only available if your version of the MPFR library supports it; otherwise, setting ROUNDMODE to "A" has no effect.

The default mode roundTiesToEven is the most preferred, but the least intuitive. This method does the obvious thing for most values, by rounding them up or down to the nearest digit. For example, rounding 1.132 to two digits yields 1.13, and rounding 1.157 yields 1.16.

However, when it comes to rounding a value that is exactly halfway between, things do not work the way you probably learned in school. In this case, the number is rounded to the nearest even digit. So rounding 0.125 to two digits rounds down to 0.12, but rounding 0.6875 to three digits rounds up to 0.688. You probably have already encountered this rounding mode when using printf to format floating-point numbers. For example:

BEGIN {
    x = -4.5
    for (i = 1; i < 10; i++) {
        x += 1.0
        printf("%4.1f => %2.0f\n", x, x)
    }
}

produces the following output when run on the author’s system:101

-3.5 => -4
-2.5 => -2
-1.5 => -2
-0.5 => 0
 0.5 => 0
 1.5 => 2
 2.5 => 2
 3.5 => 4
 4.5 => 4

The theory behind roundTiesToEven is that it more or less evenly distributes upward and downward rounds of exact halves, which might cause any accumulating round-off error to cancel itself out. This is the default rounding mode for IEEE 754 computing functions and operators.

Rounding Modes and Conversion

It’s important to understand that, along with CONVFMT and OFMT, the rounding mode affects how numbers are converted to strings. For example, consider the following program:

BEGIN {
    pi = 3.1416
    OFMT = "%.f"        # Print value as integer
    print pi            # ROUNDMODE = "N" by default.
    ROUNDMODE = "U"     # Now change ROUNDMODE
    print pi
}

Running this program produces this output:

$ gawk -M -f roundmode.awk
-| 3
-| 4

The other rounding modes are rarely used. Rounding toward positive infinity (roundTowardPositive) and toward negative infinity (roundTowardNegative) are often used to implement interval arithmetic, where you adjust the rounding mode to calculate upper and lower bounds for the range of output. The roundTowardZero mode can be used for converting floating-point numbers to integers. When rounding away from zero, the nearest number with magnitude greater than or equal to the value is selected.

Some numerical analysts will tell you that your choice of rounding style has tremendous impact on the final outcome, and advise you to wait until final output for any rounding. Instead, you can often avoid round-off error problems by setting the precision initially to some value sufficiently larger than the final desired precision, so that the accumulation of round-off error does not influence the outcome. If you suspect that results from your computation are sensitive to accumulation of round-off error, look for a significant difference in output when you change the rounding mode to be sure.


Footnotes

(101)

It is possible for the output to be completely different if the C library in your system does not use the IEEE 754 even-rounding rule to round halfway cases for printf.


Previous: , Up: FP Math Caution   [Contents][Index]