Linux-kernel mailing list archive 1998-31,: Re: floating-point abuse in 2.1.113

# Re: floating-point abuse in 2.1.113

Mikael Pettersson (mikpe@csd.uu.se)
Tue, 4 Aug 1998 14:41:35 +0200 (MET DST)

- Messages sorted by: [ date ][ thread ][ subject ][ author ]
- Next message: Jamie Lokier: "Re: !!Warning!! UDMA and Fujitsu HD"
- Previous message: Karsten Petersen: "Re: VBE 2.0 (was Re: FB and MTRR)"
- Maybe in reply to: Mikael Pettersson: "floating-point abuse in 2.1.113"
- Next in thread: Rogier Wolff: "Re: floating-point abuse in 2.1.113"
- Next in thread: Stefan Traby: "Re: floating-point abuse in 2.1.113"
- Reply: Rogier Wolff: "Re: floating-point abuse in 2.1.113"

On Tue, 4 Aug 1998, Geert Uytterhoeven wrote:

> On Tue, 4 Aug 1998, Mikael Pettersson wrote:
> > - frequency = (frequency / 16.0) * 100; /* massage data a bit */
> > + frequency = (frequency * 100) / 16; /* massage data a bit */
>
> Shouldn't these be rounded, i.e.
>
> frequency = (frequency * 100 + 8) / 16;

No, there is no rounding in the original calculation:
C's floating-point-to-integer conversion is truncating.
The only concern is that the integer version might overflow while
calculating (frequency * 100). If this is a real problem, note
that 100.0/16.0 == 6.25, and reformulate as:
frequency = frequency*6 + frequency/4; /* multiply by 6.25 */

/Mikael

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html

- Next message: Jamie Lokier: "Re: !!Warning!! UDMA and Fujitsu HD"
- Previous message: Karsten Petersen: "Re: VBE 2.0 (was Re: FB and MTRR)"
- Maybe in reply to: Mikael Pettersson: "floating-point abuse in 2.1.113"
- Next in thread: Rogier Wolff: "Re: floating-point abuse in 2.1.113"
- Next in thread: Stefan Traby: "Re: floating-point abuse in 2.1.113"
- Reply: Rogier Wolff: "Re: floating-point abuse in 2.1.113"