Corruption when incrementing float.

Former Member over 8 years ago

Here is output from incrementing a float by 0.1.

2.200000
2.300000
2.400000
2.500000
2.600000
2.700000
2.799999
2.899999
2.999999
3.099999

Im at a loss. My current fix is to use slope = slope.format("%0.1f").toFloat();

Former Member over 8 years ago

I think what you're experiencing are some rounding issues inherently associated with floating point arithmetic. Here's a link that will do a better job than I ever could explaining some of the difficulties: https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

travis.vitek over 8 years ago

Yes. This is exactly the problem. You should be careful about accumulating values into a float as this introduces error over time. Here is a tiny bit of C code based on your scenario that shows this error...

#include <stdio.h>

int main()
{
    float f = 2.2f;
    float g = 2.2f;

    for (int i = 0; i < 10; ++i) {
        fprintf (stdout, "f=%f g=%f\n", f, g + (i * 0.1f));

        f += 0.1f;
    }

    return 0;
}

Here is the output...

<Mon 06/27/2016 14:19:01.30>
C:\test>cl t.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 18.00.40629 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

t.cpp
Microsoft (R) Incremental Linker Version 12.00.40629.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:t.exe
t.obj


<Mon 06/27/2016 14:19:25.40>
C:\test>t
f=2.200000 g=2.200000
f=2.300000 g=2.300000
f=2.400000 g=2.400000
f=2.500000 g=2.500000
f=2.600000 g=2.600000
f=2.700000 g=2.700000
f=2.799999 g=2.800000
f=2.899999 g=2.900000
f=2.999999 g=3.000000
f=3.099999 g=3.100000

Former Member over 8 years ago

Ok. That's news to me. Thanks guys.

Interestingly had initially tried not to do floating point arithmetic. I tried to take the value, multiply by 10, and then work with it. I failed to see exactly what was happening at the start, and couldn't understand why I could not increment beyond 2.1. I have since noticed that this was because the multiplication was turning 2.2 into 21.999998. It was then rounding down when I turned this into a whole number. My main point in this exercise has been to take a users input and round it to 1 decimal place. I will probably just switch to the public beta and use the round() function from this point on. I do realise a simple hack for rounding is to add 0.5 and convert toNumber(). I expect CIQ 2.1.0 will be well out of public beta by the time my app is ready.

jim_m_58 over 8 years ago

Shark - I might be speaking out the blowhole in my head (you can call me "friendly_dolphin"!)

But what if you use doubles instead of floats? More memory, but more precision, IIRC...

Former Member over 8 years ago

Shark - I might be speaking out the blowhole in my head (you can call me "friendly_dolphin"!)

But what if you use doubles instead of floats? More memory, but more precision, IIRC...

My use case jim is the user must enter a value between 0 - 25.5 it is a 1 byte field that gets converted to range 0 -255. But I want to store it as the true value, which is the 0.1 precision < 25.5. If a double would eliminate the increment error then no reason why I couldn't I suppose.