Precision problems ...

BlitzMax Forums/BlitzMax Programming/Precision problems ...

Hi,

I seem to be running into a weird issue, and i'm trying to figure out if it's my machine or BMAX. The following code prints out:

25.000000000000000
25.000000000000000
0.00000000000000000

which is incorrect. can anyone please run this and confirm if their numbers look ok? I seem to be having precision problems here :/

Local t1:Double = 25.0000002
Local t2:Double = 25.0000001
Local t3:Double
t3 = t1 - t2
	
Print t1
Print t2
Print t3


when I run :
Local t1:Double = 25.002
Local t2:Double = 25.001
Local t3:Double
t3 = t1 - t2
	
Print t1
Print t2
Print t3



i get :
25.002000808715820
25.000999450683594
0.0010013580322265625


almost looks like my CPU is having problems, can anyone confirm please?

thanks in advance

What's unusual about this? Even with double precision floating point numbers, not every number can be perfectly represented in 64 bits.

Whats unusual is that i dont know how to treat those numbers ...

Local t1:Double = 25.6000061
Local t2:Double = 25.6000004
Local t3:Double
t3 = t1 - t2

Print t1
Print t2
Print t3



results in :

25.600006103515625
25.600000381469727
5.7220458984375000e-006


5.7220458984375000e-006 isnt the result im looking for

Those are stunning results. (oh, I failed to notice the "e-006")

5.7220458984375000e-006 isnt the result im looking for

What result are you looking for?

that's 0.0000057220458984375 which is as close to zero as you need:
Local t1:Double = 25.6000061:Double 'numbers with decimals default to floats, force double to keep some precision
Local t2:Double = 25.6000004:Double
Local t3:Double
t3 = t1 - t2

Print t1
Print t2
If t3 < 0.00001 Then
	Print 0
Else 
	Print t3
EndIf


my problem comes when i try to compare some numbers, i'll clip them manually this time, thanks for the help.

As regards the intial post, the problem is that the constants are being defined as single precision floats. So add ! to make them double precision.

Edit. oops. Obviously I did not read the whole thread and missed Perturbatios response.