Floats

Is there a description somewhere of the limits on floating points numbers? I've noticed that if I do a#=500000, then a#=a#+0.1, I still get 500000.

When I was writing a FloatFraction function I noticed that Floats are limited to a max of 6 meaningful places, sometimes but rarely you'll get 7 because of rounding up/down. I used to think because of the way floats are calculated, with exponents and all that, you could have for example, 500000.1 but you can't really. One other interesting thing about floats I found was that after the last digit printed it's not just blank/0's, there are values there but they're ignored, I noticed it when I was rounding up/down floats, for example writing 500000.1 and 500000.6 give different values even though that last digit is not printed.

There's a double.dll in the code archives if you want greater accuracy.

It's not so much the size of the float but the precision that is the problem.

32767.123 is the highest possible number in blitz, -32767.123 is the lowest.

hoped this helped.

Not really, what with it being wrong and all...

In my modest opinion if by example, floats are stored in IEEE 754 simple presition 32 bits are used with 24 (23 stored bits) significant bits (abour 6.5 decimal digits more than 6, less than 7 so an so) and an exponent in 10 base about +/-38. In few words, 6 significant digits in base 10 are secure. I don't know wich floating point format uses blitz 3d so, we are expeculating.

best regards

Juan

You only get a certain precision. I think it's 6 numbers. However, the highest and lowest values are the same as an integers i'm sure.

I were using a bank to store a float and then peek byte to analise one by one and I'm 99% sure that the format is IEEE 754 simple presition (a standard used in the floating point unit fpu inside every intel and clones cpu's).
the format is
implicit base 2
1 bit (first for sign)
8 bits for exponent (in excess 127) so the range is +/- 127
23 bits for the significand


the numbers are normalized: 1,Significand so always are a 1 that is not stored so the format store 23 bits but has 24 bits of precision.

the inverse of 2 exp 23 is: 0,00000011920928955078125 that's the granularity, the minimal ammount stored (with an exponent of zero).
that number has the first non zero decimal digit at the 7th position, so the firs 6 are secure.

with exponent's greater than 0, the absolute error is great
with exponent's less than 0, the absolute error is less
but the relative error always stay about 1/(2 exp 23): six decimal digits aproximately.

for the representation of 0, the format admits an exponent 0 (8 zeros in the exponent field), in this case, the normalization differ and is 0,significand. (just for complete the explanation)

Juan

Simple terms:

Print 50000.1

That is your range. Not the 50000 part, but the number of digits of precision. You get 6 digits. The leftmost digits take priority obviously.

Anything higher than 99999.9 and you will lose the floating point precision :o)