monkeyc floating point precision and trig funcs

Former Member
Former Member
I'm trying to determine the distance between 2 points using longitude and latitude as below

Distance = 6371000.0d * Math.acos(Math.sin(lLatA) * Math.sin(lLatB) + Math.cos(lLatA) * Math.cos(lLatB) * Math.cos(lLonB - lLonA));

with the following example data (already converted to radians)

lLatA = -0.554793
lLonA = 2.024043
lLatB = -0.554654
lLonB = 2.023912

The above are values printed using System.print in monkeyc.

I'm always getting 0.000000 but if I use another language (have tried a basic interpreter and Delphi) I get 1134.702958

The basic code is

>list
10 llata = -0.554793
20 llona = 2.024043
30 llatb = -0.554654
40 llonb = 2.023912
50 ans = 6371000*acos(sin(llata)*sin(llatb)+cos(llata)*cos(llatb)*cos(llona-llonb))
60 print "ans",ans
>run
ans 1134.702958
>

the monkeyc debug is (ccors are showing llata,llona,llatb,llonb in radians)

-0.554793,2.024043,-0.554654,2.023912
sin(llata) = -0.526768
sin(llatb) = -0.526649
cos(llata) = 0.850009
cos(llatb) = 0.850083
cos(llonb - llona)) = 1.000000
Total = 1.000000
acos = 0.000000
Dist = 0.000000

It looks to me like its not working with a high enough precision. Does the trig functions use single or double precision math ? The doco simply says

(Float) acos(x)

I'm suspecting it could be in single precision or is possibly losing some precision somewhere as I'm sure the basic and delphi tests I was using were both using single precision.

Maybe someone has a better working formula ?

Andrew