last day (15 days later) » 

HRJ
5:32 PM
If you join Jeff, type a message. I will probably get notified (I am new to this).
 
5:52 PM
I have never used chat... Are you there?
 
HRJ
yes, but got busy with something.
resume after 20 to 30 mins?
 
Sure
 
 
1 hour later…
HRJ
7:23 PM
Hi, are you there? Sorry, had to urgently upgrade a server.
The problem seemed interesting, but I am not that good at math.
When we multiply or divide by PI to scale the angle, we would have to use a higher precision variable, and then convert the value to single precision. That should take care of my recent question.
So back to my analysis: do you still think it is wrong?
 
7:42 PM
Yeah, it seems like it should be 2^-23 * PI * R
But really, 2^-23 should be 2^-24
All of the analysis assumes the math is being done in higher precision, probably double
Otherwise, nothing is gained via conversion
 
HRJ
I am not sure. Let's say I have a value f store in single precision, in the range [0, +/- 1]
 
ok
So you mean the range [-1, 1]?
 
HRJ
I will scale it to radians by multiplying with PI
 
ok
So it is a longitude
 
HRJ
I now effectively have an angle stored in radians, but with higher precision.
(for simplicity I am using the same scale for both longitude and latitude)
 
7:48 PM
Yes, assuming the multiplication by pi is using double-precision
 
HRJ
Yes, exactly. The multiplication is in double-precision, but I can store the scaled value in single-precision.
 
For lat, you would scale by PI / 2.
Yes, all of that is correct.
If storing in single-precision, the best you can do is using normalized values.
 
HRJ
Ok, so do you agree that my analysis is correct?
 
If you are only concerned with longitudes in one part of the world, you can do better.
*normalized values means: in the range (-1, 1]
Your analysis in this chat, or in your answer?
 
HRJ
Both are the same :)
 
7:51 PM
Not really, in your answer, you missed the step "I will scale it to radians by multiplying with PI"
In this chat window, you mentioned that step
 
HRJ
Well, the scaling is implied when you want the answer in radians..
 
Then the only other issue is the power of 2 being used to perform the calculation.
 
HRJ
if you want it in degrees you would use a different scaling factor.
 
In your answer, you have the equation: 2^-23 * 6,378,137 m = 0.7603 m (76 cm)
2^-23 is NOT in radians, it is in half revolutions.
 
HRJ
Hmm, yeah, my explanation in the answer is rather crude. Touche.
I can't express Math that well, formally.
 
7:55 PM
The equation should be, 2^-23 half-revs * PI rad / half-rev * R instead.
 
HRJ
What should be on the RHS? :)
 
The only other issue I have with it is the exponent, which I believe should be -24 instead of -23
 
HRJ
Ah yes, I agree, and -24 would give an even better precision...
but what's on the RHS?
 
2^-23 * PI * 6378137 = 2.3887m
I also updated my answer with analysis with normalized numbers.
This line appears there: 2^-24 * PI rad * 6,378,137 m = 1.1943 m (3.9184 ft)
I didn't think the difference between normalized and radians would be so large.
 
HRJ
Ok, so according to you my equation is wrong, not just the way I expressed it.
 
7:59 PM
You had a great idea there, storing in the range (-1, 1].
 
HRJ
(leaving aside the -24 thing)
yup, but I am still not convinced it is wrong. Will chew on it further.
 
Basically, what we need to do is calculate the distance between consecutive points.
The full ircumference is 2 * PI * R, do you agree?
 
HRJ
yup
and we can make 2^24 slices of it.
 
The fraction that we care about, with θ in the range (-1, 1] is θ / (1 - -1) = θ / 2. Do you agree?
 
HRJ
uh, I didn't follow the last message, but now I agree with your equation :)
 
8:04 PM
Ok, so the formula then becomes 2 * PI * R * θ / 2 = PI * R * θ.
Then the question becomes, what is the minimum distance between two consecutive measurements in the worst case.
Let's assume we have a floating-point format that is very similar to single, but only gives us 3 bits in the mantissa.
 
HRJ
Hey!
I appreciate your explanation.. but I gotta go.
It's way past bed time...
 
Ok, let me know if you have any other questions for me.
 
HRJ
Sure
It was great talking to you.
Bye!
 

last day (15 days later) »