Re: String representations of numeric values

From: Ed Shaya <eshaya-at-umd.edu>
Date: Fri, 21 Apr 2006 13:54:04 -0400


Francois Ochsenbein wrote:
> Hi Ed,
>
> I don't understand your arithmetics -- double have 52 bits for the mantissa
> (53 if you include the hidden bit), that means a dynamics of 2^53 or 10^16.
>

Just slightly less, with the result that one can only reliably convert to/from 15 digits.
> If the largest angle is 360degrees, to get an accuracy of 1 nano-arcsec
> you need only 51 bits only (360deg = 1.296E+15 nano-arcsec).
>
> 243 degrees plus 9.13 microarcsec, with an accuracy of 0.01micro-arcsec,
> is just 243.00000000254 or 243.000000002536 -- the number 243.00000000025361
> you quote is 243 + 0.9130 microarcsec, i.e. expressed with an accuracy of
> 0.1nano-arcsec ...
>
>

Well, yes. The 0 in 0.9130 is at .1 nanoarcsec precision. Why do I want this precision? It has to do with what I described in my last message about astrometric measurements. SIM is primarily an astrometric mission and one will want to quote each measurement with full fidelity as in 0.9130 +/- 0.0306 random +/- 0.835 systematic. One wants to do as little rounding as possible and when you do round one still needs to represent the intervals of rounding to high precision. Generally, precision should exceed accuracy by 3 or 4 digits to allow for accurate subtraction or trigonometric functions as Anita mentions. Also there are some results that are not as dependent on systematic errors such as understanding drift of components in the spacecraft.

Ed  

>> As a representative of the SIM mission I should mention that SIM will be
>> regularly be making measurements of positions with precision order 0.1
>> microarcsec (although accuracy may be limited by fiducial grid accuracy).
>> 1 microarcsec = 1E-6/3600 = 2.778E-10.
>> So to express, say 243 degrees plus 9.13 microarcsec or
>> 243.00000000025361 which has 17 digits, is not expressable in double
>> precision (where the significand holds a bit over 15 digits).
>> Therefore we need to introduce quad precision if VOTable is to be used
>> in SIM and GAIA preparatory work.
>>
>> Although I would prefer to use hh mm ss.ssssssss since quad precision is
>> way overkill.
>>
>> Ed
>>
>>
>>
>
> ================================================================================
> Francois Ochsenbein ------ Observatoire Astronomique de Strasbourg
> 11, rue de l'Universite F-67000 STRASBOURG Phone: +33-(0)390 24 24 29
> Email: francois-at-astro.u-strasbg.fr (France) Fax: +33-(0)390 24 24 32
> ================================================================================
>
Received on 2006-04-21Z19:54:48