Re: String representations of numeric values

From: Arnold Rots <arots-at-head.cfa.harvard.edu>
Date: Fri, 21 Apr 2006 15:28:44 -0400 (EDT)


Within XML schema there is a simple solution to this, too: use xs:decimal and employ as many digits as you need. If VOTable would support this, your problem would be solved.

Ed Shaya wrote:
> Francois Ochsenbein wrote:
> > Hi Ed,
> >
> > I don't understand your arithmetics -- double have 52 bits for the mantissa
> > (53 if you include the hidden bit), that means a dynamics of 2^53 or 10^16.
> >
> Just slightly less, with the result that one can only reliably convert
> to/from 15 digits.
> > If the largest angle is 360degrees, to get an accuracy of 1 nano-arcsec
> > you need only 51 bits only (360deg = 1.296E+15 nano-arcsec).
> >
> > 243 degrees plus 9.13 microarcsec, with an accuracy of 0.01micro-arcsec,
> > is just 243.00000000254 or 243.000000002536 -- the number 243.00000000025361
> > you quote is 243 + 0.9130 microarcsec, i.e. expressed with an accuracy of
> > 0.1nano-arcsec ...
> >
> >
> Well, yes. The 0 in 0.9130 is at .1 nanoarcsec precision. Why do I
> want this precision? It has to do with what I described in my last
> message about astrometric measurements. SIM is primarily an astrometric
> mission and one will want to quote each measurement with full fidelity
> as in 0.9130 +/- 0.0306 random +/- 0.835 systematic. One wants to do as
> little rounding as possible and when you do round one still needs to
> represent the intervals of rounding to high precision. Generally,
> precision should exceed accuracy by 3 or 4 digits to allow for accurate
> subtraction or trigonometric functions as Anita mentions. Also there
> are some results that are not as dependent on systematic errors such as
> understanding drift of components in the spacecraft.
>
> Ed
>
> >> As a representative of the SIM mission I should mention that SIM will be
> >> regularly be making measurements of positions with precision order 0.1
> >> microarcsec (although accuracy may be limited by fiducial grid accuracy).
> >> 1 microarcsec = 1E-6/3600 = 2.778E-10.
> >> So to express, say 243 degrees plus 9.13 microarcsec or
> >> 243.00000000025361 which has 17 digits, is not expressable in double
> >> precision (where the significand holds a bit over 15 digits).
> >> Therefore we need to introduce quad precision if VOTable is to be used
> >> in SIM and GAIA preparatory work.
> >>
> >> Although I would prefer to use hh mm ss.ssssssss since quad precision is
> >> way overkill.
> >>
> >> Ed
> >>
> >>
> >>
> >
> > ================================================================================
> > Francois Ochsenbein ------ Observatoire Astronomique de Strasbourg
> > 11, rue de l'Universite F-67000 STRASBOURG Phone: +33-(0)390 24 24 29
> > Email: francois-at-astro.u-strasbg.fr (France) Fax: +33-(0)390 24 24 32
> > ================================================================================
> >
>


Arnold H. Rots                                Chandra X-ray Science Center
Smithsonian Astrophysical Observatory                tel:  +1 617 496 7701
60 Garden Street, MS 67                              fax:  +1 617 495 7356
Cambridge, MA 02138                             arots-at-head.cfa.harvard.edu
USA                                     http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------
Received on 2006-04-21Z21:29:30