Re: Spectral DM document update

From: Doug Tody <dtody-at-nrao.edu>
Date: Mon, 9 Oct 2006 09:09:49 -0600 (MDT)


Hi Roy -

In the VOEvent case the discussion was of coordinate vectors containing typically only a couple of points, which it could be useful to be able to address directly at the XML level. The case here is quite different as we are talking about vectors typically containing thousands of points. It is not useful to address individual elements of such large vectors at the XML level.

SSA uses FITS binary table as the FITS serialization. There is also a VOTable format. Both are vector-based, i.e., there is a "flux vector", "wavelength vector", etc. This is what most analysis applications want; one typically plots flux vs wavelength, does line width analysis on the flux vector, etc.

An additional issue with SSA and the Spectrum DM is that most vector valued elements can also be scalars, for example an error vector can also be a scalar value. This is easy to deal with if the data is decomposed into separate elements, which may be either scalar or vector valued. If instead one uses an array of point structures, either scalar values cannot be supported, or the structure has to be modified to reflect a change between a scalar or vector value.

On your point about experience with code generators etc., we also have the counter point from Lazslo and the JHU spectral services, which found that a simple array serialization was more convenient.

(None of this may matter in the end as most people will probably use VOTable and FITS for spectra, but nonetheless array handling in XML is an important issue to consider).

On Mon, 9 Oct 2006, Roy Williams wrote:

> On Oct 8, 2006, at 9:53 PM, Jonathan McDowell wrote:
>> arrays like <Flux>1.2 1.3 1.4 ... </Flux> where all data
>> point attributes are in separate arrays, instead of having
>> Point No. 1 (flux, err, spco, ...) then Point No. 2 (flux, err, spco...)
>> etc.
>
> Just as a matter of interest, you may recall a big fight between VOEvent and
> Space-Time Coordinates, about a year. The main sticking point was the use of
> arrays in this manner, which some parsers and code-binders would not accept.
> The resolution was that STC agreed to give up XML arrays and make the array
> elements into separate XML elements. Therefore I urge you not to use these
> XML arrays again. Or if you insist on doing so, please justify by doing the
> hard work in advance to decide if these are parsable easily by the people who
> would use them.
>
> On a larger point, let is not get caught in the trap of trying to make
> "readable XML". If the XML is too verbose, it is time to get computers to
> read and write it instead of using notepad and vi ;-) If there is a timing
> inefficiency in processing too much XML, then use a binary format. Remind me,
> is there a FITS serialization for the Spectrum Data Model?
>
> Roy
>
> California Institute of Technology
> 626 395 3670
>
Received on 2006-10-09Z17:10:31