> Ian McCrea from RAL brought up an interesting point at an AstroGrid solar /
> STP meeting yesterday; in some STP time series, the time intervals within a
> single time series might not all be the same length. (For example, point 0 at
> 0 seconds, point 1 at 10 seconds, point 2 at 47 seconds, etc). There's
> potentially a case for the same thing with SDO data in 3 years' time as one
> of the instruments can optionally take data at half cadence (but only half
> the CCD pixels are read), or observers can choose 8 out of 10 channels in any
> one observation cycle. Either case would lead to weird time series data for a
> specific channel on this SDO instrument.
>
The sampling on any axis can be irregular -
Spatial - adaptive smoothing
Spectral - an SED or even datacube built up from a series of observations in different filters/bands (e.g. quite common to produce a cube of radio continuum flux density at 1.4, 5, 8, 15 .. GHz in order to make a spectral index map, or of pol. angle at each freq to produce Faraday rotation maps...)
Time - as above, or the maser movie I pointed to and many other such observations (probably also Igor's example?) where data are taken whenever the telecope is available. These may be interpolated onto a regular grid before data release, or thy may not.
The Characterzation model tries to allow for all such eventualities and it would be useful to know if it is adequate for the sorts of Solar as well as astronomical 3D data under discussion.
This discussion did start as a more specific and immediately practical consideration of access to 3D/ND data, which raises an interesting question of how the detail is represented in the data itself. For example, in a radio data cube with spectral points at 1.4, 5, 8, 15 .. GHz, the frequency axis in the FITS header will probably be just a counter (planes 1, 2, 3, 4...) with the true frequencies hidden in the history, if anywhere. Does Doug or anyone know a better way actually in present use? Additional metdata is needed to make them accessible to a discovery tool, or to most processing tools (apart from custom tools which can dive into histories..). How are the other cases handled?
My feeling is that even if it is too difficult to use the DAL model to handle full details of irreguarly sampled data, the simplest solution - just to begin with - would be to supply the typical sampling interval on each axis, and/or the number of samples, with a flag for whether the sampling was regular or not (leaving the fuller levels of detail to Char). That would at least allow data discovery and the user could seek extra information manually if necessary (just as I often have to follow up references from Vizier to find the resolution of tabulated data, for example). That would require having a firm link to the provenance of any data.
best wishes
Anita