My answer to Tom applies here, but I thought of some more to say.
Nulls are just a subclass of a general need for attaching meaning to
specific values in tables.
Tables will often have a letter or number in a column and then a
footnote that gives the meaning of that letter. A simple case is -5 =
E0, -4=E1, 1=Sa. This type of information is more than just common, it
is unusuall to have a ADC/CDS catalog that does not do this. Sometimes
a column will act like a mask on another column, as in a ":" means
"highly uncertain" and "~" means approximate. So even to
understand a mask one needs a mechanism to understand the values in data.
Ed
Doug Tody wrote:
> On Thu, 22 Jun 2006, Ed Shaya wrote:
>
>> Personally, I think it would be good idea, when designing a new XML
>> format, to allow the mask and the data be combined by allowing
>> special numbers assigned to the different Nulls. Sometimes one wants
>> both a mask value and a data value for the same pixel, but that is
>> rare enough that one should simply holds those as triplets
>> (i,j,maskValue). If there is still a need for a quality mask, then
>> this would not exclude it.
>
>
> On the contrary, one generally wants to preserve the questionable
> data hence the mask is a separate object which is associated with
> the data array. As others have pointed out, data quality masks
> allow information to be encoded about what is wrong with a data
> sample, plus since they are sparse they are highly compressible.
> There are also efficiency advantages for processing, as storing the
> mask separately means that the data array can be processed without
> having to check every pixel/sample explicitly for magic values.
>
> In simple transport applications with FITS we may use a magic value
> to avoid the issues of interoperable masks, but this is a compromise.
> Actual data processing systems generally use a mask.
>
> I have never found actual IEEE NaN to be very useful. One has to
> be very careful with NaN as naive code which does not check for NaN
> is prone to going into an infinite loop. NaN can make a reasonable
> magic value for interchange of data, but to avoid runtime problems a
> system might remove the NaN values upon input, and use them to make
> a mask instead.
>
> While this is an interesting discussion it probably doesn't have
> a whole lot to do with VOTable. NaN is useful as a magic value in
> VOTable as in FITS to provide a simple mask capability, but ultimately
> we will probably want to define a standard interchangeable mask
> format instead (some on this list will recall there have already
> been previous attempts to do this).
>
> - Doug
Received on 2006-06-23Z20:17:47