Hi Kona -
On Fri, 9 Mar 2007, Kona Andrews wrote:
>> If we describe the list of tables which a given service can access, the
>> response is a table wherein each row describes one such Table object.
>> The columns or fields of the table are the attributes used to describe
>> a Table object. These attributes can be anything we want, regardless
>> of how the information is conveyed or stored (this has nothing to do
>> with VOTable, which is just a container).
>>
>> If we describe the list of columns or fields in a Table object, the
>> response is a table wherein each row describes one such TableColumn
>> object. The columns of the returned table are the attributes used to
>> describe a data table TableColumn. Again, these can be anything we
>> want (probably some combination of SQL92 and VOTable PARAM/FIELD-like
>> attributes, but it is arbitrary).
>
> I think we are talking at different levels here; if the metadata
> is embedded in VOTable as you describe, then existing tooling
> automatically understands how to *syntactically parse* that document,
> but not necessarily how to *semantically parse* it. We would have to
> define a competing standard expressing the *meaning* of the data
> encoded in those rows (as you say, "these can be anything we want").
> The point about the VOResource description is that it already specifies
> both how to encode the metadata *and* how to interpret it.
Yes, all the VOTable tools get you is a generic way to extract information from the serialization. The data model used to describe a table or table column has to be defined by TAP in any case, and understood by applications, no matter what we do. As I said earlier, the work done on VOResource by the registry group is one thing we should be looking at (I'm not suggesting we ignore this), but we have other things such as SQL92 and the work done earlier on the Catalog data model to consider as well.
> I understand that it simplifies your own client coding to avoid the
> ingestion of VOResource (although XML libraries make ingestion of
> VOResource descriptions into an object model pretty trivial).
I'm not talking about my own client coding, rather I am trying to look at things from the point of view of a user writing an application. To query an arbitrary catalog they will need to first query the metadata, then query the table itself, and it would be simpler to use the same mechanism for both. For system code such as the query builder you describe (or my own VOClient code for example), this is less of a concern as the tool builders can probably deal with anything we give them.
> If we were to mandate that TAP services provide their metadata in
> both VOResource and (e.g.) VOTable representations, then your services
> would only have to *export* the VOResource descriptions and your clients
> would not need to ingest them. Producing such descriptions is generally
> just a matter of populating a template, so not a vastly complex programming
> task.
I agree that for a good service implementation it would be no problem to support multiple output formats. Basically one just computes the metadata and sets the fields of a data model, then this can be easily serialized to any supported output format.
If we should decide to provide a uniform interface for querying both table data and metadata, supporting multiple output formats is probably something we want to do anyway. In this case, VOTable, XML (probably based on VOResource or the Catalog data model), and CSV/TSV would be reasonable choices (maybe even FITS binary table). An advantage of using a uniform access mechanism is that one can afford to do this, since if one already does this for data access, it costs nothing more to do it for metadata queries.