Community feedback re TAP metadata

From: Kona Andrews <kea-at-roe.ac.uk>
Date: Wed, 7 Mar 2007 11:20:43 +0000

Dear all,

I have consulted with the community here regarding our TAP metadata query proposals, based on the draft document Francois has kindly prepared at http://vizier.u-strasbg.fr/~francois/ivoa/TAP-metadata.htx.

I received quite a number of responses, which I shall summarize here as key points.

---

Key point 1: VOTable may not be not adequate/easily extensible

---

There are concerns about whether VOTable is adequate as a metadata description format, now and in the future. One correspondent writes:

• 8< ----
  1. [The question about whether VOTable is the best format to use for returning service metadata in TAP] is a good question to ask; there seems to be a bit of a tendency in the IVOA to use VOTable for everything and ask questions later, so some consideration of whether it's a good match to the problem is sensible (though I'm not saying that it's not a good match).
  2. Sec. 4 of [the draft] already comments that table/key relations can't be encoded in VOTable, which sounds like a limitation. Actually it probably is possible to encode this information by ingenious use of GROUPs, but that may not be very elegant (the intention of such documents will probably only be clear with reference to some additional documentation). It further comments "should such a description be added to the VOTable schema?". I'd oppose such ad-hoccery in VOTable on technical grounds, and also note that the record of [external working groups] getting changes accepted to VOTable is not encouraging.
• 8< ----

VOTable is not intended/optimised for expressing service metadata, and may be overly restrictive. Additionally, should there be a future requirement for even more expressive service metadata, it may not be possible to extend VOTable to cover this.

---

Key point 2: The IVOA already has a draft standard for service metadata

---

Correspondents point out that considerable effort has already been put in place in the IVOA Registry effort to define metadata description formats for data services.

• 8< ---- It is the job of the registry wg to come up with the structures needed for describing resources. If the VOQL wg tries to come up with an alternative structure then we're heading for an almighty row in the IVOA.
• 8< ----

We are asked if we have objections to those descriptions:

• 8< ---- Is there anything that is not supported in Registry schema?
• 8< ----

It is suggested that we work with the Registry team if so, to produce a better metadata description:

• 8< ---- If [the VOQL group] thinks the registry schema ought to be extended or improved then take that argument to the registry group.
• 8< ----
• 8< ---- The registry metadata for a resource ought to be enough to completely describe the resource, so all that is needed is a 'getDefinition' call and the metadata comes back in the same format as it is (or might or ought to be) stored in the registry. If the registry metadata is not enough then change the registry, don't add stuff into the resource service which is not available in any other way.
• 8< ----

  ---

Key point 3: Data services are likely to have to provide registry metadata in future

---

The VOSI (VO Support Interfaces) working draft already describes a method for getting metadata, in VO Registry format, that all services should support:

http://www.ivoa.net/twiki/bin/view/IVOA/IvoaGridAndWebServices#VO_Support_Interfaces_proposal

A correspondent writes:

• 8< ---- I thought the whole point of [the draft] VOSI "Support Interface" is it has a getMetadata call which returns back Resources that conform to the Registry schema.

      I suspect there will be a requirement for IVOA services to support the 
      VOSI and I believe getMetadata is a required method.  

      I think [the current working draft of VOSI] is a little outdated 
      because it tends to refer a little more towards SOAP services, but I 
      am quite certain there is an intention for it to be used with Http-get 
      as well. 

• 8< ----

In other words, data services are likely to have to support returning service metadata in registry format; if so, even the simplest TAP services will have to supply such metadata to be VO-compliant.

To this, I would add another point: if a service is to be a compliant, discoverable VO service at all, it will need to be registered. Even the simplest TAP services will therefore need to be able to generate registry metadata (or have it generated on their behalf) - so we don't avoid that particular piece of work by not providing a proper registry-format getMetadata call in TAP. Or are we really suggesting that TAP services need not be part of the VO?

---

Key point 4 - There is not a strong objection to additional metadata methods

---

Correspondents do not seem to oppose the provision of additional metadata methods (e.g. those returning results in VOTable), as long as the registry-format metadata call is provided.

• 8< ---- If TAP is going to have its own metadata type calls much like SkyNode whereby you ask it for columns and such, then fine, let it be there. But [...] let the 'getMetadata' call defined by the VOSI return a more full VOResource entry [...] [and] then I think we would be happy.
• 8< ----
• 8< ---- If the service wants to implement *extra* methods like getTables, getColumns etc then that is not a problem but it *must* implement the getRegistration method to return the fully completed registry record applicable to that service.
• 8< ----

  ---

Summary

---

Based on these inputs, I therefore propose that services implementing the TAP protocol *must* provide a metadata method that returns a proper registry-format service description. I propose that we consult with the Registry group for advice on this matter (i.e. for advice regarding what resource description/schema is most relevant). If the current standard for data service descriptions is inadequate for our purposes, we should provide feedback about this to the registry group.

Please note that providing registry-formatted metadata descriptions does *not* imply that TAP needs SOAP; we could return the relevant XML description via REST (just as a simple conesearch can return VOTable).

As to providing metadata in VOTable format, I would not object to this being supported in TAP, and Francois' draft document is an excellent starting point. I am not sure that this alternative method should be compulsory, although it probably wouldn't hurt if it were.

In general, as an IVOA working group I do think we have a duty to try and work with the standards already defined by the IVOA, even if that means a bit more work for us in the short term.

Best wishes,
Kona

-- 
Kona Andrews        kea-at-roe.ac.uk
AstroGrid Project   http://www.astrogrid.org
IfA, Royal Observatory, Blackford Hill, Edinburgh EH9 3HJ

Received on 2007-03-07Z12:20:55