As promised, here are some collected comments from the Registry WG and
others (NVO) concerning the issue of caching dataset (table) metadata
in the registry. I have heard these views consistently for some time,
and just wanted to ensure that this point of view was being passed on
to our group to inform our discussions tomorrow. - Doug
Hi Ray -
We have another TAP telecon coming up tomorrow morning. A question has come up regarding what the Registry wants or requires for registering a tabular resource. In particular, does the registry require that table metadata be included in the resource description cached in the registry, or is this an optional extension?
The context here, is that for TAP we have discussed returning only service metadata (service description and capabilities) via the getCapabilities operation, with the response being similar to or the same as the VOResource description for the service. A separate runtime interface would be used to query database or table metadata, probably using the same interface used to query actual table data, since table metadata is 1) dataset instance metadata, and not something one would normally cache in the resource registry, and 2) is tabular in nature, and very similar to querying the data table itself. The opposing view is that table metadata should be included in the resource record in the registry instead.
Basically I would just like to have the view of the Registry WG on this issue, to advise our discussions in the telecon tomorrow morning. I will forward your response (or any other comments received) to the group. Thanks.
I'll just insert my view, which is, as usual, table metadata (by which I mean column names and their associated metadata) should be determined through a query to the TAP service and not stored in the registry. Some of the DBs that will be interfaced via TAP are very complex, just as is the case with SkyNodes, and I think this pushes the registries too far.
Bob
On 3/7/07 1:45 PM, "Doug Tody" <dtody-at-nrao.edu> wrote:
> Hi Ray -
>
> We have another TAP telecon coming up tomorrow morning. A question has
> come up regarding what the Registry wants or requires for registering
> a tabular resource. In particular, does the registry require that
> table metadata be included in the resource description cached in the
> registry, or is this an optional extension?
>
> The context here, is that for TAP we have discussed returning only
> service metadata (service description and capabilities) via the
> getCapabilities operation, with the response being similar to or the
> same as the VOResource description for the service. A separate runtime
> interface would be used to query database or table metadata, probably
> using the same interface used to query actual table data, since table
> metadata is 1) dataset instance metadata, and not something one would
> normally cache in the resource registry, and 2) is tabular in nature,
> and very similar to querying the data table itself. The opposing view
> is that table metadata should be included in the resource record in
> the registry instead.
>
> Basically I would just like to have the view of the Registry WG on
> this issue, to advise our discussions in the telecon tomorrow morning.
> I will forward your response (or any other comments received) to
> the group. Thanks.
>
> - Doug
Hi Doug,
On Wed, 7 Mar 2007, Doug Tody wrote:
> We have another TAP telecon coming up tomorrow morning. A question has
> come up regarding what the Registry wants or requires for registering
> a tabular resource. In particular, does the registry require that
> table metadata be included in the resource description cached in the
> registry, or is this an optional extension?
A service with TAP capabilities should be registered as a "CatalogService". This allows one to include table metadata, but it is optional. Given our local discussions in the NVO, it's fair to say that we do not recommend including it.
hope this helps,
Ray
Even though "in principle" schemas don't change (hi, hi, ...), including the fine grain in the Registry is a YELLING call for inconsistency.
my 2 cents
Maria
On Wed, 7 Mar 2007, Robert Hanisch wrote:
> I'll just insert my view, which is, as usual, table metadata (by which I
> mean column names and their associated metadata) should be determined
> through a query to the TAP service and not stored in the registry. Some of
> the DBs that will be interfaced via TAP are very complex, just as is the
> case with SkyNodes, and I think this pushes the registries too far.
>
> Bob
>
> On 3/7/07 1:45 PM, "Doug Tody" <dtody-at-nrao.edu> wrote:
>
>> Hi Ray -
>>
>> We have another TAP telecon coming up tomorrow morning. A question has
>> come up regarding what the Registry wants or requires for registering
>> a tabular resource. In particular, does the registry require that
>> table metadata be included in the resource description cached in the
>> registry, or is this an optional extension?
>>
>> The context here, is that for TAP we have discussed returning only
>> service metadata (service description and capabilities) via the
>> getCapabilities operation, with the response being similar to or the
>> same as the VOResource description for the service. A separate runtime
>> interface would be used to query database or table metadata, probably
>> using the same interface used to query actual table data, since table
>> metadata is 1) dataset instance metadata, and not something one would
>> normally cache in the resource registry, and 2) is tabular in nature,
>> and very similar to querying the data table itself. The opposing view
>> is that table metadata should be included in the resource record in
>> the registry instead.
>>
>> Basically I would just like to have the view of the Registry WG on
>> this issue, to advise our discussions in the telecon tomorrow morning.
>> I will forward your response (or any other comments received) to
>> the group. Thanks.
>>
>> - Doug
>
--
------------------------------------------------
Maria A. Nieto-Santisteban (nieto-at-pha.jhu.edu)
Johns Hopkins University
3400 N. Charles St.
Physics & Astronomy Department
Baltimore, MD 21218 (USA)
Tel: 1 410 516-7679 Fax: 1 410 516-5096
---------- Forwarded message ----------
Date: Wed, 07 Mar 2007 12:43:08 -0800
From: Roy Williams <roy-at-cacr.caltech.edu>
To: Robert Hanisch <hanisch-at-stsci.edu>
Cc: Doug Tody <dtody-at-nrao.edu>, Ray Plante <rplante-at-ncsa.uiuc.edu>,
NVO Technical Working Group <techwg-at-us-vo.org>
Subject: Re: [nvo-techwg] TAP and registry
Robert Hanisch wrote:
> table metadata (by which I
> mean column names and their associated metadata) should be determined
> through a query to the TAP service and not stored in the registry. Some of
> the DBs that will be interfaced via TAP are very complex, just as is the
> case with SkyNodes, and I think this pushes the registries too far.
I agree with Bob and Maria that it is asking for trouble to have these "rich
registries". In exchange for centralizing and greater efficiency, we have the
agony of making sure that the cached content is accurate.
A similar issue arises with the inventory services that John Good and Tom
McGlynn are making. The service provides a new view of some dataset, which
requires sucking up the whole dataset first. But if the underlying data changes
or new datasets appear, then we need to do the sucking again. Either by an
active update triggered by the change or the publication of the dataset, or by
some kind of passive process like a web crawler.
Roy
Received on 2007-03-08Z05:56:22