Re: table metadata and the registry

From: Ray Plante <rplante-at-poplar.ncsa.uiuc.edu>
Date: Tue, 8 May 2007 09:26:42 -0500 (CDT)


Hi Tony,

On Tue, 8 May 2007, Tony Linde wrote:
> One last query, if the URL only returns the 'extra' metadata, where does the
> core service metadata come from? The registry only? Does this mean the
> service provider has to maintain metadata in two locations? Surely one
> additional benefit of the getWhatever method is that a service provider can
> update their registry record simply by changing the VOResource record served
> up by getWhatever?

Let's look at this in two evolutionary stages. In the near future, the user still goes to a publishing registry and fills out a form. Consider...

   ...an SIA service: the user need not enter URL for the table metadata;

        this can be set automatically based on the base URL and the
        translater service.
   ...a theoretical TAP service:  some near-future standard protocol could
        provide this URL-based access as part of its definition; in this
        case the URL is set automatically in the record.  If the standard
        does not provide this, it can still be handled like an SIA through
        a translator.
   ...a Data Collection:  since there is not an associated service to
        provide this information, the publisher can optionally enter this
        URL explicitly.  A registry may provide tools for formatting this
        information.

When the form is submitted, the registry decides whether it wants to retrieve the table metadata.

In the next phase, when VOSI is more widely supported, essentially the same process happens at service machine when the publisher sets up the VOResource description getRegistration(). The VOSI standard provides a straightforward way to recognize service resource descriptions that support getRegistration() (the VOSI capability extension). The publisher provides the VOSI URL to the registry. The registry retrieves the VOResource description. At that time the registry decides whether it wants to retrieve the table metadata.

In either case, the registry has access to the table metadata if it wants it, but if it doesn't want it, it doesn't get it anyway.

>> to see is that all of the table information be retrievable in one URL.
>
> That is certainly more reasonable. But what I don't understand is why, if
> we're to have a single method of getting the metadata, it cannot be returned
> in the format recognised by the registry: VOResource? It would take no more
> work to format the metadata as VOResource than some other format so why not
> stick with what we have, why come up with another format?

I hope you noticed my suggestion that the VOResource table schema could provide the format because we are already using it. This is my preference.

The issue is not about another format. It's about providing access to the information that is not exclusively through the registry.

> But, as I said previously and above, this will lead to registries serving
> information in ways no other does and so, applications which rely on that
> information will only be able to work against specific registries.

Are you going to force us to do registries your way? We need interoperability, but this does not mean our registries must behave in identical ways. We need--you need--to be able to explore new ways to support discovery and access that best suits the needs of our respective communities without sabotaging each others efforts. That's what I'm looking for. Do you agree with this?

>> To put it in concrete terms, the AstroGrid registry effectively
>> pressures the NVO registries into supporting fine-grained table
>> ... First, you
>> encourage your publishers to provide table metadata. We harvest these
>> records which in turn go out to our users as a result of queries. We
>> have to then help users make sense of this information.
>
> I don't understand this - the information is the same whether you get it from
> the registry alone or from the registry plus the service. What is the
> difference?

If the fine-grained information is in our registry to begin with, then we do not have to deal with the metadata problems within the context of the registry.

>> When there are
>> problems with the information, it reflects poorly on us, not you.
>
> Poor metadata information is a problem we all have to tackle, not just
> AstroGrid: it is exacerbated by poor registry population applications and has
> nothing to do with the type of registry.

Do you see that we have to bear some responsibility for the metadata you export from your registries? And vice-versa?

>> Second, your application in effect encourages our publishers to provide
>> table metadata to our registry if they are to be used in your
>> application, because your application only gets this information from
>> the registry.
> This is not going to change, however the metadata is collected, whether by
> entry into the registry, by VOSI:getWhatever from the service or by the
> metadata URLs you propose. We will continue to provide full registry
> information and applications that rely on it: it is the only effective way to
> build responsive applications.

I want you to be able to do this. I want you to have a way to get the information you need from our services without requiring that you get it directly from our registries (via the harvesting interface).

> It is also, I might add, the only way to discover resources based on the
> additional metadata. If someone wants to discover x-ray catalogs with a given
> type of informaiton (specified by a ucd), how does it do this from a
> pointer-only registry, apart from getting every possible x-ray source and
> querying every one of them?

It is the only way today. NVO is working on other techniques for drilling down into the information for discovery outside the context of the Registry.

> I guess if we do follow your proposal it will, over the next couple of years,
> show what the application developers really want by the number that tie
> themselves to AG-style registries vs NVO-style registries.

That is fine. We only ask for the freedom to discover this ourselves.

> I'd like to expose more to the list since I won't be at the meeting.

(I'm hoping to capture it in a Note.)

cheers,
Ray Received on 2007-05-08Z16:27:01