Re: VOStore/VOSpace use cases

Hi Matthew -

On Thu, 11 Aug 2005, Matthew Graham wrote:
> Hi Doug,
> Guy has mentioned to me that you had some specific VOStore/VOSpace use cases
> - are these already documented somewhere or could you provide a short
> description of them?

Probably this refers to my data access related use-cases. The are three primary cases:

1. Service generates datasets and puts them in a local VOStore. Client picks up the images with a getData request via URL (this is the default and is what we do currently).
2. Client operates a local VOStore. Client issues a staging request to a remote service to generate datasets and deliver them to the Client's VOStore. Service sends a message to the client as each dataset is delivered (or client gets the messages when it polls etc.). In a typical scenario a client may issue many such asynchronous requests which execute concurrently. Support for authorization is needed since the local VOStore is externally writeable, and to support access to proprietary data.

       This is the scenario I see for VO-Client functionality integrated
       into a desktop data analysis system.  The VO-Client implements
       the client side of the data access services including support for
       queries, asynchronous services, authentication/autorization, etc.
       Data is fetched and placed in a local VOStore in the local VO-Client
       daemon.  Data analysis software can then directly access the files
       as they would a local file, or we can put an object API in front
       of the data and the analysis code will see an object API and can
       completely ignore all the plumbing.

       In this use-case the client may also function as a service,
       generating new data products which it exposes to the VO via the
       local VOStore.

    3) As in 2), but the staging request specifies that data is to be
       delivered to a third VOStore, not co-located with either the
       controlling client or the service.  This permits general grid
       workflows.  The client "application" is generally controlled from
       one location, but may have components which execute at multiple
       locations.

Note that asynchronous services, VOStore, and security all need to be interoperable to do these things. Ideally we need some mechanism for asynchronous messaging. A polling-based mechanism would be adequate for simple cases but does not scale well.

All of these use-cases emphasize dataflow and data caching rather than permanent storage of data. The only exception might be 2) where we might want to keep the data around indefinitely. However, it is still primarily a network data cache mechanism.

Another major use case is a user filestore where data is maintained indefinitely, possibly distributed to multiple locations. This is more the VOSpace type of scenario.

• Doug