US (NVO/VAO) opinions on HTTP GET in VOSpace 2.0
dtody at nrao.edu
Tue Sep 29 13:52:05 PDT 2009
Hi again -
Well this is an awful lot of trouble to go to just to get a file
from a storage service using a standard protocol. In effect what
I am suggesting is that we integrate this resolver service directly
into each vospace implementation.
What you describe is not very scalable; in the described scenario
we can have any number of vospaces, but only one of these well-known
resolvers. Plus there is an additional runtime dependence and if the
resolver goes down or is unreachable the access fails. A use case
where this would not work is if we have a vospace integrated into a
desktop analysis framework and our desktop (eg laptop) is operating
without an Internet connection.
Of course there are ways to work around such a problem, but a storage
mechanism should provide a simple means to access a stored file. Yes,
this might mean making HTTP a special case with optimized support.
I don't like special cases either but this is a pretty important one
for a network storage mechanism. It need not interfere with support
for runtime negotiation of more advanced transport protocols.
On Tue, 29 Sep 2009, Dave Morris wrote:
> On 29/09/09 04:33, Douglas Tody wrote:
>> Regardless of what transports the VOSpace supports it has
>> to implement the VOSpace service interface. The service interface
>> itself is already HTTP-based, hence adding something which speaks GET
>> on the client end and FTP, GridFTP, or whatever on the back-end would
>> be pretty trivial. One just opens a back-end connection stream of
>> some sort and copies data out on the stream back to the client; this
>> is trivial, at least for nonauthenticated transfers.
> This is pretty much what I meant in my earlier email when I referred to a
> separate portal or resolver service.
> If I recall correctly, the proposed HTTP GET form of the VOSpace service
> would accept ?param=xxx parameters and then redirect the client to a
> corresponding HTTP URL to download the data. In which case, it would not be
> difficult to create a resolver service for use in the NVO. It could sit at a
> well known location and you could use GET params or a POST form to pass it
> any vos:// URI. The resolver service would do all messy stuff resolving the
> vos:// URI in the R connect to the VOSpace service and negotiate a HTTP
> download URL for the data, which it would then pass back to the client as a
> HTTP redirect.
> Note that although the resolver service would be in a central location, the
> data transfers would still happen direct from the VOSpace service to the
> client - it would be making use of the 3rd party data transfer facilities we
> deliberately added to the VOSpace specification to support just such a
> If the target VOSpace service didn't support HTTP GET then, as you say, it
> would be pretty trivial to add FTP, GridFTP, or whatever on the back-end of
> your resolving service. The resolver service could negotiate the transfer
> using whatever protocols were available and supply a HTTP GET URL to your
> client. In this case the resolving service would act as a HTTP proxy between
> the client and the VOSpace service. The data transfers would no longer be
> direct from the VOSpace service, but if we are talking about the 90% of use
> cases you describe that would be supported by a simple HTTP GET, then we
> won't be transferring TBytes of data, so this should not be a problem.
> All of the above would give you simple HTTP GET using whatever parameters you
> want to define, and no need to mangle the VOSpace API into trying to support
> two incompatible forms of interfaces.
> Do you have a use case that could not be solved by providing a separate
> resolver service for HTTP only clients ?
More information about the vospace