US (NVO/VAO) opinions on HTTP GET in VOSpace 2.0

[Doug Tody]

Hi again -

Well this is an awful lot of trouble to go to just to get a file
from a storage service using a standard protocol.  In effect what
I am suggesting is that we integrate this resolver service directly
into each vospace implementation.

What you describe is not very scalable; in the described scenario
we can have any number of vospaces, but only one of these well-known
resolvers.  Plus there is an additional runtime dependence and if the
resolver goes down or is unreachable the access fails.  A use case
where this would not work is if we have a vospace integrated into a
desktop analysis framework and our desktop (eg laptop) is operating
without an Internet connection.

Of course there are ways to work around such a problem, but a storage
mechanism should provide a simple means to access a stored file.  Yes,
this might mean making HTTP a special case with optimized support.
I don't like special cases either but this is a pretty important one
for a network storage mechanism.  It need not interfere with support
for runtime negotiation of more advanced transport protocols.

 	- Doug


On Tue, 29 Sep 2009, Dave Morris wrote:

>
> On 29/09/09 04:33, Douglas Tody wrote:
>>
>>  Regardless of what transports the VOSpace supports it has
>>  to implement the VOSpace service interface. The service interface
>>  itself is already HTTP-based, hence adding something which speaks GET
>>  on the client end and FTP, GridFTP, or whatever on the back-end would
>>  be pretty trivial. One just opens a back-end connection stream of
>>  some sort and copies data out on the stream back to the client; this
>>  is trivial, at least for nonauthenticated transfers.
>
> This is pretty much what I meant in my earlier email when I referred to a 
> separate portal or resolver service.
>     http://www.ivoa.net/forum/vospace/0908/0215.htm
>
> If I recall correctly, the proposed HTTP GET form of the VOSpace service 
> would accept ?param=xxx parameters and then redirect the client to a 
> corresponding HTTP URL to download the data. In which case, it would not be 
> difficult to create a resolver service for use in the NVO. It could sit at a 
> well known location and you could use GET params or a POST form to pass it 
> any vos:// URI. The resolver service would do all messy stuff resolving the 
> vos:// URI in the R connect to the VOSpace service and negotiate a HTTP 
> download URL for the data, which it would then pass back to the client as a 
> HTTP redirect.
>
> Note that although the resolver service would be in a central location, the 
> data transfers would still happen direct from the VOSpace service to the 
> client - it would be making use of the 3rd party data transfer facilities we 
> deliberately added to the VOSpace specification to support just such a 
> scenario.
>
> If the target VOSpace service didn't support HTTP GET then, as you say, it 
> would be pretty trivial to add FTP, GridFTP, or whatever on the back-end of 
> your resolving service. The resolver service could negotiate the transfer 
> using whatever protocols were available and supply a HTTP GET URL to your 
> client. In this case the resolving service would act as a HTTP proxy between 
> the client and the VOSpace service. The data transfers would no longer be 
> direct from the VOSpace service, but if we are talking about the 90% of use 
> cases you describe that would be supported by a simple HTTP GET, then we 
> won't be transferring TBytes of data, so this should not be a problem.
>
> All of the above would give you simple HTTP GET using whatever parameters you 
> want to define, and no need to mangle the VOSpace API into trying to support 
> two incompatible forms of interfaces.
>
>
> Do you have a use case that could not be solved by providing a separate 
> resolver service for HTTP only clients ?
>
>
> Regards,
> Dave
>