Implementation thoughts on discussion points

[Guy Rixon]

On 5 Oct 2009, at 20:16, Matthew Graham wrote:

> Hi,
>
> Given that Dave and Doug are elsewhere this week, I a;m going to  
> play devil's advocate and make implementation suggestions for each  
> of the discussion points for you to shoot down:
>
> 1)   Pagination
>
> The listNodes method is to be removed - listing will be possible  
> with a straightforward HTTP GET to the node. The representation  
> returned can be altered with the use of a 'detail' keyword:
>
> http://nvo.caltech.edu/vospace-2.0/mjg/mytable1?detail= {min|max| 
> properties}
>
> When the node is a container, it will return a list of direct  
> children nodes in the container. Now if the container contains a lot  
> of nodes (>10000) then the client can specify a numerically limited  
> subset using 'limit' and 'offset' keywords:
>
> http://nvo.caltech.edu/vospace-2.0/mjg/mydir1?limit=500&offset=2500
>
> The ordering is determined by the server.
>
> The aim here is that listing no longer makes use of continuation  
> tokens but that pagination of results is controlled by the client.
>
> findNodes can employ a similar mechanism but is directed against the  
> appropriate search resource and not the node since a search can be  
> an asynchronous activity.

+1. I feel more RESTed already :)

"The ordering is determined by the server": yes, but we should be  
careful of the details. The ordering must be consistent between calls  
such that each {limit, offset} is drawn from the same, ordered  
sequence. That's implied, of course, but I would state it explicitly.

What happens if a client specifies {limit, offset} in a GET request to  
a leaf node? Do we treat it as an error or ignore the paging parameters?



>
> 2) Simple HTTP GET for client-server transfers
>
> The aim here is to maintain the protocol negotiation capability to  
> support
> advanced protocols, but adding simple retrieval access via HTTP as  
> well for
> cases where nonauthenticated access is permissible.
>
> The HTTP URL should be persistent, i.e., it should be possible for  
> multiple clients to retrieve the file given a single URL.  Even  
> where authentication is required we might want to retain the same  
> URL, but merely require HTTPS to access it.
>
> A view needs to specified to return the appropriate resource - a  
> basic HTTP GET call to a node just returns the resource  
> representation (listing). So the minimum call is:
>
> http://nvo.caltech.edu/vospace-2.0/mydata/table3?view=ivo://net.ivoa/core/views/votable-1.1
>
> This may result in a redirection to another endpoint if appropriate.

I have two problems here.

First, the mapping for a node from the vos:// URI to http:// is  
broken. That's a bigger problem that I won't detail here, but it has  
to be sorted out before this can work.

Second, most of the time one doesn't want to specify the output  
format; either there's no choice or the client wants the default.  
However, a get without a a view is already specified to get the node  
metadata.

I would make the basic form like this:

   http://nvo.caltech.edu/vospace-2.0/mydata/table3?view=data

then use HTTP negotiation to sort out the available formats. We didn't  
use HTTP negotiation in VOSpace 1.x because SOAP breaks it. We haven't  
used it in DAL because DAL is supposed to be independent of HTTP.  
Here, we define VOSpace to be based on HTTP so we can use HTTP features.

>
>
> 3) Parameterized version as alternate to XML representation
>
> Instead of HTTP POSTing a transfer representation to the space (http://nvo.caltech.edu/vospace2.0/transfers 
> ):
>
> <transfer xmlns="http://www.ivoa.net/xml/VOSpaceTypes-v2.0">
>    <target>vos://nvo.caltech.edu!vospace2.0/mydata/table3</target>
>    <direction>pushToVoSpace</direction>
>    <view uri="ivo://ivoa.net/vospace/core#votable"/>
>    <protocol uri="ivo://ivoa.net/vospace/core#http put"/>
> </transfer>
>
> we can HTTP POST a parameterized representation:
>
> http://nvo.caltech.edu/vospace2.0/transfers?target=vos://nvo.caltech.edu!vospace2.0/mydata/mydata/table3&direction=pushToVoSpace&view=ivo://ivoa.net/vospace/core#votable 
> &protocol=ivo://ivoa.net/vospace/core#http-put
>
> This would return the URI of the transfer resource giving  
> operational details of the transfer such as the endpoint.

+1, I think (still trying to mentally track down the implications).  
This is a correct use of POST to create a child resource as per the  
original HTTP spec. In an actual request, the URIs in the parameters  
would need to be encoded, I think.



>
> ---
>
> 4) UWS for transfers
>
> Under UWS, the elements in the transfer resource representation  
> become parameters in the UWS Job resource representation. A transfer  
> job could thus be created by a parameterized HTTP POST to the  
> transfers endpoint:
>
> http://nvo.caltech.edu/vospace2.0/transfers?target=vos://nvo.caltech.edu!vospace2.0/mydata/mydata/table3&direction=pushToVoSpace&view=ivo://ivoa.net/vospace/core#votable 
> &protocol=ivo://ivoa.net/vospace/core#http-put
>
> This would return the jobid and it could then be initiated with a  
> POST of a single parameter PHASE=RUN to the endpoint:
>
> http://nvo.caltech.edu/vospace2.0/transfers/{jobid}/phase
>
> A transfer could be aborted at any time with a POST of a single  
> parameter PHASE=ABORT to the same endpoint.


+1, but I would add the ability to run the transfer from the original  
post by adding PHASE=RUN there.


>
> ---
>
> 5) Extra information requested
>
> As part of the information returned in the resource representation  
> of a container, the amount of space available within that container  
> should be returned.
>
> ---
>
> 	Cheers,
>
> 	Matthew