On 17.08.2006, at 16:28, Roy Williams wrote:
> Dave & VOSpacers:
>
> Perhaps I can ask my questions more specifically.
>
> (1) Why not take the Amazon definition and just use that exactly?
In general terms the VOSpace and S3 interface defintions are pretty
similar (there are only so many functions you can add to a file
storage system!)- however VOSpace
1. tries to be more general, but not tying down association with
particular transport mechanisms, nor having a fixed set of node
metadata.
2. has built-in some of the associations with other VO components and
their associated semantics
As Dave has pointed out the VOSpace interface is in a sense more general than the S3 interface and as such it should be possible to write a VOSpace proxy for S3 that would allow S3 to be used as the storage, and it might be reasonably efficient as the S3 REST interface could be used as a VOSpace transport protocol
However, the real point is that S3 is really about the implementation rather than any particular niceties of the interface - being fast/ scalable/reliable are some of the most important features of the service - implementation and deployment are a key to wanting to use S3, and unfortunately we are unlikely to be able to use the Amazon implementation as they are charging for it. If we were to be storing all of our future data S3 then it might be worth adopting their proprietary interface, but in the VO we need a "vendor-neutral" interface to allow us to try to hide the differences between the inevitably varied systems from the end user astronomer.
> (2) What can Amazon do that VOSpace cannot (and why is that feature
> there?)
it would appear that S3 has a subset of VOSpace 2.0 features.
> (3) What can VOSpace do that Amazon cannot (and which use-case
> needs that?)
VOSpace 2 will have the concept of links to create connections
between different VOSpaces - S3 does not appear to have that.
An important use case for VOSpace is the ability to be able to run
data processing applications locally to the storage, I do not think
that S3 has had such a driver behind its design.
> (4) How does the metadata model differ between Amazon and VOSpace?
> (5) What are the differences in security model? If the VO model is
> different, why is it different?
As far as authentication is concerned we have had a long and hard
debate and we have a mechanism that is compatible with the Globus and
the wider grid world - S3 does not appear to be compatible with this
out of the box
> (6) What does VOSpace do that Amazon cannot (and who is demanding
> that extra feature?)
> (7) Is there a problem in implementing Amazon on top of SRB?
not an expert on SRB, but I suspect that the S3 interface *could* be implemented as a facade, but see the comments on 1) - essentially I think that our interface design should be driven by the ability to use it as an efficient facade on existing systems such as SRB and NGAS because the majority of the value is in the underlying implementation rather than the interface - the VOSpace specification/ interface is driven by that aim. I think that we have a better architectural view than in the two layer VOSpace/VOStore days, which did not fit well with these other systems which all present a view of the storage that is symmetrical between servers.
Ultimately I think that we remain more flexible with our own more general interface definition, however, if useful clients of S3 are built then there would probably be some merit in S3 interfaces being added to SRB and NGAS as well as VOSpace ones.
Paul Harrison
ESO
>
Received on 2006-08-17Z18:09:36