RE: ADQL - Careful what you wish for

I can appreciate that starting from scratch to build a completely new general query language would be a long, hard job. However we don't seem to have a suitable query language to hand that we can use, so we need to modify/adapt something.

We can't use SQL: it's not standard. We can define an SQL-like language limited to the lowest common denominator but we have to write parsers etc that will handle them correctly, including our extensions. This is not trivial, whereas it is much easier to define these in XML.

XQuery doesn't seem to be well understood by anyone on the list (or?) and could do with some more research. Even if there are no good implementations yet, we are going to have to write our own implementation (and have already written for AstroGrid) for whatever language we use, so this should not stop us. XQuery does not however look very easy to use...

I'm putting the finishing touches to an XML query language (ADQL2?) based, like ADQL, on SQL. Being SQLish we borrow SQL principles. Being XML allows us all to look at, understand and (dis)agree its syntax, gives us tools to validate it, tools to build GUIs around it (in fact even Eclipse's XML Buddy helps me construct such queries) and it's straightforward to read/write, straightforward to convert to/from SQL and has some interim capability to include XPaths as well as columns.

I am very conscious that we have people using and demonstrating services now, who require a usable query language. Perhaps we can use this in the meantime while we look at XQuery?

Cheers,

Martin

Quoting Tony Linde <ael-at-star.le.ac.uk>:

> Hi Jim,
>
> I don't think anyone wants to create a 'new' language. Whatever form ADQL
> takes, it will always be based on SQL. The reason for having an xml-based
> rahter than text-based language is simply to make it easier to construct and
> parse and easier to translate into the (usually) SQL variants that the
> back-end repositories require. It will also make it easier to specify
> vectors, arrays and structures as values in WHERE clauses and as parameters
> in functions, all of which are in 'standard' SQL but which require messy
> representations in textual form (eg Region ('CIRCLE J2000 19.5 -36.7
> 0.02')).
>
> Cheers,
> Tony.
>
> > -----Original Message-----
> > From: Jim Gray [mailto:gray-at-microsoft.com]
> > Sent: 02 February 2004 15:28
> > To: Tony Linde; Wil O'Mullane
> > Cc: voql-at-ivoa.net
> > Subject: RE: ADQL - Careful what you wish for
> >
> >
> > sage advice:
> > As one who has watched language designers struggle for many
> > years with languages like Fortran, VisiCalc, SQL, and HTML, I
> > have learned that it is not an occupation for dilettantes like us.
> > So, to the extent that you can hijack an existing language
> > like SQL, do it.
> > There are no "minor" changes to these languages; they all
> > have any tight interconnections.
> > Adding XML data in the middle of some language is non trivial.
> > If you really love XML, you might look at xQuery, but it has
> > rather poor implementations so far, and it lacks
> > update/inset/delete; and some describe the language as "ugly" .
> > The Sky Query folks extended the table names with an extra
> > level of qualification and added a function (cross match).
> > That was safe and easy.
> > I recommend you try to use SQL's built in extension
> > mechanisms. It you really! want to do language design, do not
> > couple that research with the main path of the VO effort.
> > It would make a fine side-project for the computer science
> > guys -but it should not be on the critical path for the VO
> >
> > Jim Gray
> > Microsoft Research, Suite 1690, 455 Market, SF CA 94105,
> > tel: 415 778 8222 fax: 425 706 7329
> > Gray@Microsoft.com http://research.Microsoft.com/~gray
> >
> > -----Original Message-----
> > From: owner-voql-at-eso.org [mailto:owner-voql-at-eso.org] On
> > Behalf Of Tony Linde
> > Sent: Sunday, February 01, 2004 2:28 AM
> > To: 'Wil O'Mullane'
> > Cc: voql-at-ivoa.net
> > Subject: RE: ADQL - it aint so great
> >
> > > Personally I also do not feel we need an XML form of ADQL.
> > > But at IVOA meetings there was a general request to have
> > ADQL in XML
> > > format. I made a stab at and XML format. Modified it
> >
> > I still think we need the xml version. Astronomy includes
> > values which are not simple scalars; representing structures
> > even as simple as a polar coordinate is messy in a textual
> > language. As we add more complex features/funcitons to ADQL
> > we don't want to have to invent text equivalents of vectors,
> > arrays and more complex structured values.
> >
> > An xml document is also easier to construct from user
> > selections in a query screen and much easier to translate
> > into the many variants of SQL (and OQL and ...) that are used
> > in astronomical repositories since you don't have to
> > deconstruct the textual version first.
> >
> > Personally I'd drop the textual version of ADQL - if anyone
> > wants to type the query in textual form, provide a
> > pass-through option so they can type the end repository's SQL
> > query (or whatever language it requires). (If they want the
> > same query to go to multiple databases then they either
> > construct ADQL using whatever portal front end they prefer or
> > they type all the queries in manually.)
> >
> > Cheers,
> > Tony.
> >
> > > -----Original Message-----
> > > From: owner-voql-at-eso.org [mailto:owner-voql-at-eso.org] On
> > Behalf Of Wil
> > > O'Mullane
> > > Sent: 31 January 2004 19:27
> > > Cc: voql-at-ivoa.net
> > > Subject: Re: ADQL - it aint so great
> > >
> > >
> > > Personally I also do not feel we need an XML form of ADQL.
> > > But at IVOA meetings there was a general request to have
> > ADQL in XML
> > > format. I made a stab at and XML format. Modified it etc
> > ... This of
> > > course may be discussed again at the meeting.
> > > Even internal to SkyServer there is some parsing going on as Clive
> > > suggests. People generally felt having a parsed version
> > would help in
> > > this area.
> > >
> > >
> > > We are aware of some of the problems mentioned especially with
> > > java. Vivek is currently working on this and I believe will
> > > post some new tutorials etc in the coming days.
> > >
> > >
> > > wil
> > >
> > >
> > > On Sat, Jan 31, 2004 at 04:40:11PM +0000, Clive Page wrote:
> > > > On Fri, 30 Jan 2004, Noel Winstanley wrote:
> > > >
> > > > > I tried and I failed.
> > > > >
> > > > > I can;t get any joy with the latest version of ADQL.
> > > >
> > > > > I'm really beginning to wonder whether ADQL is the most suitable
> > > > > query language for astrogrid datacenters. What are its benefits?
> > > >
> > > > I agree with your comments, Noel, and I've been asking for
> > > some time
> > > > why we need ADQL. The only answer I was able to get was
> > > that it made
> > > > the checking of the query simpler and improves security. I am not
> > > > convinced that we need to be all that fussy in checing
> > > syntax: if the
> > > > user generates the query from our menu-style
> > registry-driven portal
> > > > then it will be hard to make a syntax mistake; power users who
> > > > generate their own SQL by typing it into a text box will
> > > find out soon
> > > > enough if they make a mistake: any DBMS parses the query
> > > and returns
> > > > an error message instantly. The experience of JHU with their
> > > > skyserver was given at the last ADASS meeting by Wil
> > > O'Mullane: indeed
> > > > he made a mistake in his live demo and got a message
> > back, allowing
> > > > him to correct it. This isn't a perfectly user-friendly
> > > system, but
> > > > it seems adequate to me. As far as security goes, a simple
> > > solution
> > > > initially is to restrict the first keyword of any query to
> > > be SELECT,
> > > > and in particular prevent non-authenticated users from
> > > issuing DROP or
> > > > DELETE statements. Later, maybe, we can allow things like CREATE
> > > > TABLE, INSERT, UPDATE, and so on. That shouldn't be all
> > that hard
> > > > even without ADQL.
> > > >
> > > >
> > > > --
> > > > Clive Page
> > > > Dept of Physics & Astronomy,
> > > > University of Leicester, Tel +44 116 252 3551
> > > > Leicester, LE1 7RH, U.K. Fax +44 116 252 3311
> > >
> >
> >
> >
>
>
>

-- 
Martin Hill
07901 55 24 66
www.mchill.net