Re: TAP draft

From: Doug Tody <dtody-at-nrao.edu>
Date: Thu, 15 May 2008 17:29:21 -0400


Hi All -

As part of the "TAP/Param" effort we surveyed some existing table query interfaces representing current practice of large astronomical data centers (these are production interfaces in daily use, not prototypes). These are interesting to study to see what is the current practice and what works for users. I append some notes and links below.

These cases all represent the simple parameter-type interface, which is what is needed for most simple end-user use to query individual astronomical catalogs (which represents most user research-oriented table access in astronomy). Where we need ADQL/VOSpace/async etc. (and we do) is to support more sophisticated analysis software, and to federate resources at data centers to be able to do large scale cross matches and the like - currently this is done at a few places, but it is done on a site specific basis with proprietary software. Both of these use cases are essential, and there is no reason that TAP can't support both.

Study of Existing Data Center Table Access Interfaces


    The remainder of this document summarizes the existing interfaces     to several major data centers (CDS, IRSA, HEASARC), all of which     are HTTP GET and parameter based.

CDS ASU Syntax/Capabilities



URL: http://vizier.u-strasbg.fr/doc/asu.html

    Syntax

        -option         query option or parameter
        field           field name

    Parameters
        source          catalog or data source (multiple catalogs permitted)
        sql             url-encoded SQL expression
        c               ra{+-}dec|objname, c.eq=equinox, c.epoch,
                        c.rm=[min/]max, c.bm=x[/y]
        mime            output format
                        (html,fits,tsv,text,gif,jpeg,starbase,ps,pdf,...)
        out             list of fields to be returned (also "all", "basic"
        out.max         maximum number of output fields
        out.next        to step through large queries (server dependent)
        out.object      return only object identification (name, ra, dec)
        oc              coordinate frame for output coordinates
        sort            sort by field[,field[,...]]
        resolv          which name resolver to use (ned, simbad)

    Features
        Multiple catalogs can be queried (union of results is the default)
        Query by position is a special case
        Supports query by name or position
        Can input coords in sexagesimal notation
        Variety of time stds for equinox and epoch
        Radius includes both min and max values
        Box search rectangle tan projection aligned with frame (undef at poles)
        Frame is specified with the equinox
        Can specify coordinate output format
        Can specify sort order of output table

    Numeric Query Constraint Operators

= != >= <= < > equality, less than, greater
.. numeric range & | AND/OR, e.g., "1..5 & 7..9" , list of values ! NOT (logical) null Unspecified/blank value now For DATE comparisons JDnnn Julian date (otherwise UT) Character Query Constraint Operators ~ Caseless pattern matching (default, wildcards) = Pattern matching (wildcards) =~ Caseless equality == Strict equality ! !~ != NOT of pattern matching, caseless PM, strict equality > >= < <= Alphabetical comparisons Pattern matching supports [<set>], [^<set>] (NOT of set), *, ? Examples http://host_name/cgi_path?-sql=select%20distinct%20name%20from%20table

IRSA Gator Syntax/Capabilities



URL: http://irsa.ipac.caltech.edu/applications/Gator//GatorAid/Gator_Api.html

    Syntax

        only simple parameters
        URL encoding can use "+" to replace a space
        +/- used to delimit RA DEC pairs

    Parameters
        spatial         Type of spatial query (Cone,Box,Polygon,NONE)
          polygon       Convex polygon, ra dec pairs, comma delimited
          radius        Cone search radius (default 10 arcsec)
          radunits      Units cone search (arcsec, arcmin, deg; default arcsec)
          size          Width of a Box in arcsec
          objstr        Target name or coordinate of center of spatial region
        catalog         Catalog name as defined by IRSA
        outfmt          3=VOTable, 2=SVC msg, 1=ASCII table, 0=HTML (default)
        desc            Pass-through description of catalog for result page
        order           Sort output by this column
        constraint      SQL WHERE clause
        onlist          Used to get at new catalogs for testing
        selcols         Columns to be returned
        outrows         Number of output rows

    Features
        CatQuery        Query a given catalog
        CatList         List of catalogs (XML or HTML)
                          supports region queries, SQL queries
        CatDD           List the fields of a catalog (XML or HTML)
                          default columns or all columns
                          can use this in a UI to build a query
                          description, units, index, DBType
                          support MULTI-OBJECT SEARCH

    Numeric Query Constraint Operators
        Uses SQL

    Character Query Constraint Operators
        Uses SQL

    Examples
        http://.../Gator/nph-query?
            outfmt=1&objstr=m31&spatial=Cone&catalog=fp_psc
        http://.../Gator/nph-query?
            outfmt=1&objstr=10.68469+41.26904&spatial=Cone&
            radius=1&radunits=arcsec&catalog=fp_psc
        http://.../Gator/nph-query?
            outfmt=1&order=j_m&objstr=m31&spatial=Polygon&
            polygon=10.1+10.1,10.0+10.1,10+10&catalog=fp_psc
        http://.../Gator/nph-query?
            outfmt=2&objstr=m31&spatial=Box&catalog=fp_psc&
            selcols=ra,dec,j_m,h_m,k_m&size=10


HEASARC CGI Syntax/Capabilities



URL: http://heasarc.gsfc.nasa.gov/W3Browse/w3batchinfo.html

    Syntax

        Same as "browse" web interface
        >5000, 4..10, 3C*273, etc.

    Parameters
        table           Table name (mandatory)
        position        Object name or position
        coordinates     Spatial frame ("equatorial" or "galactic")
        equinox         Equinox for equatorial coordinates (default 2000)
        radius          Radius for search in arcmin (def 60)
        fields          "standard", "all", or comma-delim list of fields
        name_resolver   NED or Simbad
        infile          For multi-object searches; each line is a "position"
        outfile         Save results to a file (for Perl script)
        format          HTML, FITS, EXCEL, VOTable, Text
        sortvar         Sort field
        param           "param=name,value" (or "name=value")

    Features
        Positions support multiple values delimited by semicolons

    Numeric Query Constraint Operators
    Character Query Constraint Operators

    Examples

        browse_extract.pl table=rospublic position=3c273 name_resolver=ned Received on 2008-05-15Z23:32:36