RE: Collaboration on Source Catalogue DM, ADQL and SkyNodes

From: Mark Taylor <m.b.taylor-at-bristol.ac.uk>
Date: Wed, 21 Dec 2005 14:42:02 +0000 (GMT)


On Wed, 21 Dec 2005, Jim Gray wrote:

> There is also the cross match that Tanu & Maria implemented in SQL.
> That may be the easiest way to do the implementation if the nodes all
> have a SQL backend.
> When things are "in" the database it runs fast ~100k object/sec for
> small radius.
> But, getting objects into a node dominates all other costs (moving stuff
> thru xml is expensive).
> We recently found some logic errors in our "zones" cross match (near the
> meridian and poles) but seem to have a fix for them now.
> Maria can tell you all about this (it is the subject of her PhD).

Jim (and lurkers),

certainly the STIL/STILTS match is not going to compete with an SQL one which makes use of precomputed indices; it's designed for the case where you've got data with no advance preparation (typically just an unsorted list of (RA,Dec,other-data) rows, for instance a FITS table or VOTable streamed or on disk). Since Pedro says he's interested in client-side matching, I presume that the data's not going to be in a database and in general no RDBMS will even be available for use. However, it depends a bit on what is meant here by "client".

Mark

-- 
Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
m.b.taylor@bris.ac.uk +44-117-928-8776 http://www.star.bris.ac.uk/~mbt/
Received on 2005-12-21Z15:42:54