Problems about the Spectrum Data Model from the view of a Web Service programmer

Hello DM group,

For those who don't know me, I'm Alex Szalay's and Tamas Budavari's student at the JHU and I was working on the Spectrum Service (http://voservices.net/spectrum) in the last three years. I'm studying physics at the Eotvos University, Hungary and before starting the university I was working for years as a database and distributed application programmer for commercial companies.

I just ran through the document quickly, because I was interested in the Web Service issues. I'm pretty sure that from the scientific view the data model is very well designed, and your work on collection and organizing the enourmous number of different metadata fields is appreciated, but I've found some problems that may cause trouble in a web service aware application.

Here are my observations:

• 1. XML serialization puts the value of the fields between opening and closing tags. It's okay for an XML document, but SOAP (the web services XML protocoll) can deserialize _only strings_ if they're written between the tags and other attributes are also specified at the same time. The correct way to store values is in elements, like

	<Wavelength>
		<Unit>nm</Unit>
		<Value>700</Value>
	</Wavelength>

or store in attributes, like

        <Wavelength Unit="nm" Value="700" />

But the following is not correct:

        <Wavelength Unit="nm">700</Wavelength>

We should keep in mind that even though this third version looks nicer in a text editor, XML is nor for reading by humans, but for using in web services, so I suggest to follow the SOAP standard instead of making good looking documents.

• 2. The model now supports serializing data points (i.e. the elements of the arrays) two ways (struct-style and flat). From the programmers approach, both may be problematic. Flat-style is obviously introduced in order to shrink the resulting xml files to reduce network traffic, which is nice.

The problem is the following: If one creates program classes in Java or C# or whatever and wants to expose his/her functions as a web service, he/she might also want to add the optional fields to his/her data modell. The problem is with the normal data types. Let's consider an everyday double variable. It can have the value of 0, but the SOAP serializer is going to write it into the XML anyway. To avoid a value written into the XML the variable should be set to null, but only reference types (pointers) can be null. In the Spectrum Service I'm developing I used the following approach: stored the header field as usual, in classes producing a hierarchical set of variables and stored the actual data in simple arrays of doubles (or ints). Because arrays are reference types, they can have the value of null, so if I don't want to use any of the optional axes, I simply set it to null.

If I would have used an array of structs (i.e. simple data types grouped together), I couldn't make this trick, I would have to set each unused field in each instance of the struct to null... but fields of the struct are simple types thus cannot be null...

So if we want to create a data model that supports the web services and still want to keep the xml file size small, we should consider storing the actual data in simple arrays instead of arrays of structs. It is also going to help in the future, when the binary web service protocold will be available!

• 3. Francois Ochsenbein and I found some issues about VOTables too. No correct proxy class can be generated according to the VOTable xsd, the resulting code must be modified by hand at several places, at least in windows .net and linux mono framework (the two main web service implementations).
• 4. STC uses different data types than the spectrum header and characterization use. Thay should be synchronised sooner or later, since if we start implementing a unit conversion software sometime in the (far) future, it will be very useful to have everything in the same structure. Thus, a Quantity object with field of unit, ucd, value are required. I have such an object in my spectrum model implementation (which currently implements the previous version of the document, unfortunately, without the characterisation and STC), and I can automatically convert it to VOTable or serialize as an XML using a single line of code and I should not modify the serializer/deserializer code if the underlying object structure (DM) changes.
• 5. The SSA protocol uses a different data model to return the spectrum info in a table. Why not to return an array of spectrum headers instead without the data points?

I've attached a document that I wrote a year ago but somehow missed the DM group.

I Hope I could help eliminating these problems about Web Services. Unfortunately I cannot go to the Moscow meeting, but most of you will be at the ADASS, so if you're interested, I can show you the actual Spectrum Service code and explain the programming tricks that bridge the gaps between the complicated data model and the web service standard.

-Laszlo