[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: Time for Software
I agree with Chris on the first two items; i.e. use FITS for images and star
lists. The last item is a little trickier. The problem is matching the
observations to the TASS catalog at individual TASS sites since the catalog
doesn't yet exist. To make this work we would need some way of seeding and
updating the catalogs between databases so we can synchronize observations.
Mike G.
-----Original Message-----
From: chrisja@jps.net [mailto:chrisja@jps.net]
Sent: Wednesday, August 02, 2000 4:31 PM
To: Gutzwiller, Michael; 'Tom Droege'; 'tass@listserv.wwa.com'
Subject: RE: Time for Software
>I'll take a whack at the pipeline stages:
>
>Stage Input Output
>
>Camera Calibration Photons Dark Field, Flat Field
>Image Acquisition Photons FITS Image
>Star List Generation FITS Image, Star List (x,y)
> Dark Field,
> Flat Field
>Registration to Catalog Star List (x,y) Star List (ra,dec)
>Color Calibration Star List (x,y) Star List (ra,dec)
>Insert to database Star List (ra, dec) Observations database
>
>There may be other steps for calibration or post processing like FlatComp
>for Mark III images. Some steps may be combined as Star does with star
list
>generation and registration to catalog. Some steps may be reversed such as
>doing color calibration on the observations database as was done with the
>tenx catalog.
>
>I believe the relevant places to standardize are the same as for the Mark
>III - FITS image, Star List (ra,dec), and database format.
I agree here. Three "standard" data formats and then let
each camera operator figure out how to process his own
data. Now tohammer out the standrds. Here are some
gals, I'll propose:
1) FITS for images. (I thik we all understand this and
have only mimimal technical things to work out.)
2) Star Lists. I propose we use FITS here also.
Reasons are:
a) It is a self decribing format.
b) FITS has a published spec. pre-written
c) there are many readers available to read FITS table
data for UNIX, DOS and Window.
d) There is a good library (cfitsio)
e) If we adpt FITS we can change our convention later, say
add a new column and not break old software. This is
a big plus.
f) If we go with ASCII FITS we can still look at the file
in a plain text editor as FITS ASCII tables are just
plain old text files.
e) I already have a worksing FITS --> Postgres program.
so the database import problem is solved.
3) Database processing: The MkIII model (everyone sends
star lists to Michael who then puts then into his database)
will not scale to the Mk IV. Not unless Michael gets a
_much_ bigger computer. I propose that we do some preliminary
catalog matching at each camera site and then send Michael
prematched lists. This would off load most of the work from
him. This means we ned a third file format to searve as
an interface betwen a camera site and a database maintainer.
This file format will likely NOT be one file per image frame.
It would be sorted by star and make N observations per star
taken ib either pass band and on several nights and have
(where possible) a TASS catalog number for each star. I'd
suggest using FITS for this too.
>
>
>Mike G.
>
>-----Original Message-----
>From: owner-tass@listserv.wwa.com [mailto:owner-tass@listserv.wwa.com]On
>Behalf Of Tom Droege
>Sent: Wednesday, August 02, 2000 12:38 PM
>To: tass@listserv.wwa.com
>Subject: Re: Time for Software
>
>
>I understand that each of you will want to do it all yourself. Actually
>there are probably two classes, those who will tend to do everything
>themselves, and those who want to get everything done by someone else. The
>latter class has at least one member (me.)
>
>I think what is really important is that we all agree on formats/standards
>so that we can share programs. It would be nice to be able to process the
>same data through several pipelines and compare outputs at various stages
>in the process. This implies that there are stages.
>
>Unless we define stages, all we will know is that pipeline A has some
>systematic difference from pipeline B when we compare the outputs. Some
>stages might be star finding, flat field correction, color correction,
etc..
>
>OK, I am far from an expert here. I am just trying to stimulate some
>discussion as to what the pipeline should look like. Some of you have some
>real experience on a lot of different systems. Some of you have little
>experience with anything but Mark III and Mark IV data. The later group
>might be major contributors to the software effort. But to do so, they
>need somehow to get input from the experienced as to how to partition the
>effort. This partition will also allow several people to create one
>pipeline.
>
>Tom Droege
>
>
>
>