Re: TASS Database, versions > 1.0

[Glenn Gombert <gleng@infinet.com>]

Chris,

        I would add a fourth option to the list. I would be interested in
producing astrometricaly/photometric calibrated lists of observations of
stars (like I am presently doing) as well as sending star-lists into the
master database. I think that many other people would be interested in doing
this as well.....

Glenn G. 

At 06:37 AM 4/3/98 +0000, you wrote:
>(This is another opinion poll, see question in last couple
>paragraphs above +++++)
>
>I am still working, slowly, on my "version 1" database software
>but I'm looking ahead to version two and the MkIV camera system.
>I just tonight got the catalog seeding operation to run at
>2000 records/second, so the whole operation took only about
>1200 seconds and nothing crashed. 
>
>I am finding that it takes a very long time to do astrometric
>matching when your lists are numbered in the millions of rows.
>The current system which will be continued in version 1 is that
>camera operators FTP their lists to the ftp.tass-survey.org site
>where they are merged into a central database.  This puts a lot
>of load on that one central site.  What we could do (in a future
>version two system) is this:  
>
>  (1) Each camera site periodically downloads the "tass catalog".
>  this is just three columns, ID Number, ra, dec.
>
>  (2) they save up several nights or weeks of star lists then run
>  the whole batch through a program that matches each list to
>  the downloaded catalog and tags each observation with a tass-ID.
>  For observations not matched to catalog entries the catalog is
>  extended.  Finally the starlist is sorted by tass-id then FTP'ed
>  along with the extended portion of the catalog to the normal place.
>
>This pre-processed data would be much faster to process into the
>database system.  Fast by a very large amount, maybe 4x to 10x.
>Unprocessed star lists could still be sent and they could be imported
>to the database by the current method.
>
>The above will require new software at the camera sites.  Or will it?
>+IF+ (big if) the people running those cameras would install the same
>database software suite the Michael has installed on his computer then
>they could maintain a local database of local observations.  They
>could do their own local matching using existing database software.
>Periodically they would produce dumps that are FTP'ed to the central
>database.  From my point of view this is the best solution as it
>requires very little more work on my part.
>
>I strongly suspect that many people running TASS cameras will not want
>to run local copies of the Postgres database software and will want
>a simpler program that works on flat ASCII files and runs under DOS.
>
>So I can see there being three types of TASS camera sites in terms of
>what they forward to the central database site
>
>1) Those that send in un-processed star lists as is currently done.
>2) Those that use a DOS based pre-processing program to perform a
>   locally computed match to the tass catalog
>3) Those with local Postgres servers.
>
>My question now is how many of each type site will there be?  I for
>one essentially have a #3 system now.  Is anyone else interested?
>#2) will require some programming and will likely not be implemented 
>until method #3 gets ironed out.  Method #1 is now in place.
>
>In guessing at the number of each, think about the Mk IIIs yet to
>come on-line and where the Mk IVs will likely go.
>
>+++++++++++++
>
>Now a farther off but related subject.  Once a dozen or so 2K chips
>get into productive use the current scheme of a comprehensive central
>database may not work.  The system I can see evolving will have a set
>of local databases, each holding local data.  The central database
>would do what I'll call "cream skimming".  It will hold only the
>"best" data from each site.
>-- 
>   --Chris Albertson             home: chrisa@wavenet.com           
>     Redondo Beach, California   work: chris@topdog.logicon.com
>
>
Glenn Gombert <gleng@infinet.com>