RE: TASS Database, versions > 1.0

["Gutzwiller, Michael" <mgutzwiller@lanvision.com>]

I'm currently not interested in adding Linux and Postgress on my already
loaded systems so I'm leaning towards option 2 myself.  In fact the code
to do the matching is already present in the ip software library used by
the star program.  The effort to generate a Windows based program to
pre-process the star lists should not be too large and could even be
added to the starpost program.

Mike G.


	-----Original Message-----
	From:	Chris Albertson [SMTP:chrisa@wavenet.com]
	Sent:	Friday, April 03, 1998 1:38 AM
	To:	tass@wwa.com
	Subject:	TASS Database, versions > 1.0

	(This is another opinion poll, see question in last couple
	paragraphs above +++++)

	I am still working, slowly, on my "version 1" database software
	but I'm looking ahead to version two and the MkIV camera system.
	I just tonight got the catalog seeding operation to run at
	2000 records/second, so the whole operation took only about
	1200 seconds and nothing crashed. 

	I am finding that it takes a very long time to do astrometric
	matching when your lists are numbered in the millions of rows.
	The current system which will be continued in version 1 is that
	camera operators FTP their lists to the ftp.tass-survey.org site
	where they are merged into a central database.  This puts a lot
	of load on that one central site.  What we could do (in a future
	version two system) is this:  

	  (1) Each camera site periodically downloads the "tass
catalog".
	  this is just three columns, ID Number, ra, dec.

	  (2) they save up several nights or weeks of star lists then
run
	  the whole batch through a program that matches each list to
	  the downloaded catalog and tags each observation with a
tass-ID.
	  For observations not matched to catalog entries the catalog is
	  extended.  Finally the starlist is sorted by tass-id then
FTP'ed
	  along with the extended portion of the catalog to the normal
place.

	This pre-processed data would be much faster to process into the
	database system.  Fast by a very large amount, maybe 4x to 10x.
	Unprocessed star lists could still be sent and they could be
imported
	to the database by the current method.

	The above will require new software at the camera sites.  Or
will it?
	+IF+ (big if) the people running those cameras would install the
same
	database software suite the Michael has installed on his
computer then
	they could maintain a local database of local observations.
They
	could do their own local matching using existing database
software.
	Periodically they would produce dumps that are FTP'ed to the
central
	database.  From my point of view this is the best solution as it
	requires very little more work on my part.

	I strongly suspect that many people running TASS cameras will
not want
	to run local copies of the Postgres database software and will
want
	a simpler program that works on flat ASCII files and runs under
DOS.

	So I can see there being three types of TASS camera sites in
terms of
	what they forward to the central database site

	1) Those that send in un-processed star lists as is currently
done.
	2) Those that use a DOS based pre-processing program to perform
a
	   locally computed match to the tass catalog
	3) Those with local Postgres servers.

	My question now is how many of each type site will there be?  I
for
	one essentially have a #3 system now.  Is anyone else
interested?
	#2) will require some programming and will likely not be
implemented 
	until method #3 gets ironed out.  Method #1 is now in place.

	In guessing at the number of each, think about the Mk IIIs yet
to
	come on-line and where the Mk IVs will likely go.

	+++++++++++++

	Now a farther off but related subject.  Once a dozen or so 2K
chips
	get into productive use the current scheme of a comprehensive
central
	database may not work.  The system I can see evolving will have
a set
	of local databases, each holding local data.  The central
database
	would do what I'll call "cream skimming".  It will hold only the
	"best" data from each site.
	-- 
	   --Chris Albertson             home: chrisa@wavenet.com

	     Redondo Beach, California   work: chris@topdog.logicon.com