dB timings - RE: A Data Reduction Proposal

["Creager, Robert S" <CreagRS@LOUISVILLE.STORTEK.COM>]

More exact data is in the attachment.  It took a little over 1 hour to
import all the stars, and then a little over 6 hours to merge them (264000).

> -----Original Message-----
> From: Creager, Robert S 
> Sent: Monday, February 04, 2002 11:08 AM
> To: 'Chris Albertson'; tass@listserv.wwa.com
> Subject: RE: A Data Reduction Proposal
> 
> 
> 
> Chris,
> 
> For matching approximately 200,000 stars, using PostgreSQL 
> 7.2b2 (and vacuuming every 10,000 stars processed, maybe a 
> little excessive) on an AMD K6-2 500MHZ, 640Mb RAM, ATA-IDE 
> disks (slow machine by today's standards), it took me ~4 
> hours.  I can get exact timings and such when I return home 
> tonight.  I believe inserting the records (not copying) took 
> about 1 hour.
> 
> A little slow, but doable for a single site.  I figure a 
> new(er), server style machine might be able to keep up, 
> especially if only 'interesting', or a subset of data is sent 
> after initial crunching on the local machine.
> 
> I'd be happy to dump the dB and send out a CD.  Or, I can 
> send out (e-mail) the schema, scripts, trigger and star lists 
> (you'll have to supply the raw Tycho2 catalog, which a script 
> will populate the dB with).  Note that the dB is in rough 
> form as I wait till I have something working before I go back 
> and clean it up.  Notably, I use a trigger to update the 
> 'master' tables, but I have not added a pre-delete rule so 
> the tables cascade correctly.
> 
> Later,
> Rob
> 
> > -----Original Message-----
> > From: Chris Albertson [mailto:chrisalbertson90278@yahoo.com]
> > Sent: Monday, February 04, 2002 9:59 AM
> > To: Creager, Robert S; tass@listserv.wwa.com
> > Subject: RE: A Data Reduction Proposal
> > 
> > 
> > I tried that a while back and got very poor performance and
> > gave up.  That was usiing Postgresql 6.x  How fast are you
> > matches going?  What size data sets and could you describe the
> > computer you use?
> > 
> > This is really the best way to go if it works for TASS Mk IV
> > sized data but it was taking _days_ with Mk III data so I did
> > it in pure "C".
> >

2157.clip