RE: A Data Reduction Proposal

["Mark Pitts" <PITTS@shands.ufl.edu>]

Chris et al,

I think database selection is going to take some experimentation (assuming we use a database, which I believe is a good idea if it's feasible).  Database performance can be finnicky, but all my experience is with large commercial databases like DB2 and Oracle.  I've never used the open-source databases like Postgresql, so I can't speak (yet) as to how they might be optimized.

I'm thinking that perhaps we should derive our database design and routines that use (or will use) SQL calls independently of the platform.  Assuming we use standard SQL, we should be able to port the product to various database platforms fairly easily.  We might even do some benchmarking between the various options to select our "preferred" platform.

My plan for the moment is to wait for the arrival of the CD's that Tom has sent to me, and then acquire a better understanding of the data reduction process by getting the MR pipeline running.  

I like the idea of using something like SourceForge to help manage the project.  I'm registered at SourceForge, and, over the next few days, I'll review the requirements for registering a project like ours there.  They provide some very handy (and free!) services to support a project like ours.  I'll report back on my findings.  If anybody else knows of a potentially better alternative, please investigate it.

As to the Lone Wolf Development v. Collaborative Development discussion, I'm pretty open-minded.  I have to admit that I like working alone more than I like working in a group.  I'm presently managing a multi-disciplinary team here at Shands on a fairly large project, and there are many times when I long for the days when I just wrote my code and was left alone.  I don't see any reason why we can't get value out of both approaches.  For those who would prefer to continue to work on their own, by all means do so.  For those who want to work collaboratively, then let's proceed with a collaborative effort.  No matter which way each individual wants to contribute, we are all working toward the same goals, and each individual's contribution has value.

Mark

>>> Chris Albertson <chrisalbertson90278@yahoo.com> 02/04/02 11:58AM >>>
I tried that a while back and got very poor performance and
gave up.  That was usiing Postgresql 6.x  How fast are you
matches going?  What size data sets and could you describe the
computer you use?

This is really the best way to go if it works for TASS Mk IV
sized data but it was taking _days_ with Mk III data so I did
it in pure "C".



--- "Creager, Robert S" <CreagRS@LOUISVILLE.STORTEK.COM> wrote:
> 
> If anyone is interested, I have some Perl/SQL (PostgreSQL) which does
> #5 in
> Tom's list below for a single radius (.004 degrees is what I
> currently use
> along with at least 10 measurements).  Basically, all the radius
> matching is
> accomplished using PostgreSQL's circle_contain_pt and using a
> bounding box
> around the circle to gain index usage to minimize the stars to
> search.


=====
Chris Albertson 
  Home:   310-376-1029  chrisalbertson90278@yahoo.com 
  Cell:   310-990-7550
  Office: 310-336-5189  Christopher.J.Albertson@aero.org 

__________________________________________________
Do You Yahoo!?
Great stuff seeking new owners in Yahoo! Auctions! 
http://auctions.yahoo.com