[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Wild Speculation

--- Tom Droege <tdroege2@earthlink.net> wrote:

> one or more high points.  My dumb approach would be to compute sigma
> for 
> the measurements of each star and then look for 5 sigma high points. 
> I 
> could probably do this in a couple of days.  My guess from manually
> paging 
> through the data is that I might get a couple of hundred objects from
> my 2 
> million star list.


It's not a dumb approach . It's a very simple database query.  It
should take 30 seconds to find the 100 matches out of 100,000,000
observations.  Databases use methods like you'd use to find a 
name in a phone book, it's quick.  You don't need to read the
whole phonebook to find one name.  With todays PCs even 10E9
observations is not a lot to keep on-line.  Recently bought a
200GB disk for <$200.  and 1GB of RAM is now affordable.

Who is doing the on-line database? 

> Would anyone want to work on this data set if I make it?
> Even better, does someone want to write a perl script that will
> process the 
> data file and output a string or interesting star measurements?

Tom, It's a one-liner.  Almost an exact transliteration of what
you wrote.  Something like this:

     Select * from xxx where sigma(mag) > 10 and Nobs > 10
       Order by RA, DEC;

(Just plug in the correct names for the columns, I guessed in the
above example.)  It should print out a nice table sorted by ra and
dec.  The neat thing is that even if the data exists on a remote
computer only the "hits" have to travel over the slow Internet
connection.  Learning just a little SQL can save you from writing
huge amounts of Perl and it's orders of magnitude (literally) faster.

Chris Albertson
  Home:   310-376-1029  chrisalbertson90278@yahoo.com
  Cell:   310-990-7550
  Office: 310-336-5189  Christopher.J.Albertson@aero.org

Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.