[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Wild Speculation
--- Tom Droege <firstname.lastname@example.org> wrote:
> one or more high points. My dumb approach would be to compute sigma
> the measurements of each star and then look for 5 sigma high points.
> could probably do this in a couple of days. My guess from manually
> through the data is that I might get a couple of hundred objects from
> my 2
> million star list.
It's not a dumb approach . It's a very simple database query. It
should take 30 seconds to find the 100 matches out of 100,000,000
observations. Databases use methods like you'd use to find a
name in a phone book, it's quick. You don't need to read the
whole phonebook to find one name. With todays PCs even 10E9
observations is not a lot to keep on-line. Recently bought a
200GB disk for <$200. and 1GB of RAM is now affordable.
Who is doing the on-line database?
> Would anyone want to work on this data set if I make it?
> Even better, does someone want to write a perl script that will
> process the
> data file and output a string or interesting star measurements?
Tom, It's a one-liner. Almost an exact transliteration of what
you wrote. Something like this:
Select * from xxx where sigma(mag) > 10 and Nobs > 10
Order by RA, DEC;
(Just plug in the correct names for the columns, I guessed in the
above example.) It should print out a nice table sorted by ra and
dec. The neat thing is that even if the data exists on a remote
computer only the "hits" have to travel over the slow Internet
connection. Learning just a little SQL can save you from writing
huge amounts of Perl and it's orders of magnitude (literally) faster.
Home: 310-376-1029 email@example.com
Office: 310-336-5189 Christopher.J.Albertson@aero.org
Do you Yahoo!?
The New Yahoo! Search - Faster. Easier. Bingo.