Re: Data Cleaning

[Stupendous Man <richmond@stupendous.cis.rit.edu>]

  Arne wrote:

> If you remember way
> back when, I posted software for the Mark III system that would
> do inhomogeneous ensemble photometry for variables.  It would seem
> that the logical thing to do for those stars that are obvious
> variables in the Mark IV dataset is to forget about the frame-to-frame
> zeropoint determination, which makes mistakes, and just do local
> calibration for those stars using such an ensemble approach.  Has
> this been done?  Get decent light curves and worry about absolute
> photometry later.

  Tom asked:

> I don't understand what you are asking below or how it is relevant?

  Let me try to clarify things.  There are (at least) two main uses
for the Mark IV data.

     a) generating a catalog of lots of decent V and I magnitudes
             for stars all over the sky -- is good for reference

     b) seeking variable stars

  The current pipeline concentrates on the first goal.  It treats
each night's measurements independently; to a large degree, it
treats the data from each _image_ independently.  A large network
of Tycho2 measurements are used to turn the instrumental magnitudes
from each image into (nearly) standard V and I magnitudes.

  There are several systematic errors which can creep into this
current dataset.  Example 1: if there are small flatfielding
errors across the image, all the stars in the upper-right corner
may end up 0.03 mag fainter than they should be.  On another night,
if these stars happen to fall in the lower-left corner of the
field, they may all be set 0.02 mag BRIGHTER than they should be.
Example 2: if broken clouds pass over during an image, stars on
one half of the field may have small offsets compared to the
other half.  

  If one gathers 30 or 50 measurements of each star -- as Tom is
doing -- then on AVERAGE, all these errors will tend to cancel,
and the _average_ magnitudes of the stars will be pretty good.
No big deal for a catalog.

  However, if one's interest lies in finding variable stars, then
these small systematic errors are a pain in the neck.  They cause
an individual star to jump up and down by small amounts from
one night to the next.  If one looks only at the measurements
of that one star, one might think it was varying intrinsically.

  Arne is pointing out that there is an additional step in
the processing which can improve the search for variables.
If one grabs all 30 or 50 measurements of a small group of 
neighboring stars -- say, all those within 0.5 degree of each
other -- and looks at them together, then one will see that
on Oct 23, ALL the stars were 0.03 mag fainter than average .... and on
Oct 25, ALL the stars were 0.02 mag brighter than average.
Aha!  Once you know this, you can adjust the magnitudes for
all the stars on these nights to correct for these errors, and
THEN calculate statistics to look for variables.  This technique
of examining a small group of stars together is called 
ensemble photometry.

  Arne has written code to do the work for Mark III measurements.
You can find it here:

      http://stupendous.rit.edu/tass/software/software.html#collate

I have my own code which can do the same thing, but I haven't
built a framework to 

         a) grab a bunch of measurements from the database
         b) run the code 
         c) stick the results back into a database

  This is an open area for anyone to jump in.  The result of this 
analysis would not REPLACE the current database; it would be an
additional data product, which would be used by variable star
fans, but not by Joe Average Astronomer who just wants to know
how bright that star next to his pretty galaxy is.

  A good reference:

     http://adsabs.harvard.edu/cgi-bin/nph-bib_query?
         bibcode=1992PASP..104..435H&db_key=AST&high=3d7610c00c23593


                                         Michael Richmond