Author: Michael Richmond Date: 970621 Revision: #0 970621 Key Words: photometry, techniques, computation
This is the first, preliminary version of this Note, which will probably grow for several weeks. I hope that other TASS members will contribute to its goal. Please read the plea for help at the end of the Note.
Table of Contents:
At the 109'th meeting of the American Astronomical Society, after I presented results of our analysis of TASS observations of Fields A and B, several people asked, "Why is there are floor of about 5% in your photometry?" One of them, Peter McCullough, participated in a very helpful discussion on the topic with Tom Droege, Michael Gutzwiller and myself the night before. The goal of this Note is to find out if there are any techniques we can adopt to improve the precision of our measurements, ideally down to the limit imposed by photon noise.
This Note, like Technical Note 31, focuses on a very small portion of the sky. I've chosen an area near RA = 13:33 (or 203 degrees), Dec = -1.0, because I have a set of images covering this portion of the sky. By applying different methods to extract magnitudes from these images, we can find out which method, if any, provides the most accurate results. The images are those described in Technical Note 25. Raw versions of the images can be found the archive of Test Data from Tom Droege's triplet, taken in Feb 1997.
How to find the "best" method of extracting magnitudes
Here's how I'll choose the "best" method of extracting magnitudes. Let us use notation I(i) to denote the i'th image in the set; if there are M images, then i = 1, 2, .., M. Furthermore, let us use the index k to count the stars in each image; if there are N stars in image 0, then k = 1, 2, ..., N. Suppose that there are three methods for extracting magnitudes: A, B, C. For each method, we do the following:
raw(A,i,k) = raw magnitude of star k in image i, by method A
raw(A,1,k), raw(A,2,k), ... , raw(A,M,k)
refer to magnitudes of the same star in different images.
mag(A,i,k) = raw(A,i,k) + z(i)
The idea is to repeat the above analysis for each different method A, B, C, etc., of extracting magnitudes. The method which yields the smallest deviation from the mean, for the vast majority of stars, is judged to be "best."
Note that this does not address the issue of placing the magnitudes onto the standard Johnson-Cousins UBVRI system. That is a matter which I shall not address in this Note.
What is inhomogeneous ensemble photometry?
For a good explanation of this method of analysis, see a paper published by R. Kent Honeycutt in Publications of the Astronomical Society of the Pacific, vol. 104, page 435 (1992). The basic idea is to combine measurements from a set of stars in a number of different images. Inhomogeneous means that stars don't have to appear in all the images to be included in the solution. Ensemble means that no single star is chosen as a reference; all stars are used to measure changes in the relative brightness of any particular star, but bright stars are given more weight than faint ones. The method requires an input star list which contains not only magnitudes, but estimated uncertainty in magnitudes, for each star.
I implemented this code for another purpose (the Sloan Digital Sky Survey), and so the routines are embedded in the SDSS software environment. It will not be easy for me to provide code to TASS members in any executable, although I can give copies of most of the source code (C and Tcl) to interested parties. One routine is part of a numerical library which may have restrictions on its use.
As stated above, I used images from the archive of Test Data from Tom Droege's triplet, taken in Feb 1997. Specifically, I chose 4 I-band images which included an overlapping area of about 1x2.5 degrees.
# name date time camera RA Dec sky skysig FWHM g0483967.fts 03-02-1997 11:13:48 0 I H 202.34 -1.90 -27918 55 3.4 g0493946.fts 13-02-1997 10:42:37 0 I H 204.18 -1.89 -28030 48 3.3 g2483931.fts 03-02-1997 10:20:42 2 I H 202.90 -0.79 -28869 45 3.8 g2493909.fts 13-02-1997 09:49:22 2 I H 204.71 -0.77 -28010 56 3.8
I removed the bias and created median flatfield vectors using these frames themselves, as described in Technical Note 25. However, I corrected my error in the process of creating the flatfields. I then subtracted the bias and divided by the flatfield vector.
Image g0483967.fts has quite a strong variation in the sky value from top to bottom (i.e. along 3 degrees of Right Ascension). This is probably due to clouds. In addition, the number of stars found in this image was much smaller than the others, only about 1000 versus 2000, another indication that clouds were passing overhead during its exposure.
What is the theoretical uncertainty in the photometry?
Suppose that we could use a perfect method to extract the magnitudes. There would still be some errors, due to the limited number of photons detected, and due to errors in determining the sky's contribution to the total.
One way to answer the question is to start from scratch and calculate the number of photons from a star of some magnitude M which enter the atmosphere, within the area of a telescope's aperture. One can estimate the number which are absorbed by the atmosphere, the amount to which they are spread out as the pass through the air and the telescope optics, the number which are detected by the camera, etc. Given some area on the focal plane, one can figure out the number of photons from the star which fall within the area, and so determine the irreducible uncertainty due to this finite number of events. One can similarly calculate the number of photons from the background sky which fall onto the same area, which in turn contributes a measure of uncertainty to the final extracted magnitude. One might call this the throughput approach. It's only as good as the input values you provide.
I have made such calculations for the case of a TASS camera operating in the I-band. Since these particular images were taken at Tom's site in Illinois, I guessed that the sky brightness would be I=17.5 mag per square arcsecond, one magnitude brighter than the value I measured in rural Vermont. The KAF-0400 has a QE of 17% at 8500 Angstroms, so I've estimated that the overall camera efficiency is only 15%. I have not included the extinction due to the Earth's atmosphere, which is probably about 10% -- one might just incorporate this into the overall QE numbers, I guess. I used a value for readout noise (20 electrons) measured by Tom Droege in Technical Note 7. Since the FWHM of these frames was about 3.5 arcseconds, I calculated numbers for the light falling within a circular aperture of radius=3.5 arcseconds.
Well, here is the output of my calculations, in the form of Signal-to-Noise as a function of magnitude. The uncertainty in a magnitude is roughly equal to the reciprocal of Signal-to-Noise; thus, if S/N=100, then the uncertainty is about 1/100 = 0.01 mag.
Table 1. Signal-to-Noise for TASS I-band camera
-----------------------------------------------------------------------------
mag 7.00: star 1721556.93 sky 816610 read 15203.16 -> S/N 1077.37
mag 8.00: star 685364.16 sky 816610 read 15203.16 -> S/N 556.42
mag 9.00: star 272848.39 sky 816610 read 15203.16 -> S/N 259.60
mag 10.00: star 108622.90 sky 816610 read 15203.16 -> S/N 112.01
mag 11.00: star 43243.55 sky 816610 read 15203.16 -> S/N 46.23
mag 12.00: star 17215.57 sky 816610 read 15203.16 -> S/N 18.68
mag 13.00: star 6853.64 sky 816610 read 15203.16 -> S/N 7.48
mag 14.00: star 2728.48 sky 816610 read 15203.16 -> S/N 2.99
mag 15.00: star 1086.23 sky 816610 read 15203.16 -> S/N 1.19
In the table above, the fourth column contains the number of electrons produced by the star which fall within the aperture of radius 3.5 pixels, the sixth column the number of electrons from the background sky, and the eighth column the number of electrons due to readout noise. The rightmost column contains the Signal-to-Noise ratio.
Based on this table, we expect that bright stars with I=8 should have uncertainties of only about 0.002 mag (less than one percent), while faint stars of I=12 should have uncertainties of about 0.05 mag.
Another way to esimate the expected uncertainty in a single magnitude is to use the quantities which are actually measured on a given CCD image itself. For example, suppose that a photometry program is measuring the brightness of a star. It must determine an appropriate sky level, plus the uncertainty in that value, using portions of the image far from the star. It must also determine the number of counts within some area of interest, directly underneath the star. If one gives the program the "gain" (conversion factor from counts to electrons) and the "readout noise", then it has all the information it needs to calculate a guess at the uncertainty of the magnitude measurement for the star. For details, read the very nice articles in
"Astronomical CCD observing and reduction techniques", edited by
Steve B. Howell, Astronomical Society of the Pacific (conference
series), 1992.
or, for a brief discussion, see my short paper on
A Brief Look at Guiding.
I used the phot program from the XVista software suite to measure aperture magnitudes in image g0493946. I set the aperture size to be equal in radius to the Full-Width at Half-Maximum (FWHM) of stars in the image, since experience has shown that this is a reasonable aperture size to choose. The program estimated the internal uncertainty of its magnitudes like so:
The curve indicates that the internal uncertainties are
The agreement here is pretty good --- evidently, I chose the right fudge factor for overall quantum efficiency in my calculations in the "throughput approach."
So, now we're ready to try different methods of extracting magnitudes from the images, and see how the external uncertainty compares to these estimates.
Method 1: Simple aperture photometry
I used the XVista program phot to perform simple aperture photometry on the test images. In each case, I summed the counts within a circular aperture around the center of the star. I experimented with three aperture sizes:
After extracting magnitudes, I matched stars up to the Guide Star Catalog to derive an astrometric solution for each image. Residuals from the fit ranged from 2.1 to 3.0 arcseconds. All stars were assigned (RA, Dec) positions based upon the solution.
Given this set of star lists, each with instrumental magnitude but calibrated position, I started up the massive machinery which performs the inhomogeneous ensemble solution. The output magnitudes have an arbitrary zero point. For convenience in comparing them with other methods, I added a constant to make the output magnitudes of 2 or 3 bright stars agree with their I-band magnitudes, as determined against Landolt standards in Technical Note 25.
The ensemble solution provided relative zero-point offsets z(i) between the test images (in which we force the offset of the first image to be 0.0):
Table 2. Relative offsets between images (mag)
aper=FWHM aper=FWHM-2 aper=FWHM+2
----------------------------------------------------------------------
g0483967.fts 0.000 0.000 0.000
g0493946.fts -0.034 +/- 0.023 0.053 +/- 0.046 -0.074 +/- 0.025
g2483931.fts -0.186 +/- 0.021 -0.263 +/- 0.047 -0.151 +/- 0.021
g2493909.fts -0.186 +/- 0.022 -0.278 +/- 0.037 -0.175 +/- 0.027
These values for z(i) are automatically added to the input magnitudes for each star by the ensemble solution, following the method describe above.
I then compared the output magnitudes for each star in all of the test images, calculating the mean value and the standard deviation from the mean. This is an indication of the external uncertainty in the magnitude measurements. I grouped the stars into bins one magnitude wide, so that all the stars in the I=7 bin are between I=6.51 and I=7.50, for example. Then I found the median uncertainty value within each bin, which I show below.
Table 3. External Uncertainty after Simple Aperture Photometry
I-band mag 6 7 8 9 10 11 12 13
-----------------------------------------------------------------------------
aper=FWHM
median sigma 0.009 0.013 0.018 0.020 0.028 0.048 0.090 0.137
N star 1 1 2 6 22 36 47 10
aper=FWHM-2
median sigma ... 0.080 0.012 0.026 0.043 0.058 0.105 0.184
N star 0 2 3 5 23 36 48 1
aper=FWHM+2
median sigma 0.002 0.009 0.014 0.015 0.031 0.076 0.127 0.183
N star 1 1 2 5 23 36 43 6
Here are plots of the external uncertainty versus magnitude, for each the three choices of aperture size:
Aperture radius = FWHM.
Aperture radius = FWHM - 2 pixels.
Aperture radius = FWHM + 2 pixels.
It's clear that the small aperture (radius = FWHM - 2 pixels)
gives inferior results, especially for bright stars.
On the other hand, the large aperture (radius = FWHM + 2 pixels)
may give slightly better results than the medium aperture
for bright stars, but does poorly on the faint stars.
It appears that the choice of radius = FWHM is a good
compromise.
Of course, to get the best results, one should really
measure the relationship between radius and cumulative
summed intensity (using a bright, isolated star), and
build a table of aperture corrections;
one can then choose an aperture for each star which is
just right for it (a small one for faint stars, a large one for bright stars)
and bring all the measurements to a common system with the
table of corrections.
However, that particular method of extraction is beyond the
scope of this Note.
One can also look at the ratio of the internal uncertainty
(calculated for each star as its magnitude was extracted,
as
described above)
to the external uncertainty
(based upon the scatter from the mean over the test images).
In an ideal world, without any sources of systematic error,
the two would be the same;
in the real world, we expect that the external uncertainty
will almost always be larger.
Here's a plot showing the two uncertainties,
internal on the X-axis and external on the Y-axis,
for each measurement of each star in the test set.
The graph shows a fairly good agreement between the two
estimates of uncertainty, especially for small values.
This indicates that there may be few sources of
systematic error between the data in the test set
(which, you must recall, all came from the same site).
I used the implementation of DAOPHOT in IRAF 2.10.4
to extract magnitudes from images in the test set.
Let me admit at the start that I did not spend a great
deal of time optimizing the parameters for each image;
my results may not be a fair representation of the best
DAOPHOT can do.
For each image, I ran the following procedures in order:
After extracting magnitudes, I matched stars up to the
Guide Star Catalog to derive an astrometric solution for each
image.
Residuals from the fit ranged from 1.7 to 2.3 arcseconds,
significantly smaller than the residuals using
XVista
positions.
All stars were assigned (RA, Dec) positions based upon
the solution.
Given this set of star lists, each with instrumental magnitude
but calibrated position, I started up the massive
machinery which performs the inhomogeneous ensemble solution.
The output magnitudes have an arbitrary zero point.
For convenience in comparing them with other methods,
I added a constant to make the output magnitudes of 2 or 3
bright stars agree with their I-band magnitudes,
as determined against Landolt standards in
Technical Note 25.
The ensemble solution provided relative zero-point offsets z(i)
between the test images (in which we force the offset
of the first image to be 0.0):
These values for z(i) are automatically added to the
input magnitudes for each star by the ensemble solution,
following
the method describe above.
I then compared the output magnitudes for each star in all
of the test images, calculating the mean value and the standard
deviation from the mean.
This is an indication of the external uncertainty in the
magnitude measurements.
I grouped the stars into bins one magnitude wide, so that
all the stars in the I=7 bin are between I=6.51 and I=7.50,
for example.
Then I found the median uncertainty value within each bin,
which I show below.
Here is a plot of the external uncertainty versus magnitude:
The very brightest stars are saturated, and so don't fit the
PSF well.
Fainter stars have consistently small uncertainties, as they
should.
One can also look at the ratio of the internal uncertainty
(calculated for each star as its magnitude was extracted,
as
described above)
to the external uncertainty
(based upon the scatter from the mean over the test images).
The graph shows that the estimates of
uncertainty from PSF-fitting were larger than
the external errors turned out to be;
this is probably a sign that the PSF varied
considerable across the frame, and/or that
I didn't do a very good job creating the PSF.
It's also possible that I made mistakes in setting the
readnoise and gain values before
I started running the DAOPHOT code (supplying
values much too large).
The purposes of this Note was to find out if there was
some way that we could improve the precision of
relative magnitude measurements from TASS images.
Let's compare several different values of scatter as
function of magnitude for I-band magnitudes:
One can note:
There are several reasons why the tests in this Note might yield
better results than those in Fields A and B:
Note that I have not carried the calculations to their final
point -- the magnitudes are still on an arbitrary zero point.
It is necessary to take two more steps to reduce the data fully:
The bottom line is that, for bright stars, I am unable to approach the
theoretical precision we ought to be reaching with TASS
images, even with this comparatively homogeneous subset
of images.
However, it is possible that we might be able to reduce
the scatter significantly from that derived in Fields A and B.
We need further tests to find out where the root of the problem lies.
It may be
It would be nice if several people could apply their own methods
for extracting magnitudes from the test images used in this Note;
if people could do so, and then send me their output star lists,
I could run them through my ensemble photometry code and calculate
the scatter as a function of magnitude.
We could then see if there are any significant differences between
many different methods of extracting magnitudes.
Back to Index of Technical Notes
Method 2: PSF-fitting with DAOPHOT
Table 4. Relative offsets between images (mag)
PSF-fitting
----------------------------------------------------------------------
g0483967.fts 0.000
g0493946.fts -0.066 +/- 0.032
g2483931.fts -0.098 +/- 0.024
g2493909.fts -0.086 +/- 0.022
Table 6. External Uncertainty after PSF-fitting Photometry
I-band mag 6 7 8 9 10 11 12 13
-----------------------------------------------------------------------------
PSF-fitting
median sigma 0.046 0.018 0.023 0.027 0.032 0.047 0.070 0.098
N star 1 1 2 6 24 41 56 3
Table 7. Scatter as a function of I-band magnitude
source 7 8 9 10 11 12 13
------------------------------------------------------------------------------
many nights, four sites
Field A (TN 31) 0.031 0.027 0.033 0.051 0.091 0.155 0.211
Field B (TN 31) 0.039 0.037 0.039 0.064 0.098 0.146 0.200
theoretical
throughput method 0.001 0.002 0.004 0.009 0.020 0.053 0.134
2 nights, one site
aperphot 0.013 0.018 0.020 0.028 0.048 0.090 0.137
PSF-fitting 0.018 0.023 0.027 0.032 0.047 0.070 0.098
This Note has addressed only possibility 2 above.
Back to TASS home page