Tom Droege mailed me a set of CD-Roms with data from his Mark IV camera, taken on Feb 11, 2000. I've been looking at the data for a few days. It seems logical to put my analysis in the Tech Note, rather than simply mailing it to Tom. So, here it is. I won't be finished with this data set for a long time -- treat this as a work in progress.
There are four CDs with data from Feb 11, 2000. They cover the period MJD 1586.686 to 1587.016, a stretch of about 0.33 days or 7.9 hours. There are images from two cameras, in V-band and I-band. The images come in sets of 4. Each set consists of images of the same area of the sky, with an exposure sequence of short-long-short-long. The short exposures are about 15 seconds, and the long exposures about 150 seconds. Logs show that each set takes about 11 minutes to complete.
Interspersed throughout the night are sets of four exposures, with the same short-long-short-long pattern, in which the shutter was not opened. These are "dark" images, and the focus of this first analysis. The dark sets occur every fifth set.
Following the helpful hints of Ted Woodhouse (thanks, Ted!), I found that the orientation of the images has changed since the last set I'd examined. These images are oriented as follows:
East (row 0)
-----------------------
| |
| |
| |
South | | North
(col 0) | | (col 2000+)
| |
| |
| |
-----------------------
West (row 2000+)
There is an offset between the RA and Dec reported in the FITS header, and the actual (RA, Dec) center of the image. It was very handy to find Procyon in image H3R1586.720! I find an offset of
true center of image RA = FITS RA - 7.3 degrees
Dec = FITS DEC + 3.0 degrees
Note the relatively large changes in water temperature near the start of the night, but small changes later on. This correlates reasonably well with the changes in temperature of camera 0 (which I think is the V-band). The other camera appears to have a rock-steady temperature. Any ideas, Tom?
As a passing note, one can see that 170 images from each camera, at 8 Meg per image, comes to 2.7 GBytes of data. That's a single night from a single site. Yikes!
One of the goals of my work is to find a good way to use the columns near the edge of each image to scale a master dark frame for optimal dark subtraction. Here's what I have found so far:
The V-band camera
In the V-band camera, the columns at the "low-col" edge (aka the South edge) have the following general characteristics:
In order to see how the values of the pixels in these edge columns compare to the values of pixels in the data area of the array, I defined four "boxes", all 50x50 pixels.
Now, for all the dark images taken during Feb 11, 2000, I calculated eight quantities: the mean value of pixels in each edge column (cols 0, 2, 4, 6) and the mean value of pixels in each box. In the following plot, these are shown as
The plot shows mean pixel value on the y-axis, and "dark image number" on the x-axis. "Dark image number" is a stupid kludge: it's simply an integer that rises each time another dark image is taken. Recall that the dark images are taken in sets of 4; there should be big gaps in time between every set of 4 data points on this plot. It covers the entire night.
Note a couple of things:
Here is one of those V-band dark images: this is H3R1586.734, a long dark exposure, at two different contrast displays:
One can see extra light in the upper-left (row = 0, col = 0) corner, and more in the lower-right (row = 2000+, col = 2000+) corner. There's also a very faint pattern of banding. The first few rows appear to have slightly higher values than the rest.
The I-band camera
One can perform the same analysis on the I-band camera, but the results are somewhat different.
In the I-band camera, I once again looked at some of the edge columns: cols 0, 2, 4, 6. I defined the same four "boxes" in the data area. For all the dark I-band images taken during Feb 11, 2000, I calculated eight quantities: the mean value of pixels in each edge column (cols 0, 2, 4, 6) and the mean value of pixels in each box. In the following plot, these are shown as
Note a couple of things:
Here is one of those I-band dark images: this is H4R1586.734, a long dark exposure, at two different contrast displays:
One can see extra light near the lower edge of the frame, which is probably driving the means in boxes C and D upwards. I don't see the same sort of banding as in the V-band camera. The first few columns appear to be higher than the rest.
The following section was written by Tom Droege. MWR
This will attempt to look at the causes of the dark current variation seen in the above data. We will concentrate on the I camera data where the temperature was held constant.
First some definitions:
Operation Uncertainty: Commands over the RS-232 link are of uncertain timing due to the workings of the Windows operating system as it interacts with QBasic. There may be an arbitrary delay in executing an operation of order 10 seconds. We will mark such operations with * in the code examples.
Exposure Time: The time between the start of the open command and the start of the shut command. Subject to uncertainty.
Read: The time it takes to read out the CCD after and exposure or dark exposure. This is done by a crystal controlled clock and the time depends on the details of the PROM programming that controls the read out process. It is very close to 47 seconds and fixed. The process involves transferring the charge from the CCd pixels, digitizing it, and transferring it to the memory in the Memory card in the PC.
Transfer: The time it takes to transfer the data stored in the Memory card by a Read operation to the hard disk. This is done by a QBasic program and is subject to the Windows operating system. Timing depends on whether other windows are open. The attempt is to take data with all other windows closed. This is a fairly constant number, but can change if Windows decides to do something. The time is about 64 seconds on my 200 MHz K6.
Clearing Scan: A Read operation where there is no attempt to save the data.
Shortest Dark Exposure: The shortest possible dark exposure is one taken by two successive scans. A Clearing Scan followed immediately by a Read. Since it takes 47 seconds to read out, a shortest possible exposure will mean that any one pixel has been subject to the accumulation of dark current for 47 seconds.
To understand the variation in dark current we have to look at the code that produces the short, long, short long exposure images. I have marked the lines subject to the "10 second" windows delay by an *. It does not happen often. But it can happen at any time. It could even happen several times in a row, but this is rare. Here is a simplified version of the software with the time for significant operations as (9) in seconds:
Init stuff * Clearing Scan (47) Do until FileNumber >= MaximumFileNumber * Fast Forward RA Drive (5) seconds to remove backlash * RA Drive to sidereal * Open Shutters (if not a dark frame) (1) Wait (15) seconds * Close Shutters (1) * Read (47) * Open Shutters (if not a dark frame) (1) Transfer (64) A or first short exposure Delay (70) * Close Shutters (1) * Read (47) Transfer (64) B or first long exposure * Open Shutters (if not a dark frame) (1) Wait (15) seconds * Close Shutters (1) * Read (47) * Open Shutters (if not a dark frame) (1) Transfer (64) C or second short exposure Delay (70) * Close Shutters (1) * Read (47) Transfer (64) D or second long exposure * Stop RA Drive (1) * Go Back to Starting RA position (35) LoopWe can now add up the time that a pixel accumulated dark current for the various exposures, A short, B long, C short and D long.
Note that dark current accumulates between reads:
A The Very first short 69 Seconds B All first longs 183 Seconds C The second short 128 Seconds D The second long 183 Seconds A All other first short 169 Seconds
We can now look at the curve of dark current. We see that the very first short exposure has the least leakage current. It also has the shortest leakage time. This matches the curve data. We see that B and D, the second and forth exposures have the same leakage time. They tend to have similar leakage on the curves. After the initial frame, the short exposure C has the shortest leakage time. It is also consistently lower on the curves.
Some comments:
Contrary to an earlier statement, there are no big gaps in the data. The data is slightly irregular as indicated by ABCD timings above. The data was taken as one continuous run. (Hmmm! possibly this is not correct. It may have been taken as two runs with a break. The log book is not clear. Sigh!)
The code example above was written to optimize the time taken for an exposure at the expense of regular timing. We overlap the time taken to read out the Memory Card to the hard disk with the long exposure. We can't do this for the short exposure though.
The two cameras used for this run do not have heat sink grease between the CCD and the spacer block that contains the thermometer. Sigh! I just forgot and then glued them together with epoxy. It is the only way I can keep them tight to keep out moisture. The CCDs are thus at an unknown temperature. The V camera probably has some problem in the thermal control loop. There has been a problem there from time to time, it has not yet worked it's way to the top of the work list.
The leakage current seems high for this device. The scale factor is roughly 3 e- per ADU. The difference between C and D exposure is 53 seconds. In ADU this is estimated at 80 from the curve. At 3 e- per ADU, this is 4.5 e- per second per pixel. The data sheet says 350 e- per second per pixel in MMP mode at 25 C. (27,900 per second non MPP) If the current is reduced by a factor of two for each 5 C, then the measured dark current indicates a temperature of roughly -7 C vs the -22 C read by the thermometer in the block on the other side of the air gap. This might even be correct. The factor of two for 5 C is just rough. It might be 6 or 7 C. This would bring the result even closer to the expected value. Without thermal grease, there is a big heat loss in the gap. We are clearly operating in MPP mode or the dark current would be much higher.
Let us suppose that I can't improve the dark current. For a 183 second dark current (the sky exposure will be about 50 seconds less) exposure it might be 80/53 * 183 = 276 ADU = 828 e- This is a noise level of 28 e- or 10 ADU from the leakage current. Not great, but the sky that I see is always much more than this.
I combined dark frames to make a "master" dark frame, which I will use for further reductions. I used all the dark images from Feb 11, 2000. First, for all V-band (or I-band) images, I calculated the mean value in a box near the center of the frame: box center at (1000, 1000) and 500 pixels in size. Then, I scaled the darks so that they all had the same mean value inside that box (it required very little scaling in most cases). Finally, I created a "master dark" frame by taking the median of all the pixel values at each (row, col) position; I used 31 frames as input to this median procedure.
When I looked closely at the distribution of pixel values in these "master" frames, I discovered the same old pattern of multiple peaks (just as was seen in Mark IV images from months ago). Here are the graphs of the pixel distribution for the V-band master dark:
and here for the I-band master dark:
I used the "master dark" frames as follows:
Whoops. I should have used the average of columns 2 and 4 for images in the second half of the night.
To make flatfield frames for this set of data, we have no choice but to combine images of the night sky. Exposures in each filter come in two varieties: 15-second exposures and 150-second exposures. I think it's best to use the 150-second exposures only, since the signal from the sky will be much larger in them. Recall that the exposures are gathered in sets of four for each pointing: short, long, short, long. I checked to see if the two long exposures were pointed at exactly the same location; and, indeed, they were. That's bad, because it means that stars fall on exactly the same pixels in each exposure -- which makes combining the two less effective when making a median flatfield frame. So, I decided to use only the second long exposure from each set of 4 images.
The pixel values in a box near the center of the frame varied quite a lot during the course of the night. Here is a plot of those values for the V-band frames used in the flatfield creation:
The upper panel shows the mean pixel values. Note that there are large excursions from a lower limit of about 2200 DN. I looked at the frames with high values, and saw a different pattern of illumination; some had streaky features. I conclude that the frames with high sky values were taken while the sky was cloudy.
To create the "master flat" frame, I threw out all images with high values, leaving 15 frames in the V-band which looked reasonably clear. I subtracted a shifted dark frame from each (as described above). Then I created a median flatfield frame by
Here's a picture of the resulting flatfield frame: the contrast varies from 2400 DN for white, up to 2700 DN for black.
The two "seagull" features in the upper left are due to ice crystals on the CCD. I don't know what causes the linear streak down the left side, but I suspect some sort of contaminant on the lens ... or possibly glare from some light off to the side of the camera?