During the past month, several people have noted discrepancies in the dates and/or times stored with the measurements in the Mark IV database. I decided to check the Mark IV dates and times by using asteroids: if a known asteroid appears in the Mark IV database at the position predicted by its ephemeris for the time stored in the database, then that time is correct.
Summary: the was an error in the Mark IV pipeline which caused all JD values after Feb 29, 2004, to be in error by -1.0 days (i.e. the Mark IV values are 1.0 day too small). I have fixed the error and sent a new version of the software to Tom Droege, so that future Mark IV reductions will have the proper JD. I suggest that Michael Sallman run a script on the Mark IV database to fix the JD values for all data taken after Feb 28, 2004 = 2453064.5.
How to track down asteroids in the Mark IV database
I wrote a script to do the job, since I wanted to
run through a very large amount of data.
Since Tom distributes the Mark IV data in
chunks of roughly one month on a CD-Rom,
I arranged to check one chunk at a time.
I don't claim that this is in any way the
best way to look for asteroids, but it works
for me.
Note 1: I used a program built on the libnova library of astronomical routines to calculate the rough position of each asteroid. Spot checks indicate to me that the accuracy of these positions is something like one arcminute. That is good enough to pick asteroids which fall within the current chunk's rectangle on the sky, but not good enough to verify that any particular detection is an asteroid ....
Note 2: To be sure that the asteroid positions were accurate, I used the JPL Horizons system to create an ephemeris at 6-hour intervals, and then interpolated within each interval to the exact time of any particular event. I could get only very slightly better accuracy by making an individual request to Horizons at the time of each Mark IV detection (better by a few arcseconds), but that would entail an order of magnitude more requests to the Horizons server. I thought that would be unkind (and perhaps unwise).
Let me provide an example of the output of my script, so that you can see the information I will use to evaluate the accuracy of Mark IV times and dates. Here is the output for one particular night:
night tr1/Mhra2608879.list diff 2452608.87900 2452609.52438 = -0.65 match 39 2452609.94728 0.00 -0.8 0.5 0.9 10.89 -0.26 match 39 2452609.95836 0.00 0.0 0.1 0.1 10.88 -0.27 match 41 2452609.95472 0.00 1.6 -2.0 2.5 12.32 0.13 match 45 2452609.64628 0.00 -0.7 0.3 0.7 12.57 0.50 match 86 2452609.61122 0.00 -0.7 -0.6 1.0 12.74 0.17 match 119 2452609.57799 0.00 1.5 3.6 3.9 12.37 -0.29 match 121 2452609.62411 0.00 -0.9 -0.9 1.3 12.09 0.22 match 121 2452609.63517 0.00 -0.2 -0.8 0.8 11.96 0.09 match 129 2452609.72383 0.00 -1.2 -1.7 2.1 12.23 0.13 match 201 2452609.59833 0.00 2.7 -3.8 4.7 12.38 0.05 match 287 2452609.92696 0.00 -2.3 3.8 4.5 12.15 0.10 match 409 2452609.59449 0.00 -2.2 -0.1 2.2 11.90 0.19
The first line describes the night: the dataset tr1 contains nights from November and December 2002, and this particular night has a code Mhra2608879. The 'h' means this was camera TOM1, which scans near the celestial equator. The digits indicate -- in an unfortunate manner -- the rough date of the night: the first four digits '2608' are the final 4 digits of the integer portion of the Julian Date at the start of the measurements, and the following three digits '879' are the beginning of the fractional part of this Julian Date. Thus, we would expect that this night was
JD 2452608.879
However, there is a one-day difference between this date
and the actual Julian Dates recorded on this night;
note that the first detection occurs at
match 39 2452609.94728 0.00 -0.8 0.5 0.9 10.89 -0.26
^^^^
I can't remember exactly how this one-day difference occurs; it may have something to do with the one-day difference between calendar date and Julian Date at the start of twilight in Illinois. However, please note that this difference is not significant, as it occurs only in the filename which holds the Mark IV measurements for a night. The measurements themselves have a proper Julian Date (in this example), and it is the proper date which is stored in the Mark IV database. So this is not relevant to the question, "is there a date/time error in the Mark IV database?"
Each subsequent line of output describes one possible match between an asteroid and a detection in the Mark IV database.
match 39 2452609.94728 0.00 -0.8 0.5 0.9 10.89 -0.26
The columns are:
This particular line indicates that one Mark IV detection does happen to match the position of a known asteroid at exactly the date and time listed in the Mark IV database. In other words, this match suggests (strongly) that there is no error in the time recorded for Mark IV measurements made by this camera on this night.
The entire list of matches for this night show that there were 10 different asteroids which match Mark IV detections, two of them seen on different Mark IV images. This indicates even more strongly that there is not any significant date or time error in the Mark IV database for this camera on this night.
How large an error in times might there be, even if an asteroid appears in roughly the right place? A typical main belt asteroid will move by 10 to 70 arcseconds per hour when it is within several hours of the meridian at midnight. My criterion for a match is that the detection and ephemeris agree to within 15 arcseconds, so any error in time of even one hour would lead to few matches. However, errors of a few minutes would be too small to detect by this method (one might look for a systematic error in RA for all the asteroids in a particular night, but I doubt that would turn out well). Note that the typical precision of a single Mark IV (RA, Dec) measurement of a faint star is about +/- 1 arcseconds (see Accuracy of individual positions in Tech Note 85).
Note 3: since people reported some instances of one day errors in the time recorded in the Mark IV database, I made a special check in my script: in addition to comparing each Mark IV detection to the position of a candidate asteroid at the exact time and date of the detection, I also considered the position the asteroid exactly two days before the database time; exactly one day before the database time; exactly one day after the database time; and exactly two days after the database time. Thus, I would catch errors of off-by-exactly-one-day or off-by-exactly-two-days.
Interested parties can download the output files produced by my script:
A brief digression on asteroid photometry
As a check on my method, I decided to look at the accumulated photometry of an asteroid. Asteroid (71) Niobe was detected 64 times in the datasets TR1 through TR16, corresponding to the period Aug 5, 2003, to Dec 12, 2003. A plot of V-band magnitude versus time shows a brightening near opposition and subsequent fading:
The catalog of asteroid light curve parameters lists Niobe has having a period of 14.38 hours = 0.59917 days. If we phase the Mark IV measurements with this period, we see a nice double-peaked light curve:
The amplitude of this phased "light curve" is much larger than the value of 0.12 mag listed in the catalog because I have combined data taken at different places in its orbit.
This clean photometric light curve adds weight to the evidence from the positions alone that the Julian Date values in the Mark IV database during this period have no errors.
Those who wish to use asteroid measurements made by the Mark IV may find the following information handy: it is a simple ASCII text file with two columns: the first column is the number of times an asteroid was detected, and the second is the ID number of the asteroid.
Which dates are good and bad?
So, what do the asteroids in the Mark IV database tell us?
I'll list the results by dataset.
The columns in the table below mean
set cam span num_in_area nights good bad unknown -------------------------------------------------------------------------- tr1 TOM1 Nov02-Jan03 58 20 19 0 1 tr2 TOM1 Nov02-Apr03 39 35 30 0 5 tr3 TOM2 Apr03-May03 12 21 12 0 9 tr4 TOM1 Apr03-May03 34 24 20 0 6 tr5 TOM3 Apr03-May03 0 19 0 0 19 tr6 TOM1 Jun03 26 15 11 0 4 tr6 TOM2 Jun03 2 15 5 0 10 tr6 TOM3 Jun03 0 15 0 0 15 tr7 TOM1 Jul03 10 1 1 0 0 tr7 TOM2 Jul03 68 6 0 0 6 tr7 TOM3 Jul03 0 8 0 0 8 tr8 TOM1 Aug03 30 17 16 0 1 tr8 TOM2 Aug03 18 20 15 0 5 tr8 TOM3 Aug03 1 19 0 0 19 tr9 TOM1 Sep03 69 23 22 0 1 tr9 TOM2 Sep03 26 23 18 0 5 tr9 TOM3 Sep03 1 22 0 0 22 tr11 TOM1 Oct03(late) 16 16 15 0 1 tr12 TOM2 Oct03(late) 13 13 11 0 2 tr13 TOM1 Oct03(early) 14 13 12 0 1 tr13 TOM2 Oct03(early) 4 13 12 0 1 tr14 TOM3 Oct03 1 12 3 0 9 tr15 TOM1 Nov03 45 8 8 0 0 tr15 TOM2 Nov03 32 8 7 0 1 tr15 TOM3 Nov03 2 2 0 0 2 tr16 TOM1 Dec03 48 3 3 0 0 tr16 TOM2 Dec03 31 3 3 0 0 tr17 TOM1 Mar04 22 8 0 4 (-1) 4 tr17 TOM2 Mar04 14 7 0 2 (-1) 5 tr17 TOM3 Mar04 0 7 0 0 0 tr18 TOM1 Apr04 18 15 0 10 (-1) 5 tr18 TOM2 Apr04 13 16 0 9 (-1) 7 tr19 TOM3 Apr04 0 17 0 0 17 ------------------------------------------------------------------------
Starting in Oct03, the data taken on each night during Oct03 was written to CD-Rom in two halves: measurements made during the first half of the night (denoted by "early" in table above), and measurements made during the second half of the night (denoted by "late"). I stopped indicating this in the table after Oct03.
The lone bright asteroid which fell into the TOM3 area in Oct03 was (925) Alphonsina, which has an orbital inclination of 21 degrees.
Note that the only measurements which can be shown to have bad dates are those in March and April 2004. The JD stored in the Mark IV database is off by one day from the true date given by the asteroid ephemeris, in the sense:
(Mark IV database JD) = (true JD) - 1.0
For example, asteroid (119) Althaea matches an entry in the
Mark IV database at position RA=126.5359, Dec=+11.8200.
The asteroid has this location
The measured positions of asteroids point out that there is an error in the Mark IV data. They suggest that all Mark IV Julian Date values are off by one day starting in March, 2004.
Note that a number of people came to a similar conclusion by examining the phased light curves of variable stars; see the TASS E-mail lists for the discussion.
Given this evidence, I suspected that the problem might be an error in some piece of software involved in calculating the Julian Date: specifically, portions of the software which account for leap years. Note that Feb, 2004, was a leap month. After walking backwards through the data path in the Mark IV pipeline, I found the source of the problem: a pair of routines in the TCL file make_list.tcl, part of the Mark IV pipeline which reduces the raw Mark IV images into lists of calibrated detections. I wrote this code, so the fault is mine. Sigh.
I can provide full details upon request via E-mail, but a summary is:
(pipeline JD) = (true JD) - 1.0
In other words, the code failed to count February, 2004, as having a leap day.
One sign of this error is that the difference between the JD encoded in the name of pipeline output files and the JD values inside those files (see the discussion above ) disappears. Instead of the filename values being one day smaller than the JD values inside, the filename values become equal to the JD values inside.
After finding the errors, it was a matter of an hour or so to fix them. I re-wrote the portions of make_list.tcl and tested by hand the JD calculations. Over the period 1950 to 2101, the code now produces the right JD values.
I cut a new version of the pipeline, version 0.5, which includes this corrected make_list.tcl. It appears on TASS pipeline download page together with older versions. Tom Droege kindly downloaded this new version and ran it on images taken in 2002 and in 2004. He reports that the JD values it creates are correct in both years.
Note that one can track this change in the data produced by the pipeline. One of the three output files created for each camera and each night is a summary of all the images taken that night, called something like Mhra3096550.list. The first few lines of this file look something like this:
# software {pipeline 0.4} {match 0.7} {photom 0.6} {xvista 0.1.2} {bait 0.1}
# run on Fri Apr 2 06:56:05 CST 2004
hira3096520.fits I 2453096.51977 0.0 object 105.6 -9.2 0 0.0 0.00 0 99.000 99.000 1
hira3096521.fits I 2453096.52159 90.4 object 106.3 -4.0 24098 8108.40 0.00 0 99.000 99.000 0
Note that the very first line lists the version of each package within the software which reduces the data. In the example above, the "pipeline" is listed as being version 0.4. Any version less than or equal to 0.4 contains the error which causes JD after March 1, 2004 (JD 2453065.5) to be in error by -1.0 days.
I suggest that there is no need for Tom to re-run all the images taken since March 1, 2004, through the entire reduction process. Instead, I suggest that those who have copies of the data simply write a little script to modify all JD values after 2453065.5, increasing them by 1.0 days. In particular, I suggest that Michael Sallman make a new table in Mark IV database which contains data with fixed JD values after 2453065.5, and that, after a few tests, he modifies the nice database access code to refer to this new table. (Note that I do not suggest destroying the current tables, yet).