Table of Contents
The purpose of this Note is to document which programs are in the "software chain" that captures and processes Mark III CCD images into star lists; any data parameters or hardware conditions relevant to that software and data; and where to obtain that software. Much of this is documented "by site" to represent practices at each Mark III camera site. This Note also has information on the Mark III datasets that make up the "TASS database" as compiled by Michael Richmond as of June 1999, and from which Richmond's TenXCat catalog was generated. The TenXCat catalog is documented by Tech Note 56 (see Web references). This TN references Mark III data not represented in that database, and mentions a "previous" TASS database. This Note will hopefully be updated as "new" data is added to TASS/Richmond's Mark III database of collective observations; or as new Mark III sites or databases come on-line.
This Note may reference (but does not document in detail) the databases of individual TASS members, such as Glenn Gombert's database from which he is reporting on variable stars. This Note mentions some additional programs that MAY be useful for Mark III programs: details of those programs would be found at the Web sites referenced. This Note does NOT cover Mark IV developments or (upcoming) data or databases: a corresponding TN would be encouraged. This Note is intended to cover ACTUAL PRACTICES rather than "possible" practices (with some exceptions) at the Mark III camera and data reduction sites. As such, it also covers any loose ends or unresolved issues (in the author's opinion) in the Mark III data collection and processing process. One of those loose ends is about date and time information in the Mark III image FITS header: I have a separate Tech Note on that subject, Mark III time references and VCO rates.
Current information on ANY TASS program or data is best found from the Web sites operated by the originator of that program or data, or reviewing the information on the TASS Web page at tass_survey.org, or by contacting the originator directly (in that order, in my opinion).
As all this information was distributed across several Web sites, three years of TASS correspondence, or was not publicly discussed, I determined it would be useful to gather it into a single TASS Tech Note for general reference, discussion and documentation. In addition, this Note tries to compliment Richmond's Tech Note 56 on the TenXCat catalog.
Acknowledgements: My thanks to all the Mark III camera owners, their Web site "masters", and to the TASS authors of software for their correspondence and corrections. Nick Beser, Chris Albertson, Michael Richmond, Michael Gutzwiller, Arne A. Henden, Glenn Gombert, among others, were particularly helpful and forthcoming with details of their work. My thanks to Michael Richmond for previous discussions of Mark III data which led me to compile this information. He found an earlier version of this Note "very useful" for his Tech Note 56 and related presentations. Any reporting or editing errors in this Tech Note are of course my responsibility. Any omissions are not intended: some information was not available to me. Corrections and updates to this information are very much encouraged.
The "current" TASS Mark III database (of June 1999) consists of processed observations from image data collected by TASS Mark III cameras, situated at a number of sites. The Mark III database is managed and maintained by Michael Richmond. Tech Note 56 Description of TASS TenXCat catalog by Richmond, describes the basis of that catalog and of the Mark III database at that point.
There was a previous database effort that produced a similar database of observations from sets of Mark III image data. Discussion about and analysis of that database led to improvements and refinements in selecting and processing the image data. Consequently, some (but not all) of that previous data was "reprocessed" into the current database starting in 1998, as documented in Tech Note 43 Recalibration of measurements in the TASS database. The earlier database was abandoned but the image data that was not "reprocessed" is still held by the camera owners and may yet be added to the current database. See the Mark III site notes in this TN for information on those data sets by site.
TASS Mark III Sites, Camera Features, and Data Contributors
At this time (July 1999) there are three Mark III contributors to the current TASS database of Mark III data from which Michael Richmond compiled his catalog "TenXCat" in June 1999:
* Glenn Gombert triplet in Dayton, Ohio
* Michael Gutzwiller triplet
in Cincinnati, Ohio (latitude 39.2001, longitude 84.5828)
* Tom Droege triplet in Batavia, Illinois
The technical features of the Mark III camera can be represented by this bit of TASS email from Gutzwiller:
From: mgutzwiller@lanvision.com
To: tass@wwa.com
Subject: RE: more details of signal-to-noise calculations for Mark IV
Date: Thu, 29 Oct 1998 11:58:55 -0500
[snip]Mark III
sky noise (adu): 50 (per Tom's and my experience)
sky noise (e-) 125
mag 8.5 star (e-) 250000 (as seen on average in my analysis)
limiting magnitude (3 sigma): 13.5 (takes into account the PSF of the star)
sky charge (e-): 15625
pixel size (arc sec): 13.75
charge/arc sec2 (e-): 82.64
sky mag equivalent (/arc sec) 17.2 (matches Michael's guess pretty well)
exposure (s) : 470
charge/arc sec2/sec (e-/s) : 0.176
aperture (cm): 4.82 (135mm lens at f2.8)
sky rate (e-/arc sec2/sec/cm2) : 0.0118
[From a previous emailed calculation of the above:] The [previous] limiting magnitude calculation [for the Mark IV, not shown here] assumes the PSF is the same as the Mark III at the _pixel_ level (fwhm ~ 2.5 pixels) which means that the focus is really twice as good. The second calculation assumes the focus is twice as good again (fwhm ~ 1.25 pixels).
The positional basis for the TenXcat database is the USNO-A 2.0 catalog subset
provided by Arne Henden, which is called "tassm16" for short to seed
the catalog before we begin importing stars. (From correspondence with Richmond,
Aug 1999). Refer to the TenXCat Tech Note for details.
Other sites that have, will or might contribute, to the TASS database with Mark III data are:
* Nick Beser et al, at the JHU/Applied Physics Lab in MD:
"We have sent Mike Richmond [in July 1999] a CD-ROM with 73 nights of [reduced] observations on it. (1 years worth of data). Since he just got it, he has not [yet] added it to the data base. Prior data [for the previous database] was sent before the starpost and flatcomp program was available." (Nick Beser, private communication. See my notes about his site for details.)
* Michael Richmond in Rochester, NY (formerly camera was in Vermont)
He has collected and processed some data, his data is not in the database as of July 1999.
* Chris Albertson, Redondo Beach CA, singlet camera (V filter) "I"
His camera, provided primarily to support software development, is in occasional use. No data supplied to the TASS database as of July 1999. He has some datasets that were processed by Ted Woodhouse using the star programs, which will be provided to Michael Richmond for the Mark III database sometime after Aug 1999.
* Herb Johnson, rebuilding a damaged Mark III camera, no data collected as of Sept 1999.
The TASS Web page will likely have a list of Mark III sites and references to their Web pages (if any): check under "sites". The same will apply to any changes to the current Mark III/Richmond database, catalogs, or other databases.
The programs that are used by most TASS Mark III sites to process their data for contribution to the TASS Mark III database, are listed below in order of sequential use. The authors and rewriters of these programs are named in parenthesis immediately after the program name. Details on these programs can be found via the software Web references in this TN. Except for the tm3get and Linux driver descriptions, some of the following software descriptions are from the JHU/APL TASS Web page by Nick Besler at:
http://www.jhuapl.edu/APLastronomy/softwareover.html
(My comments within those notes are placed in square brackets.) I also thank Michael Gutzwiller for further explanations. A detailed accounting of the operations of the programs below will not be part of this Tech Note at this time. As of August 1999 other documents are in progress to describe those details, or you should look at notes provided with the software on the reference sites where you obtained the software. As of Sept. 1999 most of these programs (except the data acquisition programs) should be available for Linux and Win 95/98. The TASS Web page will probably have links to the latest versions and to the sites who support these programs.
tm3get.exe - (Droege/Molhant/Gutzwiller) data acquisition program, collects image data.
Produces FITS image file with approximate RA, dec of center of image, date and time of observation, other data in FITS header. Can be compiled and run in MS-DOS or Linux, but used only in MS-DOS. Chris comments (private correspondence, Aug 1999): "I have not heard of anyone getting this to work under Linux. Norman started to port it then gave up and worked with me on a fresh start under Linux. The present Linux based driver shares no code with the DOS based driver. "
(Linux driver) - (Molhant/Albertson) data acquisition program. collects image data in Linux client/server context. Tech Note 36, Installing Linux, RT Linux, and the TASS Driver, describes some of the features of this driver, and the installation of Real-Time Linux and the driver. Real-Time Linux uses a Linux kernel with modifications to support real time. The driver produces a FITS image file with approximate RA, dec of center of image, date of observation, other data in FITS header. Time of observation added to the FITS header since version 0.4, Nov 10 1998.
Dark - takes a dark image, creates a corresponding dark vector by finding the median value in each column, and allows you to save the dark vector as a FITS file which will be used by other analysis programs.
Flat - takes a raw image file, subtracts a dark vector, and creates a flat vector by finding the median ratios of each column, and allows you to save the flatvector [scaled to 32K] as a FITS file which will be used by other analysis programs. [The image file here represents a uniformly-illuminated field through the Mark III optics. Refer to the sites for their local techniques in creating these files.]
Star - takes a raw image, subtracts a dark vector, divides by a flat vector, and creates the corrected image. The corrected image may then be saved in FITS format. Star will also produce star lists from the corrected image. The new version of star also requires either the SAO catalog subset (147 Kb, Mar 16, 1997), the Guide Star Catalog subset (840 Kb, Mar 17, 1997) or the recommended Tycho catalog subset (1139 Kb, Sep 7, 1997) to solve for exact RA and Declination.
[Output from star is suitable for entry into the TASS database, but
often processing by the following programs is performed prior to database
entry. These constitute the "post-processing" of Mark III data as of 1999.]
In August 1999, Gutzwiller added this information about "flat":
The current version of the Flat program is not affected by changing sky
brightness because it uses a slightly different algorithm. This algorithm
avoids the problem with changes in the sky brightness and
also reduces its sensitivity to bright stars. This is how it works:
1. An ordinary image, preferably one with a high background level, is
dark subtracted.
2. The median value of each row is calculated.
3. Each pixel in the row is replaced by the ratio of its value to the
row's median to create a scaled image.
4. The median value of each column in the scaled image is calculated to
create a flat vector.
5. The scaled flat vector is written to a FITS image file where the
values are scaled to fit the 2 byte integer range.
StarPost - which takes a set of star lists generated by the star program and sorts them into two separate directories based on [user-selected] pass/fail criteria.
FlatComp - which works in two passes. In the first pass it calculates correction vectors for magnitudes based on a set of star lists and the Good Tycho Catalog subset (311 Kb, September 15, 1998). The Good Tycho catalog stars are those which have good photometry, are not too close to other Tycho stars, and fit the color range for Arne Hendon's color transformation. The second pass uses a set of [declination dependent?] correction vectors previously created and modifies the magnitudes of all the stars in the lists. [Output from flatcomp is suitable for entry into the TASS database. The use of this program on 1997-99 Mark III data reduced the residual magnitude errors from 55 mmag to 35 mmag.]
[end Beser quotes]
Operating parameters for star are critical: see the site-by-site notes in this Tech Note for information on those settings by site as used for Mark III observations. A point-spread function or PSF of Gaussian form is established, peaks in the image are found, an aperture set for them, the image in the aperture matched to the PSF, and a position and magnitude determined. The RA and dec of these candidates is found by matching the 30 brightest stars as corners of sets of triangles, compared to triangles formed from sets of bright stars in the Tycho catalog. The following Tech Notes describe the dark, flat, and star programs:
See Tech Note 11, Object Detection in TASS, and Tech Note 12, Using a Shifted PSF for Star Detection both by Gutzwiller, for an early (1996) analytical discussion of these programs. See Tech Note 33, Creating Optimized Photometric Apertures, for a discussion on that subject as implemented in the star program.
Some examples of image files and the processing of those files can be found on the TASS Web site at http://www.tass-survey.org/tass/early_data.html .
Another set of dark, flat, and star programs was created in Aug 1999 by Glenn Gombert to run as a batch under Linux. They are available via the TASS Web site at tass-survey.org. As he describes them:
"... a port of the "Star" processing program for Mark III images which will process (in batch mode) all of the Mark III images in a particular directory on a Linux system. When the program starts up it asks for: the Raw Input File Directory; Dark Vector Path; Flat Vector Path; Sigma Limit; Sum Ratio; and Site Code."
"It uses the Sortsao.txt catalog which it expects to be in the same directory as the raw Mark III images to process. This should be considered as an "Alpha" port [to Linux as of Aug 1999]..."
Additional programs are as follows:
(VCO tracker) - (Albertson) Jan 99, client program to Linux driver, adjusts VCO of Mark III camera on-the-fly vs. PC based 8kHz clock. VCO changes scan rate of camera to match sidereal rate. The VCO drifts with PC temperature, some VCO settings track better than others for given cameras. Apparently not in use on Mark III/Linux sites as of July 1999, see site info for details.
Chris says (private email, Aug 1999): "I would describe this as a classic servo. You set the period and it adjusts the VCO to give to the desired line period. The reference clock [is] the Linux system clock. This is not the PC hardware clock. The Linux clock may run faster or slower after some calibration and [could] even be sync'd to some world wide standard."
collate - (Arne/Gombert) Processes up to 3 star lists from "star"
or "sextractor".
Produces list of stars found in same location in more than one list.
trnsform - (Arne/Gombert) Processes star list from "collate" Also uses a list of standard stars. Computes the V and V-I transformations using all matched standards. Produces an output file of transformed magnitudes.
master - (Arne/Gombert) Processes star lists from "transform"
and "collate".
Finds all objects with 3 or more observations in each filter. Produces
a list of all stars that still have 3 or more observations Also produces
a file with two tables: (1) a histogrammed list of magnitude bin, mean
error in that bin, and number of stars in that bin; (2) those objects whose
magnitude variation is greater than 'average'.
diffcal - (Arne/Gombert) Processes lists from "master".
Also processes star lists from "transform" and "collate" Compares star
list items to Master items. Produces differential photometry record for
each item to output file.
The Mark III data acquisition programs tm3get and the Linux driver produce image files in the FITS format, including a FITS header. Tech Note #4, FITS file format for TASS images, describes an early version of that header and the nature of the FITS format. The Mark III FITS header as of 1999 has additional features. The following is a typical Mark III header, from an Nov 1998 image file using the Linux driver of that time at JUA/APL. [The tm3get program may produce a different header. I will include that header later or show the differences from the Linux header.] In the FITS standard, comments are preceded by the backslash "/" character; the comments below are automatically added to the FITS header.
SIMPLE = T / file does conform to FITS standard
BITPIX = 16 / number of bits per data pixel
NAXIS = 2 / number of data axes
NAXIS1 = 800 / length of data axis 1
NAXIS2 = 896 / length of data axis 2
EXTEND = T / FITS dataset may contain extensions
COMMENT FITS (Flexible Image Transport System) format defined in Astronomy and
COMMENT Astrophysics Supplement Series v44/p363, v44/p371, v73/p359, v73/p365.
COMMENT Contact the NASA Science Office of Standards and Technology for the
COMMENT FITS Definition document #100 and other FITS information.
DATE = '11/11/98' / FITS file creation date (dd/mm/yy)
DATE-OBS= '11/11/98' /
TIME = '13:49:12' /
BZERO = 3.27680000000000E+04 /
BSCALE = 1.00000000000000E+00 /
CTYPE1 = 'DEC ' /
CRPIX1 = 1.000000E+00 /
CRVAL1 = -1.368180E+00 /
CDELT1 = 3.819700E-03 /
CROTA1 = 0.000000E+00 /
CTYPE2 = 'RA ' /
CRPIX2 = 1.000000E+00 /
CRVAL2 = 1.657101E+02 /
CDELT2 = 3.819700E-03 /
CROTA2 = 0.000000E+00 /
ORIGIN = 'test ' /
TELESCOP= '"TASS Mark III ' /
INSTRUME= 'CCD0 ' /
OBSERVER= '"Nick Beser"' /
LATITUDE= 3.916440E+01 /
LONGITUD= -7.689750E+01 /
ALTITUDE= 4.380000E+02 /
RA = '11: 2:50.43' /
DEC = '-1:22:5.45' /
EQUINOX = 2.000000E+03 /
FILTER = 'V ' /
SCALE = 1.370000E+01 /
GAIN = 2.500000E+00 /
RDNOISE = 2.000000E+01 /
IMAGETYP= 'OBJECT ' /
BIASSEC = '[1:14,12:885]' / Non image pixels
DATASEC = '[15:782,12:885]' / Image pixels
DARKSEC = '[783:794,12:885]' / Non-image pixels
COMMENT Driver version: 0.4
COMMENT Driver compiled on: Nov 10 1998 11:40:59
COMMENT Driver run on host: tass.jhuapl.edu
COMMENT Driver run by user: beser
COMMENT Driver CFITSIO version: 1.230000
END
Site information and processing notes:
Programs used by Glenn to collect and process Mark III data for the TASS Mark III database:
tm3get: (MS-DOS)
dark: (NT 4.0)
flat: (NT 4.0)
star: (NT 4.0)
starpost:
flatcomp: (NT 4.0)
collate.exe
trnsform.exe
master.exe
diffcal.exe
In private correspondence of Aug 1999, Glenn says:
"I have been running tm3get11 under Dos to collect all of the Mark III data collected here in Dayton. I have been using Mike G's Dark, Flat, Star and FlatComp programs to process the collected data from the above into calibrated star-lists... under NT4.0. I have been using a modified version of Arne's programs [collate, transform, master, diffcal]."
"[I'm] using [these under] NT4.0 to produce a Master Photometry File and a master set of "diffcal" corrected measurements (from each individual night) to look for variable stars. I have also used a modified version of several of Doug Mink's WCSTools library routines to cross-reference the Master Photometry File with HGSC data."
"To try and identify new variables I have used the Welsh-Stetson technique (as outlined in the Winter 95 issue of CCD Astronomy) to run on processed data to identify new variables candidates....."
Programs Glenn Gombert has or is working with:
WCS.exe - process FITS image with RA, dec of center in FITS header.
Also uses Hubble GSC for reference stars.
Produces WCS parameters for addition to FITS image.
xy2sky.exe - (WCSTools) Processes star list of X, Y, instrumental
magnitude.
Produces RA, dec positions for each star.
Sextractor - (??) Processes FITS image w/ WCS parameters and
"zero point magnitude" (?)
Also uses in some fashion list of reference stars in image, and produces
into star list of X, Y, instrumental magnitude.
See Tech Note 27, Automated TASS
Photometric Reduction/Calibration, for Glenn's operation of the Sextractor program.
batch.exe - (Gombert) automates the calculation of the World
Coordinate System (WCS) Parameters which are
inserted in the FITS image headers prior to the Star Catalog being
calculated by the Sextractor Program. This allows
Sextractor to calculate (directly) RA and DEC for each object that is
detected in a particular image. It should provide a
complete Astrometric and Photometric solution for generating Star-Lists
from TASS Images.
Check Gombert's Web site (see references) for the status of these programs.
processing notes:
Date: Tue, 01 Sep 1998 06:16:17 -0400
To: tass@wwa.com
From: Glenn Gombert <gleng@infinet.com>
Subject: Re: ReprocessingI had a couple of minor differences between Mike's [Gutzwiller]
settings and the ones that I have been using:Sigma Limit: 2.0
Sum Ratio Limit: 2.25
RA Search Limit: 4.25
Num Stars TO Match: 40
Max Noise Level: 85From tass-request@wwa.com Mon Sep 8 14:43:25 1997
Date: Mon, 08 Sep 1997 14:45:37 -0400
To: tass@wwa.com
From: Glenn Gombert <gleng@infinet.com>
Subject: Arne's Programs Converted To Windows95/NTHello,
I have finished converting Arne's four programs to run under
Windows95/NT. They are
. collate.exe
. trnsform.exe
. master.exe
. diffcal.exeI have made a zip file and uploaded it to Storm under /incoming. It
is called "aah.zip". With collate/transform I have been able usually generate
> 35,000 stars ("V" & "I") magnitudes and positions from data taken since
the first week of August.The version of collate supports the output format from Mike G's new
version of "STAR". I have included the documentation for the programs called
"source.doc" as part of the zip-file.The main enhancement to the suite of programs is being able to
process lists of stars up to 100,0000 total. It seems sufficient for most
Mark III Processing needs. It can take 2-3 hours to collate a list of stars
from an "all-night" run however on my 133 Mhz - 586.As Arne modifies his programs over the next several months I will
try and keep up with the changes.Glenn G.
Glenn Gombert <gleng@infinet.com>
Date: Thu, 15 Oct 1998 17:51:02 -0400
To: tass@wwa.com
From: Glenn Gombert <gleng@infinet.com>
Subject: Reprocessed Dayton TASS Data On The WayHello,
By now Michael should have a CDROM with 65 nights of re-processed
Dayton TASS data to load his new database with. I used Mike G's FlatComp
Program to process the data and he should get similar results to what
Michael outlined in Tech. Note #45.I have another 7-8 nights of data yet to process, I will probably
collect data until the end of November and then stop collecting data until
Spring (and start creating a new mater-photometry file and diffcal data
using Arne's programs) This will take a while because of the size of the
data sets to be Collated, Master'ed, and Diffcall'ed. This should give
enought data to classify many new variables and their periods. (Lots of good
material for IBVS Tech. Papers, etc).More To Follow,
Glenn G.
Glenn Gombert <gleng@infinet.com>
Mike
Gutzwiller, Cincinnati OH site
Collecting and processing software:
(acquisition program not clear: tm3get or RT Linux driver?)
flat: (win 95)
star: (win 95)
starpost: (win 95)
flatcomp: (win 95)
Mike has built a Oracle database of his own observations, and has used
a number of programs to produce variable candidates since mid-1999. In
Aug 1999 he announced that he would migrate some of his software tools
to support the TenXcat and TASS database; check his Web site for details.
processing notes:
From: "Mike Gutzwiller" <deepsky@fuse.net>
To: <tass@wwa.com>
Subject: Reprocessing
Date: Mon, 31 Aug 1998 22:25:19 -0400I'm about to start reprocessing all my images through the latest
versions of Dark, Flat, Star, StarPost, and FlatComp. Before I do its
probably a good idea to agree between ourselves on what the parameter
settings should be in each program so each camera's output is comparable
to each others. These are the settings I propose we use:Dark - no options to set
Flat - no options to set
Star - Catalog tycho.cat
. Sigma Limit 2.5 (a compromise between Glenn's setting and mine)
. Sum Ratio Limit 2.0 (seems to give best results on my images)
. RA Search Limit 3.75 (some allowance if necessary for daylight savings time)
. Num stars to match 30 (may vary manually with tough cases)
. List output all except sequence # and ADUStarPost - Max noise level 100
. Max background var 0.05
. Max mag adjust diff 0.50
. Discard if match fail Yes
. Max PSF width 9
. Max PSF height 11FlatComp - Catalog tyc_good.cat
. Regions 8
. Var limit 0.01Does anyone else have suggestions, differences of opinions? I'm eager to get started.
Thanks, Mike G.
Collecting and processing software:
Tom of course designed and constructed the Mark III cameras. Numerous Tech Notes by Tom describe the hardware and operation of the Mark III. For processing his own Mark III data, Tom is apparently using the programs from Gutzwiller as follows:
tm3get: (MS-DOS)
dark:
flat:
star:
starpost:
flatcomp:
Settings for these programs are presumably the "defaults" as proposed by Gutzwiller (see Gutzwiller section). Some files may have been processed by Gutzwiller.
processing notes:
Date: Mon, 21 Sep 1998 21:13:11 -0500Nick Beser et al, JHU/APL (Maryland) site
To: tass@wwa.com
From: Tom Droege <droege@wwa.com>
Subject: October 1997I have finished processing data taken during October 1997. The result is
about 1.55 million measurements. I will try to send them to Michael
tomorrow from Fermilab. This was a relatively good month, with about twice
the average amount of data.Most files were processed with the agreed limits. A few needed Var limit
set to 0.02 to get enough stars, and most were processed with PSF width
limit at 9 and height at 13 because the V camera should have a different
clock speed, so the stars are stretched out.There were a few problems:
1) Analyze sequence does not fold over at 0.000 But it does start up
again when the run is started with the next number past 0.000 and does not
run around again.2) For some reason not explored, null files were created without catalog
match either successful or fail. So the following steps just passed these
files through.3) Several otherwise good looking frames failed to find a match through
the Milky Way. These were files created around daylight savings time, and
so I had a large RA Search Limit set [related to the one-hour error in timestamp].Tom Droege
Collecting and processing software:
Prior to Nov 11 1998: tm3get (DOS)
Subsequent to that date: Mark III Linux driver (Linux), the "first
time data was archived for analysis" (Beser, private email, July 1999)
dark: (win 95)
flat: (win 95)
star: (win 95)
starpost: (win 95)
flatcomp: (win 95)
Note: Previous data (date unknown) was processed with older versions of dark, flat, and star, and without starpost and flatcomp; see the discussion below for details. The results from this earlier process are not in the database as of July 1999. The observations from which that data was obtained were offered to Michael Richmond in Aug 1999 for processing into his Mark III database.
processing notes:
Date: Tue, 15 Sep 1998 08:57:14 -0400On Tue, 13 Jul 1999 07:52:27 -0400, Nicholas Beser <beser@aplcomm.jhuapl.edu>
From: Nicholas Beser <beser@aplcomm.jhuapl.edu>
Subject: Reprocessing Star Lists
To: Tom Droege <droege@wwa.com>
Cc: tass <tass@wwa.com>I have been reprocessing the October 97 data (slow process!), and I
noticed something rather odd. I thought that the settings that we had
agreed to use would have produced less star hits than my old settings.
That does not seem to be the case. I am seeing about twice the number of
stars identified by the star program.[snip]
Nick Beser
JHU/APLFrom: "Mike Gutzwiller" <deepsky@fuse.net>
To: <tass@wwa.com>
Subject: Re: Reprocessing Star Lists
Date: Tue, 15 Sep 1998 22:46:39 -0400
charset="iso-8859-1"Actually the new settings _should_ produce more star candidates since the
Sigma Limit is lower. In addition "Star" has been made slightly more
efficient compared to earlier versions.[snip]
Mike G.
From: mgutzwiller@lanvision.com
To: tass@wwa.com
Subject: RE: Reprocessing Star Lists
Date: Tue, 15 Sep 1998 10:09:06 -0400There are several reasons why you may be seeing more stars. The most
important is the setting for the "Sigma Limit". If you are using a setting
of 2.5 (like I am) or 2.0 (like Glenn is) you will see a lot more faint
stars (and more spurious signals) than you would at 3.0. Another setting
that will affect the number of stars seen is the "Sum Ratio Limit". A high
setting (> 2.0) will bring in more stars though ratios higher than 2.5 will
bring in more spurious signals. The new version of star is also a little
more efficient than previous versions.The catalog number problem is really a problem of the tyc_good.cat catalog.
While generating the catalog the numbers were truncated. I have created a
new version of the tyc_good.cat catalog with the correct id's and put it on
the web page.I've also been reprocessing my October 1996 and 1997 data and understand how
tedious it can be, especially when it takes 30 minutes to unload each night
from tape. I've been thinking of how a Linux version of star could be made
but haven't gotten very far.Mike G.
Date: Wed, 16 Sep 1998 13:41:26 -0400
From: Nicholas Beser <beser@aplcomm.jhuapl.edu>
Subject: Re: Reprocessing Star Lists
To: Chris Albertson <chris@topdog.pas1.logicon.com>
Cc: tass@wwa.com[snip]
I have data runs that cover two Julian dates (ex G0640 and G0641). I also have dark
current that is collected the next day and is therefore related to an earlier Julian
date.I am more partial to G100197 as a directory and G100197.DRK for the dark current
directoryI currently build several subdirectories with each data set. They include:
DARK - dark current
FLAT - flat file
PASS - Files that make it through starpost
FAIL - Files that do not make it through starpost
FPASS - Files that are adjusted by flatcomp.I should point out that this is the directory structure I use with the JAZ cartridge.
The CD-ROM's don't need PASS, FAIL or FPASS, and might need a directory for the
archived (ZIP) star lists.[snip]
Nick
On Fri, 9 Jul 1999 07:48:59 -0400, Nicholas Beser <beser@APLCOMM.JHUAPL.EDU> wrote:
*>Stupendous Man wrote:
*>
*>> Nick,
*>>
*>> I just returned from my trip to Toronto, and found your CD-Rom
*>> waiting for me.
*>>
*>> Michael
*>
*>Great,
*>
*>Let me know if you have any problems with the data (quality, etc).I
*>used the settings that we agreed on for starpost and flatcomp. I did
*>notice that some star images that looked very good were rejected.These
*>images happened to be of a very crowded part of the sky, so that the
*>large star field might have been rejected as noise.
*>
*>I also found it interesting that CCD1 which has a cooling problem was
*>accepted more than CCD2. CCD0 is slightly out of alignment with the
*>others due to the way the CCD is mounted in its frame (We tried to
*>adjust for it, but it came down to either CCD0 working or CCD1 and CCD2
*>working.). It would be interesting to see the years worth of data
*>presented has a map like you have on the TASS web page.
*>
*>The older data (before July 98) has not be reprocessed yet. Do you want
*>that as well?
*>
*>Nick
*>We have sent Mike Richmond a CD-ROM with 73 nights of observations on it.Michael Richmond, Rochester NY site
*>(1 years worth of data). Since he just got it, he has not
*>added it to the data base. Prior data was sent before the starpost and flatcomp
*>program was available.
*>
*>Our processing chain is based on Michael Gutzwiller's Windows 95 software:
*>
*>dark -> Flat -> Star -> Starpost -> Flatcomp
*>
*>I also have generated several routines for data monitoring and health checks.
*>The fits2raw code is based partially on Chris
*>Albertson's code, and generates a viewable FITS file using a web browser.
*>I also generated a java applet that I use to monitor the
*>entire evenings worth of observations [via] the internet.
*>I also have a fits header extractor program that creates html versions of
*>the header.
*>
*>What I want (which I don't have) is a unix version of the processing chain.
*>I can use that to create on line star lists, and
*>potentially merge with existing star maps in real time.*>I [previously] sent several months worth of data via FTP when we were
*>first setting up the database. This was before starpost and flatcomp were
*>written. In addition, there was no agreement as to what parameter to set
*>the STAR program to. The group discussed the parameters, and
*>agreed to a different set than had been processed. In addition, Baltimore
*>has about 4 months with really bad weather, coupled with some
*>major repairs to our CCD (The grease used to seal the CCD cell had
*>sublimated all over the optics. (We have a sealed housing to protect
*>the telescope, and during the summer it gets very hot). Bernie re-did
*>the design (no grease, instead we have dry air pumped into the cell).*>..I asked Mike if he wanted me to rerun the old data, and he told me to hold
*>off on it (I think he has his hands full)....I expect that the 73 nights
*>of [previous] observations should be added [eventually] to the database...
*>The data was processed with dark -> flat -> Star. I will have to consult
*>my records to get the actual dates.
His Mark III camera was formerly in VT, now it is in Rochester NY.
No observations from this triplet have been entered into the TASS database,
as of Aug 1999.
MarkIII Linux driver: (Molhant/Albertson) acquisition
XVista: (Richmond/Treffers) Unix package for reducing images to star
lists. See Tech Note 32, Comparison of
several methods of extracting magnitudes, for Michael's discussion
of the use of XVista on TASS images for photometric analysis.
(His note requested others to provide their methods of analysis for
comparison: as of Aug 1999 the TN did not provide them.)
other software of interest to Michael:
Michael has written a number of programs to create and/or support the TASS database. This Tech Note only addresses programs that process raw Mark III data or files created by those Mark III programs.
Chris Albertson, Redondo Beach CA site:
Mark III Linux Driver: acquisition
As of July 1999: "What data I had before [my] current [camera] problem was reduced for me by Ted Wood[house] in Canada. He used the standard star.exe process. I mailed him some CDs and [he] FTP'd the result back....The data was reduced with star and the Tycho catalog with a 2.5 sigma detection limit." This data was offered to Michael Richmond in Aug 1999 for entry into his Mark III database.
Chris has written a number of programs to create and/or support the TASS database. This Tech Note does not address database management and access programs; only programs that process raw Mark III data into files compatible for database entry.
for software:
tm3get: http://www.tass-survey.org/tass/software/data_aq/index.html
dark, flat, star, starpost, flatcomp: http://home.fuse.net/deepsky/programs.html
batch: http://www.tass-survey.org/tass/software/batch/batch.html
sextractor: http://www.tass-survey.org/tass/software/sextractor/Extra20.zip
collate, difcal, master, transform: http://www.tass-survey.org/tass/software/collate/*
(also, look for a ZIPped version of all of these.)
XVista: http://www.tass-survey.org/tass/software/xvista/index.html
MarkIII Linux driver: http://www2.jps.net/~chrisja/TASS_Software/
TASS Mark III Web sites:
TASS home page (with current links to other pages) http://www.tass-survey.org
TASS Tech Notes are at http://tass-survey.org/tass/technotes
Nick Beser et al's JHU/APL site is http://www.jhuapl.edu/APLastronomy/tassobserve.html
Michael Gutzwiller's TASS Cincinnati site is http://home.fuse.net/deepsky/tass.html
Chris Albertson's site is
http://table.jps.net/~chrisja/TASS_Software/
Glen Gombert's TASS Site is http://www.infinet.com/~ggombert/tass.html