Michael Richmond
richmond@astro.princeton.edu
Dec 1, 1996
I went up to Vermont for Thanksgiving, and discovered that the liquid which cools the TE units had frozen in its plastic tubing! I guess this shouldn't have been a surprise, since it gets pretty cold up there, and the tubing runs for 40 feet from a basement, under the lawn, to the cameras (and back).
Fortunately, there was a solution. I found two hair dryers, placed one at each end of the PVC pipe running from camera to basement, and let them run for about an hour. It was quick work to thaw out the exposed tubing near the camera itself, but it took quite a while to get rid of the ice in the middle of the tubing. At last, however, after turning on the pump, I saw the bubbles move. Phew.
So, I picked up 7 gallons of antifreeze (the non-toxic kind) and added them to the 25 gallons of water (which already had a little antifreeze in it). In addition, I've left the pump running, hoping that if the water flows 24 hours a day, all winter long, it's less likely to freeze.
While I was there, my brother and I drilled a hole in the floor so that we could run a telephone wire down from the first floor to the basement. I'll pick up a modem sometime this week, and install it on my next trip up to Vermont. My hope is that I can keep the PC running Linux continuously, programming it to fire up the (soon to appear) Linux data-aq during the night, reduce the data from images to star lists in the morning, and finally transfer the star lists over the phone to a machine down here in New Jersey.
Jure Skvarc
SKVARC@eros.ijs.si
Dec 1, 1996
This week I downloaded images Chris Albertson put on his FTP server. I have remark or two considering the format of images.
I suspect I am a bit responsible for the first thing. Namely, the BZERO value in the FITS header is set to 32768. This is nice if we want to have pixel values from 0 to 65535, what is the only natural and I also wrote this value in the Technical note 4. However, when I wanted to see these images using Fitsblink program, the CFITSIO library complained because of the overflow. The reason is that it supports only signed 16-bit integers which have range from -32768 to 32767. I made a workaround in the program, but I wonder shouldn't image files be more compatible with CFITSIO library. This would mean that we put BZERO back to 0. Of course, it is always possible to handle this problem in a program which reads images, but it introduces some complications. What do others think about this?
The other question I have is what is on the left and right side of the image. On both sides there is a strip of pixels with values which are very different from values of the sky and stars. If these are values of the covered pixels or anything like that I suggest to remove them or to put them in an extension of the FITS file. I think that now they only confuse programs which read such images.
Also I would like to announce Fitsblink 1.2. There are no functional changes from the previous version but I removed some bugs and made some code-cleaning (and new bugs, I suppose). Now you can read TASS images as they are (with BZERO=32768). It is also easier to download sources because they are separated from test images. Along with binary versions for HPUX and linux they can be found on the locations: http://kronos.ijs.si/~jure/fitsblink/fitsblink.html and http://www2.ijs.si/~skvarc/fitsblink/fitsblink.html
I will appreciate bug reports, suggestions and other remarks to my email. Thank you.
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 1, 1996
> This week I downloaded images Chris Albertson put on his FTP server. I have > remark or two considering the format of images. > > I suspect I am a bit responsible for the first thing. Namely, the BZERO > value in the FITS header is set to 32768. This is nice if we want to have > pixel values from 0 to 65535, what is the only natural and I also wrote > this value in the Technical note 4. However, when I wanted to see these > images using Fitsblink program, the CFITSIO library complained because of the > overflow. The reason is that it supports only signed 16-bit integers which > have range from -32768 to 32767. I made a workaround in the program, but I > wonder shouldn't image files be more compatible with CFITSIO library. This > would mean that we put BZERO back to 0. Of course, it is always possible to > handle this problem in a program which reads images, but it introduces some > complications. What do others think about this?Well, the problem here is that the CFITSIO library is bugged: it is not 100% compatible with the FITS standard in that the standard does not disallow the use of data with 16 significand bits (instead of 15): the conversion we do (using a BZERO equal to 32768) is perfectly all right for the FITS standard. So, rather than throw away one bit in every pixel we have, let's correct the CFITSIO library, I say.
> The other question I have is what is on the left and right side of the > image. On both sides there is a strip of pixels with values which are very > different from values of the sky and stars. If these are values of the > covered pixels or anything like that I suggest to remove them or to put them > in an extension of the FITS file. I think that now they only confuse > programs which read such images.Sorry, but I disagree: the files generated by the data acquisition program (tm3get10) are *raw* data files, and as such contain *all* the acquired data. Sure enough, those files were not designed to be seen. These strips of pixels (some of which are not pixels at all, but temperature, voltage or current measurements) are to be used during the image reduction steps, i.e.: estimation of an appropriate dark line (this requires the temperature data and may make use of the blind pixels strips left and right of the image), substraction of said dark line, division by a standard gain line (i.e.: "flat fielding"), removal of damaged lines (this is where voltage measurements are useful: data taken with the nominal +5 volts down to +3 volts are pretty much useless), etc.
A fairly complete description of these strips of pixels is to be found in the data acquisition program's documentation, i.e.: tm3get10.htm, available at Michael's TASS home page on the WEB. If you can't get it from there, please tell me: I'll email a copy of that doc to you.
Naturally enough, these strips of non-imaging pixels should be removed after that stage, and an appropriate temperature entry made in the FITS header instead, giving the average temperature and its standard deviation (both in celsius). That work, however, should not be done by the data acquisition program. It is the job of the image reduction program to transform these raw data files into clean image files. Then the data reduction program should extract light source coordinate files from these clean image files, etc.
> Also I would like to announce Fitsblink 1.2. There are no functional > changes from the previous version but I removed some bugs and made some > code-cleaning (and new bugs, I suppose). Now you can read TASS images > as they are (with BZERO=32768). It is also easier to download sources > because they are separated from test images. Along with binary versions for > HPUX and linux they can be found on the locations: > http://kronos.ijs.si/~jure/fitsblink/fitsblink.html and > http://www2.ijs.si/~skvarc/fitsblink/fitsblink.htmlThanks for a nice program!
Chris Albertson
chrisa@wavenet.com
Dec 1, 1996
> This week I downloaded images Chris Albertson put on his FTP server. I have > remark or two considering the format of images. > > I suspect I am a bit responsible for the first thing. Namely, the BZERO > value in the FITS header is set to 32768. This is nice if we want to have > pixel values from 0 to 65535, what is the only natural and I also wrote > this value in the Technical note 4. However, when I wanted to see these > images using Fitsblink program, the CFITSIO library complained because of the > overflow. The reason is that it supports only signed 16-bit integers which > have range from -32768 to 32767.Read on, I think I agree with you. BUT...
I just pulled out my copy of the FITS standard. I think the FITS files written by Norman's program are technically correct by the standard. Also by the standard I think the CFITSIO library must be in error. Here is my reasoning:
The software (fits reader) I use does the correct thing. If BZERO and BSCALE are present and BSCALE is non-zero it returns type floats. Is there a way to get CFITSIO to give you floats? Using floats is the best approach because (as you know) it is possible for a valid FITS file to overflow a 16-bit signed int.
All that said, I see no reason not to set BZERO = 0.0 or to simply remove the B* header lines completely. From my point of view it make no difference what the B* lines are set to.
I do think the array should continue to contain ADU values exactly as they came off the ADC chip.
> The other question I have is what is on the left and right side of the > image. On both sides there is a strip of pixels with values which are very > different from values of the sky and stars. If these are values of the > covered pixels or anything like that I suggest to remove them or to put them > in an extension of the FITS file. I think that now they only confuse > programs which read such images.
Joe Ronchetti
jronchetti@earthlink.net
Dec 1, 1996
When I convert fits values to true pixel values I get a range from 0 to 65000 or so. The majority of pixels seem to be between 9000 and 14000. What is the lowest value to consider above dark current and what is the highest to consider below hot pixels and such?
Chris Albertson
chrisa@wavenet.com
Dec 1, 1996
Joe Ronchetti brings up an interesting question.
At first I thought that the range 0..65000 came from the fact that I was including the "edges", the peudo-pixels along the edges of the raw data files that contain things like masked cells and instrumentation values.
It does seem very odd that the (12 or 14 bit I forget) wide ADC can generate ADU values with such a wide range (0..65K should require 16bits)
But no, I can find very small subsections of the image that contain the full 0..65K range. For example, in 30t0337_269 there are several bright stars that are saterated. These stars come right up from the background and then contain a "hole" in the middle. The hole is a bunch of zero ADU values in the center of the star. Typicaly the 65K ADU values are directly adjacent to the 0 ADU values. I think it may be caused by the ADC chip wrapping when it is fed a voltage it can not handle. I am not an expert on how this process works.
Here is an example: The index numbers assume the first cell = 1 (not 0) My first step on reading the file is to convert to floating point. It avoids numeric problems. (taken from 30t0337_269)
231 232 233 234 235 236 237 238 239 240 241
584 1.4E4 1.4E4 1.6E4 1.8E4 2.2E4 0. 2.5E4 2.1E4 1.7E4 1.5E4 1.4E4
583 1.4E4 1.5E4 1.7E4 2.2E4 3.1E4 0. 3.7E4 2.7E4 2.0E4 1.6E4 1.4E4
582 1.4E4 1.6E4 1.9E4 2.9E4 4.8E4 0. 5.3E4 3.3E4 2.2E4 1.7E4 1.5E4
581 1.4E4 1.6E4 2.1E4 3.4E4 6.3E4 0. 6.6E4 3.9E4 2.4E4 1.7E4 1.5E4
580 1.4E4 1.6E4 2.2E4 3.7E4 0. 0. 6.6E4 4.0E4 2.4E4 1.7E4 1.5E4
579 1.4E4 1.6E4 2.2E4 3.8E4 0. 0. 6.6E4 4.0E4 2.4E4 1.7E4 1.5E4
578 1.4E4 1.6E4 2.2E4 3.7E4 0. 0. 6.2E4 3.6E4 2.2E4 1.7E4 1.5E4
577 1.4E4 1.6E4 2.0E4 3.3E4 6.0E4 0. 5.1E4 3.1E4 2.1E4 1.6E4 1.5E4
576 1.4E4 1.5E4 1.8E4 2.6E4 4.1E4 5.2E4 3.7E4 2.5E4 1.9E4 1.6E4 1.4E4
575 1.3E4 1.4E4 1.6E4 1.9E4 2.5E4 2.9E4 2.6E4 2.1E4 1.7E4 1.5E4 1.4E4
574 1.3E4 1.4E4 1.4E4 1.6E4 1.8E4 1.9E4 1.9E4 1.7E4 1.5E4 1.4E4 1.4E4
Anyone know how these high valued pixels get into the file?
These are not the normal case. A histogram of the image portion of the file shows what one would expect. A peak right at the sky background value (about 12k) then dropping smoothly to near zero at about 32K but extending flat from there to 65K. The few ADU=0 points are to few to show up on the graph.
This is right in line with Joe Ronchetti's results. The vast majority of the pixels are in the range between 9000..14000. These are easy to understand the "interesting" ones to me are those in the above table.
Joe Ronchetti
jronchetti@earthlink.net
Dec 1, 1996
How do I convert the crval1 and crval2 to dec and ra?
Chris Albertson
chrisa@wavenet.com
Dec 1, 1996
For reference here is part of a header:
CTYPE1 = 'DEC'
CRPIX1 = 1.0
CRVAL1 = 0.0000
CDELT1 = 0.0038197
CROTA1 = 0.0
CTYPE2 = 'RA'
CRPIX2 = 1.0
CRVAL2 = 344.166659
CDELT2 = 0.0038197
CROTA2 = 0.0
It says,
(CTYPE1) "along dimension 1 units are DEC"
(CRPIX1) Reference pixel is pixel #1 (the first)
(CRVAL1) The reference pixel is looking at DEC=0.0000
(CDELT1) Each pixel after the reference is looking
0.0038197 DEC units farther
The header does not say it, but the units in the DEC direction
are decimal degrees.
If you are looking at the files I posted on logicon.com I think the value in CRVAL1 is wrong. I think norman may have fixed this. In real life, Pixel 1 is not looking at 0.0 (the equator) as the header implies, pixel 400 is looking there. Pixel #1 is realy looking at about -1.5 DEC
The header above says the image is above the equator but in reality it is centered on the equator.
Dimension 2 is in the RA direction. Pixel #1 is looking at RA=344.166659 (in decimal degrees) each pixel looks 0.0038197 degrees farther in RA.
The CROTA1/CROTA2 lines say the image is not rotated or skewed.
If you need to convert from RA in degrees to RA in HH:MM:SS it is easy. Just remember that there are 360 degrees or 24 hours in a circle.
Tom Droege
droege@fnal.gov
Dec 1, 1996
>But no, I can find very small subsections of the image that contain the >full 0..65K range. For example, in 30t0337_269 there are several bright >stars that are saterated. These stars come right up from the background >and then contain a "hole" in the middle. The hole is a bunch of zero >ADU >values in the center of the star. Typicaly the 65K ADU values are >directly >adjacent to the 0 ADU values. I think it may be caused by the ADC chip >wrapping when it is fed a voltage it can not handle. I am not an expert >on how this process works.Don't blame the poor ADC, it works very nicely. Blame my electronics. Or better still blame the CCD chip as that is where the problem arises. I will have to think about how to cure this in the next version, but for now you all will have to do it in software. This is of course why we have software instead of doing everything in software. Software is so felxible you know. ;^)
Here is how the problem arises. We clear the charge from the output capicitor in the CCD with the reset pulse. We now sample the voltage on the output capicitor in the CCD and store it in Hold circuit #1. The charge in the next cell is now shifted into the output capicator and we measure its charge and store it in hold circuit #2. An operational amplifier now takes the difference between the voltage on hold circuit #2 and hold circuit #1. This voltage (after amplification) is sent to the ADC. A bias voltage sets the dark difference so that there is about 8000 counts (out of 65000) for dark. This value normally assures that there are no negative differences that the ADC would see as below zero and thus read as zero, losing the value.
When a very bright light shines on the CCD, it does really funny things. The point in time where the reset value is taken becomes full scale. It just does not reset properly. This independant of how hard you drive it or how long, etc.. I think the output circuit is light sensitive. It is worse than that. The reset value can be larger than the light value under these conditions. Too complicated to draw what happens, but I have looked at it many times on a scope. When this happens, the "dark" (sample #1) sample becomes larger than the light (sample #2) and the difference is negative. The ADC properly converts this as zero.
While I can imagine "fixing" this in hardware, it is really a saturated chip problem. I suppose I could sense this and force the ADC to full scale, but probably this is better done in software, what do you all think?? There is always an "edge" problem in hardware which might be hard to cure.
So for now, a zero in the center of full scale readings should be set to full scale. Note that you have already lost calibration when the value goes above about 40,000 counts. 8000 counts is zero. 80,000 electrons divided by 2.5 electrons per ADU gives 32000 counts for the true linear range. Readings above this require bleeding between CCD cells, or charge in a well more that the stated full well capacity. Note that as I understand it, the full well capacity stated on the data sheet is really the "linear full well capacity". Above that value there still may be charge trapped but the voltage generated is no longer a linear function of the light.
I have been less active than usual here because I have been busy with other things. I am now officially "half" retired. I have hired a full time technician to help me at home in this work. The addition to my house is nearly complete, and I have actually swam in the pool. So my new life is starting! I am still trying to get this mail list switched over to Glenn who has agreed to manage it. I just haven't succeeded in attracting the attention of Dave Vrona at wwa.com. No wonder, he is the president of wwa.com and is living the exciting life of being the president of a hot start up. I lived that life once (a company called Applied Logic). We were going to manufacture "computer power" in central stagions much like electric utilitys. You should read the prospectus. By now "time sharing" was gong to be a multi-billion dollar industry. Goes to show you what stock analysts know. Think about Applied Logic when you are tempted to invest in Netscape or ... We had 350 employees at one time.
Jure Skvarc
SKVARC@eros.ijs.si
Dec 2, 1996
Thanks to Norman and Chris I see that I was in error when I assumed that those FITS files were "final version" i. e. with dark frame subtracted etc. However, I still think it may be useful to put somewhere in the FITS header information where is the "sky area" in the image.
As for CFITSIO, I don't think it does anything wrong. When it stores pixel values to memory it just checks if everything is OK and short int format I use to read values has range -32768 to 32767. But it is still possible to read data with BITPIX=16 as 32-bit integers or floating point numbers. Of course, you either spend two times more space to store such images or you have to copy data to 16-bit array later. I guess I will probably go for 32 bit integers at expense of doubling memory consumption.
Joe Ronchetti
jronchetti@earthlink.net
Dec 2, 1996
I need to know how to come up with accurate dark values to subtract from each pixel.
Glenn Gombert
ggombert@mail.erinet.com
Dec 5, 1996
I just downloaded an GREAT Windows95 software package for doing image analysis under Windows95 from the Automated Telescope Facility at the University of IOWA. It has a full featured set of routines for doing such things as SuperNova Searches, looking for new variable stars, and much, much more! Check it out at:
"http://www-astro.physics.uiowa.edu/software.html"
The package takes up 6.5 megabyte file in compressed PKZIP format and takes a LONG time to download so be prepared to wait a while...
The information was supplied courtsey of Dennis diCicco the Senior Editor of CCD Astronomy, they are going to do a review of the package in the next issue of CCDA.
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 5, 1996
I have placed a new DOS data acquisition program on the "incoming" ftp directory of "storm.fnal.gov", here are the files to be found there:
- tm3get11.exe the DOS executable as compiled by WATCOM C/C++ 10.6
(and let me tell you that this compiler is no joy to use)
- tm3get11.c the main source file
- tass_dos.1 the first subsidiary source file, #included in tm3get11.c
- tass_dos.2 the second subsidiary source file, #included in tm3get11.c
- tm3get11.htm the documentation - it has a change history near the
beginning, just after the introduction - Please, read it
if you want to use tm3get11 or the FITS files it produces.
- tass_mak.wat a makefile for WATCOM C/C++
- tass_lnk.wat a link file for WATCOM C/C++
- tass_mak.bor a makefile for Borland C/C++
- tass_lnk.bor a link file for Borland C/C++
- tass.rc an *example* configuration file for tm3get11
I'm sorry to say so, but the last version of tm3get10 was bugged - its stack was WATCOM's default size, i.e.: 2 Kb and it overflew whenever closing a set of files after opening the next set, with the usual strange behavior due to a stack overflow... I've since changed the stack allocation to 16 Kb, which is quite enough.
Tm3get11 has a pair of new features: its FITS headers are now clearly commented and there is a way to automatically add your own entries to these headers, for instance to indicate what filter you used on this particular CCD, etc.
Tm3get11 is also more clearly modular, with all its DOS dependencies localized within a pair of files, i.e.: tass_dos.1 and tass_dos.2 (what else?). I've been quite ill, hence my productivity has been low, which may or may not explain both the bug I left in the last tm3get10 and the delay in delivering a Linux version of tm3get11. Rest assured that I'm working on it with all of (what's left of) my energies. And yes, it will be the same tm3get11 that you now see running under DOS, but running (much more smoothly, I hope) under that much better of OS'es: Linux.
I'm too tired to keep working tonight, so bye for now!
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 6, 1996
Nick asked some good questions whose answers might interest some more of us. So, here are my answers: Dear Nick, you asked:
> I am setting linux up on the second drive of the tass data collection > computer. I was planning to use redhat linux. What version kernel are you > running, and are you using the new release of the XFree86 (I think it is > 3.2)?I am not too sure for the Linux kernel: I'm running DOS on the same machine right now and can't check without loosing my telecom access. I think it is kernel 1.3.0 or some such number between 1.3.0 and 1.3.21 (I'm sure it is less than 1.3.21). XFree86: I am using release 3.2, 'coz previous releases didn't work with my ATI Graphics Expression video card.
> Can you let me know when you might be ready for someone to alpha test > your code? I would like to look it over to see how it might integrate > into the remote control approach we are setting up at the lab.I'll tell everybody on the list as soon as it stops freezing & crashing my Linux box, promised! But you might prefer to tell me how you intend to go about providing access to both the FITS data files (not in real-time, but maybe nearly so, as one is produced every few minutes by each camera in a TASS triplet) and the real-time display (further info on that further down in this message).
> (I want to > collect the rough image into a periodically updated graphics file, and > send it to a heavy duty httpd server that will be accessable to the > outside world. (We are setting up the apl observatory home page to have > outside access). The observatory had two relatively low power computers > (486) one collecting data, the other managing a limited use homepage. The > home page server can run a script to convert a tass collected rough image > (4 bit pixel) into a gif or jpeg file, and then send it to the sparc 20 > that the lab uses to manage its outside web pages. I need to see how your > program sets up the image display so I can taylor the web server routines.Well, what tm3get11 does is this: as soon as it receives a completed (800 pixels) line from the TASS device driver, it copies it to the currently open FITS file (or files when in the overlap area between two successive files). Then it computes a one-dimensional histogram of the image pixels in that line, in order to find its median, which it then divides by 16, discarding 4 LS bits. Then it similarly discards the 4 least significand bits from all image pixels to get 12-bits pixel values, substracts the above calculated median (replacing negative results with 0) and replaces all results above 255 with 255, thus getting 8-bits pixel values. It then uses a look-up table (with 256 entries) to find the 4-bits approximate log of these pixel values and uses these logs to update the display. So, in order to satisfy your desire to make a fast display available for downloading, I may copy any of the following data to a Linux named pipe (or some similar device - I think streamio might also be appropriate - I'm open to suggestions here) for use by your programs:
I think the first one is redundant as new FITS files become available every few minutes. The second one is not too useful, I think. The third one is half the size of the raw data and might be nice for gif or jpeg files (I don't recommend the latter: it's a slow compression method and I have seen it generate a lot of artifacts on stellar fields - you might instead consider using the new "free" image file standard that is currently taking the WEB by storm, it compresses well and doesn't produce the obnoxious artifacts that plague jpeg). The last one is already four times smaller than the raw data, with two pixels per byte. Even for this format I would recommend avoiding jpeg, though.
So, what do you want tm3get11 to do for you?
Nick Beser
beser@aplcomm.jhuapl.edu
Dec 6, 1996
Norman,
I agree with your suggestion about JPEG. If you save 4 bit logs (what you see on the screen), that may be sufficient for determining if everything is working OK. (I can live with 8 bit or 4 bit). I like the idea of limiting the file size at the data collection machine. Since I have a second computer that is maintaining the home page, it will retrieve the latest file from the data collection machine. I would like to keep it as low overhead as possible.
One idea (just for discussion). If you are creating the 4 bit per pixel, 2 pixel per byte output from the data collection machine, you can output that to a sockets interface that will send it via the lan to the data collection computer. The matching routine will collect the data stream and output a series of gif files and home page command signals (either by semaphore files or by mail box). The home page will use the latest display file in its update to the user monitoring the system. The output pipe could have as a format:
time stamp, line number, temperature 1, temperature 2, 4-bit encoded pixels (newline)
The receiver program will take the pixel data, add it with previously collected data, and when 10 or 20 seconds goes by, convert it to GIF, output the file, and then take the time stamp, line number, temperature 1 and 2 and output that to a text file. The home page will have a task that is monitoring for the new text file, read it, output the data, and direct the GIF to the user currently logged into the home page (refreshing the screen automatically).
This way, the home page will have access to the temperature values, able to detect problems, issue shut down to 12 volts power supply if necessary, send out emergency e-mail, without interfering with the data collection. If it becomes overloaded, it just will have a data dropout.
Nick Beser
beser@aplcomm.jhuapl.edu
Dec 6, 1996
Chris Albertson wrote:
> Well, I said a conversion program would be easy to write... > > Should anyone want to periodically scan a directory for new *.fts files, > convert those files to *.gif files then move both the gif and fts files > to new directories, the following will do that for you. It should run > on any UNIX or Linux system. Most systems will already have the > required > utilities installed. If not they are easy to get off the 'net. >Most if not all of the utilities are standard with linux (redhad edition and slackware are my two data points). I planned to do something simpler. Since the raw pixels are available (downloaded from the data aquisition system as 4 bit per pixel data, I can use a library to create a GIF file. I think we are talking a few seconds (I will have to rig a set of test data to develop and check the routine. I would like the data aquisition computer to spend all of its time doing the data collection, and leave the re-format to the home page server.
Nick Beser
beser@aplcomm.jhuapl.edu
Dec 6, 1996
Chris Albertson wrote:
> I'd also extend this Idea to the UNIX real-time display. Make > it a seperate non-real-time program. It would accept pixels from > tm3get11 scale and display them on an X11 screen The display program > should be the one to scale the pixels to the properties of the display > device and the user's preference. This removes any X11 display overhead > from your real-time program. > > If you have not yet written the UNIX display code, perhaps you could > skip it and simply blast the pixels out a named pipe or socket. This > would be much easyer to do then any calls to actualy draw thoses pixels. > > Now you need another program that reads from a socket or named pipe and > does whatever it wants with those pixels, converts them to low-res GIF, > or writes them to an X-display, or does some kind of real-time comet or > supernova hunting or whatever... In any case I think the pixel scaling > should be done by the end that reads the pipe. BTW I would have made > the > FITS file writer just another one of those pipe readers too. > > .... > > Then a GIF writer would look a lot like a FITS writer: they both just > read from a pipe and write to disk. An SVGA displayer would look like > and X11 displayer, they both just read from a pipe and write to thier > display. With sockets, any of these "readers" could run on any > computer on the network that understands sockets, even Win95, Sun, > or Linux. > > Another advantage. When the Mk IV comes out you only have to redo the > part that reads from the camera and stuffs pixels into the pipe.
Chris,
This all makes sense. If we are piping the 4 bit pixels to a socket interface, then the slow data aquisition computer can just pipe the data to the faster X windows display system. If you are displaying it via a slow modem link on that fast X-11 host, then the speed differential will not effect (or should not effect) the slower data aquisition computer. It would fit in nicely with the httpd approach of providing netscape access to the latest GIF image from the telescope.
We will see if nfs access to the fits files has any effect on data aquisition.
Since we do not have a high speed computer taking the data, I was not planning on making the FITS files available until after the observing session. That also depends on the APL network management people allowing us storage for off-site ftp access. (We think that will happen, but they will not give their blessing until they see the whole system).
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 6, 1996
Said Chris, replying to Nick about the use of separate programs to transform FITS files (or 4-bits display files) into downloadable GIF files:
> I'd also extend this Idea to the UNIX real-time display. Make > it a seperate non-real-time program. It would accept pixels from > tm3get11 scale and display them on an X11 screen The display program > should be the one to scale the pixels to the properties of the display > device and the user's preference. This removes any X11 display overhead > from your real-time program.Well, yes and no... I'll have to explain the architecture of the Linux implementation of tm3get11 so we can start talking about one and the same thing, I guess. See further down in this message.
> If you have not yet written the UNIX display code, perhaps you could > skip it and simply blast the pixels out a named pipe or socket. This > would be much easyer to do then any calls to actualy draw thoses pixels.Well, possibly, but I'm not too sure. The DOS version of tm3get11 already contains a lot of display code (now neatly separated into a pair of DOS-only included files: tass_dos.1 and tass_dos.2), so writing the corresponding interface for X11 would not be that much trouble - the interface is pretty clean, anyway, and there is a subset of X11 that's fairly easy to handle too (with fixed-size windows without mousy interface and similar restrictions).
> Now you need another program that reads from a socket or named pipe and > does whatever it wants with those pixels, converts them to low-res GIF, > or writes them to an X-display, or does some kind of real-time comet or > supernova hunting or whatever... In any case I think the pixel scaling > should be done by the end that reads the pipe. BTW I would have made > the FITS file writer just another one of those pipe readers too.Well, I see we really need to share an identical view of tm3get11-under-Linux, so here goes its architecture:
> Well, here I am telling you what to do when you are doing all the work. > But if the program where not monolithic some of us could contribute a > "pipe reader" or two and all you'd have to do is read data from the > hardware and stuff the pipe.As you can see, it is not that monolithic... ;^) If there is something to be gained by separating the X11 real-time display code (which I'm not sure there is), I could do that, and if someone out there is interested in doing that part, I would readily accept such help (but I get to specify the interface data format - and I am known to be a buff of packetizing, so be warned: it would look like packetized message passing via a named pipe or whatever similar stuff is available under Linux).
Much more interesting would be to separate a possible remote access facility from the bulk of tm3get11, i.e.: some separate program that would accept remote requests, translate them into pseudo keypresses & insert them within the stream on keypresses coming from the real keyboard to tm3get11 (on one side) and send the 4-bits image lines and textual data for real-time display to both the X11 driver and the appropriate remote display facility...
Such a program would relieve tm3get11 from all its user interaction parts and thus considerably simplify its code. I have written a similar program a few years ago, when XENIX was all the Unix that PC owners could (quite expensively) enjoy... 't was fun writing, but then there was no image display facility, no X11 to interface to, etc... Much easier than this job! Although the remote control facilities were PC's running a special TSR program under DOS that disguised them as VT100 terminals for XENIX purposes while letting them run whatever program their owners wanted (for DOS purposes), and that TSR was indeed quite a challenge to write. ;^)
> Then a GIF writer would look a lot like a FITS writer: they both just > read from a pipe and write to disk. An SVGA displayer would look like > and X11 displayer, they both just read from a pipe and write to thier > display. With sockets, any of these "readers" could run on any > computer on the network that understands sockets, even Win95, Sun, > or Linux.And tm3get11 becomes a TASS server program. As I wrote above, I would prefer to cut neatly all user interactions from such a server and place them in a separate program (that would process both remote operations and X11/SVGAlib displaying and local keyboard input, exchanging only messages and data with the tm3get11 server) - any takers for such a "user interactions" program? And by the way, it would have to be able to run on a networkless Linux station, which means that (maybe regrettably) sockets would not do it (unless you know something I don't - a distinct probability, that!).
> Another advantage. When the Mk IV comes out you only have to redo the > part that reads from the camera and stuffs pixels into the pipe.No advantage on the current architecture, that: I just have to redo the part that poll the Control Card and maybe change 10 to 20 % of the device driver code to become Mark IV ready... ;^)
Chris Albertson
chrisa@wavenet.com
Dec 6, 1996
Thanks a lot for the explaination of Linux tm3get. rather then quote you I'll try to explain your design in my own words. See if I am correct. I'll try to explain it for non-UNIX types as this is going over the TASS mail list too.
Question for the reast of the group: Would the ability to easyly write "clients" as discussed below be a valueable feature?
So here goes a further refinment of my version of a Linux TASS system:
First by some magic of kernel hacking and device driver writing by Norman we will have a "special file" called "/dev/tassmk3" Now by the above magic this file is connected to the TASS camera hardware and a normal user program can open and read from it as if it were a normal file on a disk. (Close enough) Now when you read from the "special file" you get 3x800=2400 raw pixels of data at a time. This special file is esentualy an extension of the Linux OS.
Now here is where my design would differ sightly from Norman's plan. (It may seem like a large departure at first but at the end of this message I argue that it is not.) Instead of having tm3get11 directly read from the special file /dev/tassmk3, I'd write a "pixel server". A pixel server would read from /dev/tassmk3 and then supply those pixels to any number of clients via a socket interface. The pixel server would act as a "Y" or even an octopus supplying pixels to any number of "users" These users or "pixel clients" would read raw pixel data from thier socket and do whatever they whished with the data. I can see that there could be several usfull clients:
From a C programmer's point of view sockets are easy to use. Basicaly, when you create a socket you are given a "file descriptor". You do reads and writes on the file descriptor as if it where a file. So the "pixel clients" look a lot like programs that could otherwise directly read the special file /dev/tassmk3.
The pixel server simply acts as a multiplexor. It does no other processing. (Well maybe buffering a few lines would be a good idea.)
Now one more importent point about sockets. When you (a client) connect to a socket you say something very close to
connect (FD, SOCKNAME)
This is like a file open. FD is a file descriptor and SOCKETNAME is like a file name. But here is the good part: the syntax of SOCKETNAME is like this: "HOSTNAME:SOCKET". HOSTNAME can be any host on the the local Ethernet LAN or any host on the Internet. Of cource HOSTNAME could also be the name of the local machine. In that case the server and client would be co-hosted.
Sockets are not just a UNIX thing. MS Windows supports them and so does VAX/VMS and other OSes too. So those clients (#s 1 ~ 6) could run on anything, even Win95, Win NT, OS2, SunOS, you name it. So if some Microsoft fan realy wanted to see stars scroll by on his Win95 screen he could simply write a Win95 display client using the winsock.dll library. In reality, serveral clients would run on the same box as the server. If one of them became a CPU hog it could be moved ___without even a recompile___ to a second Linux machine on the local net.
Another thing about sockets is that unlike files, sockets are bi-directional. You can both read and write to them after they are connected. So a client, lets call it a "Status and Control Client" could have a simple way to send data (key strokes) back to the pixel server. The pixel server then would do who-knows- what with any incomming data. (Maybe make some iocntl calls on the device?)
Now some details: to establish a socket connection (I'll skip some parameters for now. See /usr/include/sys/socket.h and man pages for details)
There is some room for protocol design. Perhaps each client could be required to send an "ID String" first thing after the connect. Then the server would do a read after the accept. It would then know something about the client connected via that FD.
Now back to Norman's design. Because reading from a socket is very much like reading from a device file, his tm3get11 program could easly become a pixel client. I see only a few lines of code different from a tm3get11 that talks to the device directly vs. one that talks to a socket. Both are done with normal reads. But with the client/server achitechure other clients could be added that run along side tm3get11 and agment or even over time replace it's features.
Michael Richmond
richmond@astro.princeton.edu
Dec 7, 1996
Norman asked "What do you want tm3get11 do to?", so I'll give my response. It would appear that I'll be one of the less demanding users :-)
My desire is to have a single computer do all the following steps:
The idea is to have a remote site with an unattended computer. Therefore, image display isn't necessary, but the ability to start/stop the data acquisition program without pressing any keys _is_ necessary; complicated command-line switches, or interrupts, or signals, or anything would be okay by me for ocntrolling the data acquisition program.
To use Chris Albertson's terminology, I would like to have a "FITS server". Moreover, I'd like to have one that doesn't require a human to press keys (although, of course, it will be handy to be able to do that for commissioning or special purposes).
I wouldn't be surprised if each person with a camera wants to use it in a slightly different way. I hope that we don't put too many demands on our Gaffeur :-) But the impression I get on the tm3get11 design leads me to believe that it won't be hard to make the program do what I want, and I'll be able to modify it if necessary.
Chris Albertson
chrisa@wavenet.com
Dec 7, 1996
Michael,
Have you noticed the UNIX "crontab" feature? It is a feature built into every UNIX system. Basically it is a table containning a schedule and some program names. You fill in the table and then the system will automatically run the programs at the specified time.
The programs can be run one-shot or periodically with a period specified in anything from seconds to months. You should be able to do what you want without any extra work on Norman's part. One of the reasons for porting to Linux is access to system features like this one.
Nick Beser
beser@aplcomm.jhuapl.edu
Dec 8, 1996
Chris Albertson wrote:
> Question for the rest of the group: Would the ability to > easyly write "clients" as discussed below be a valueable > feature? > ... > Now here is where my design would differ sightly from Norman's > plan. (It may seem like a large departure at first but at the > end of this message I argue that it is not.) > .... > > Now back to Norman's design. Because reading from a socket > is very much like reading from a device file, his tm3get11 > program could easly become a pixel client. I see only a > few lines of code different from a tm3get11 that talks to > the device directly vs. one that talks to a socket. Both > are done with normal reads. But with the client/server > achitechure other clients could be added that run along side > tm3get11 and agment or even over time replace it's features.I like your idea, and I think that you and Norman are thinking along the same lines (with your e-mail supplying much of the implementation details). Another advantage of this concept, is that the applications working off the socket interface can be hosted on lower power (cheap or free) computers. From the standpoint of the APL observatory, I can beg and borrow many old 486 computers that are being surplused at the Lab, and distribute the processing among them. There is a traffic overhead issue which may not be a big factor if we are able to do multicast of sockets rather than multiple socket broadcasts of the data.
> Hmmm. I wasn't aware that the dark current changed by so much. > I wonder if there can be differences from camera to camera, orThe data I have is all from one triplet. I see conciderable differances between the CCD chips. On order of 100 ADU. Night-to-night variation of same CCD about the same. The exact relationship between the values of the "edge" or "masked" pixels and the image pixels appears to depend on the chip.
> even depending on the settings. I've certainly noticed that > when I start running, it takes some time for the temperature of the > chip to equilibrate -- and the time it takes can vary quite a bit,What I am calling "dark current" is realy "dark curent plus bias curent" The Mk III adds about 8000 ADUs just to insure we do not get negative numbers. Much of the drift both short term and night to night may be due to drifts in the bias current. Either way this still needs to be subtracted out. As I said in a previous post it looks like this can be done to within 1 ADU at least for the test data I have.
To Michael and anyone else with a working camera,
If you would send some dark frames from your system (put lens cap on) I'll see if the method applies to other triplets. Take some data both early just after the temp stabilizes and late after in has been stable for some time. (Please modify the tass.rc file so your serial number is correct.)
> depending on the temperature setting. I'll have to look over a > set of long dark frames very carefully.What I do is average all image pixels one a scane line and plot the average vs. scan line number. See ftp://ftp.logicon.com/open/chris/dark.gif for an example plot.
Chris Albertson
chrisa@wavenet.com
Dec 8, 1996
> > Question for the reast of the group: Would the ability to > > easyly write "clients" as discussed below be a valueable > > feature? > > Chris, > > I like your idea, and I think that you and Norman are thinking along > the > same lines (with your e-mail supplying much of the implementation > details). Another advantage of this concept, is that the applications > working off the socket interface can be hosted on lower power (cheap > or > free) computers. From the standpoint of the APL observatory, I can beg > and borrow many old 486 computers that are being surplused at the Lab, > and distribute the processing among them. There is a traffic overhead > issue which may not be a big factor if we are able to do multicast of > sockets rather than multiple socket broadcasts of the data.Yes I left out some detail in my socket write-up. The "socket" call, which creates the socket allows you to choose a protocol "famaily" and a protocol. Basically the choises are UDP or TCP. UDP allows a broadcast where you send the packet once and all clients see it. TCP adds error checking and garenteed in order delivery but is point to point so you need to send the packet once per client. This is a design issue. With a bit of work you can have it both ways were the clients pick the protocol.
When you are on an _isolated_ Ethernet network trafic is not that big an issue. I would not dump TASS pixel data on your lab's LAN. Ethernet by nature is non-deterministic and is not so good for even soft-real-time unless it is very lightly loaded. If you are going to connect a bunch of 486s together for TASS. I would set up a seperate Ethernet and then stick two Ethernet cards in one of the 486s and let that machine route traffic between the TASS LAN and the lab's LAN. You gain a lot at the cost of only one extra Ethernet card. (A 486 with a few Ethernet cards and serial ports running Linux is the world's cheapest router)
If you have not guessed, this is what I've been doing at work lately: Sockets. The curent project uses loads of them on a few rooms full of computers. So I've learned how to use a hammer and now everything looks like a nail.
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 10, 1996
Tm3get11 for Linux: discussion, architecture, specs.
OK, gang, I'm waiting for comments / offers to write some client program / suggestions for the socket interface specs / etc. There are quite a few question marks here above, let's see some answers!
Chris Albertson
chrisa@wavenet.com
Dec 10, 1996
First I don't mind people disagreeing with me. My thinking on this subject is evolving. In fact I've changed my mind on a few things. I expect I will again . The two main things I am thinking now are
Specific comments follow:
> 1) discussion: > - as Chris has aptly summarized, there are a number of advantages to a > tm3get11 implemented as a server (and accessed via a socket): it becomes > easy to provide newer/fancier client programs > (like GUIs or remote control > programs) and these clients need not be changed when > going from Mark 3 to > Mark 4 etc. Furthermore, while one person is implementing the server, > others can implement various clients inasmuch as the interface has been > clearly defined and fully frozen. > > - there is, however, one point on which I disagree strongly with Chris: to > my eyes, the tm3get11 server should not pass the raw data to clients via > the socket interface - it should create the raw FITS files and only pass > the preprocessed data with 4 bits per pixel > (maybe offer a choice between > 8 and 4 bits per pixel, but I don't know if that is a good idea) via the > socket interface, plus some textual data to be processed or displayed > (i.e.: the contents of the top 3 lines of text and the contents of the > focussing matrix). It should also accept commands from the socket > interface (both commands equivalent to the current keyboard commands and > new commands to control the Auxiliary Data In and Out ports).I think now that the "pixel server" should log the raw data itself also. because.
Now what should pass over the socket. IMO just enough data for the client but certainly not less then enough. Now that we all agree on that statement :-) let's see what it means.
> - why should the tm3get11 server not give direct access to the raw data > lines and why should it store them in the raw FITS files itself?I think you are mixing two subject. Who stores the raw pixels and who as access to them. Access is required for anything like client "c" above. Maybe not truely raw pixels but something like them. Perhaps with a crude dark curent corection applied or some such pre-processing applied.
> Quite simply, to guarantee complete uniformity of format for the raw FITS > files for all TASS installations. If that is not done by the server, > everybody and his sister will want to have its own client program storing > these raw files in his own favourite formatI agree. the server writes the raw FITS file. But it could also send out 16-bit pixels to those who need them for other reasons.
> (say, stripping the non-pixel > data off the file, for instance, or using a fancier header format with > angles specified - say - in radians or in hours instead of decimal > degrees). To put things bluntly: either I keep the format quite uniform > by providing the software that writes these files, or we will be unable > to exchange raw data files among ourselves in no time flat.I like to see the FITS files be uniform too but I hope we do not learn to depend on that uniformity. This would mean that a change would be impossable because it would break lots of data reduction code. A good FITS reader should be able to handle any odd FITS file thrown at it. For exampe Michael Richmond sent me some TASS files that are not at all like the ones tm3get writes. I was still able to make a average dark current vs. time plot with his files.
> - I'm sorry to have to say things this bluntly, but I have some experience > (not as much as Tom, but still over 22 years of it) working with teams of > both programmers and scientists (beeing one of both gangs) and one thing > I certainly learned: either someone freezes file formats and interfaces > by implementing them himself, or soon teamwork breaks down under the > weight of a multitude of slightly incompatible file formats and > interfaces. Hence, the tm3get11 server will write the raw FITS files. > Any program that must access the raw data will have to read these files.For a project as loosely organized as this one that is how they work: The first person to do something sets the standard. However the standards that last the longest as those that are robust and flexible. I think FITS helps us that way.
> 2) architecture: > > - Linux kernel modification > * I'm using Linux kernel 1.3.20 (is this what you wanted to know, Nick?) > * this part is written and undergoing tests right nowMy only concern here is that when new Linux kernel come out will they break your patch file? I assume the clock tick stuff has been stable for a long time. I'm hopingyou are not making yoursef a life-long job of updating the patch for every new kernel version.
> - /dev/tassmk3 device driver > * it is a character-based device > providing a one-line-at-a-time interface > (when using a single camera) or a > three-lines-at-a-time interface (when > using a full triplet) to the hardware for the server > * this part is also written and undergoing tests now > > - tm3get11 server program > * the server reads and processes the tass.rc configuration file > * it then sets-up the TASS triplet and starts > collecting data and storing it in raw FITS files > * it prepares the 4-bits per pixel display lines, > the 3 top lines of text and the focussing matrix data > * it provides access to these image and textual data via the socket > interface > * it also provides access to the programmable facilities of the device > driver (the Auxiliary Data In and Out ports) via the socket interface > * it is also controllable via the socket interface (update TEC and VCO > ADC's, choose one camera for focussing, start/stop focussing calcs, > clear/ramp-up the CCDs, start/stop storing data file, shut-down, etc.) > * I'm writing this part - luckily enough, I wasn't too advanced here and > do not have to throw away any of my code to make it a server > * THE SOCKET INTERFACE HAS STILL TO BE DEFINED (see further down) > > - X windows client program > * must be able to display images from one camera or from 3 cameras > simultaneously, together with the usual top 3 lines of textThat is one design. It could also be that a display client shows only one camera and you simply run three clients, one for each camera. After all some lucky person may happen to have three spare computers and want to use three CRTs one for each CCD. If I were to run a TASS system at my work we would have endless UNIX mmachines on which to run clients.
I am not arguing for any client design now. But I do think the server should not constrain the design of any clients. With X11 the display could be done in overlapping windows.
> * must be able to display the focussing matrix together with the image > from a single camera > * must provide the user with a full set of commands (to update the TEC > and VCO ADC's, to choose one camera for the focussing calculations, > to start/stop the focussing calculations, to clear/ramp-up the CCDs, > to start/stop storing raw data files, to shut-down the server, etc) > to be transmitted to the tm3get11 server program via the socket > interface > * who wants to write this part?I agree all the above requirements must be met. However I see no reason to require one client to do all of them. For example how about a display client, focus client, and control and status client?
> - telecontrol client program > * must build current/recent GIF or PGS files > (from the 4-bits image lines) > to be accessed by the WEB or FTP > * must build current/recent WEB or TEXT pages showing the top 3 lines of > text with (when appropriate) the contents of the focussing matrix > * must provide the remote user with commands similar to these provided > by the X windows client, plus commands to > download current/recent pages and image files > * who wants to write this part? > > - svgalib client program > * must be equivalent to the X windows client program > * may possibly not be necessary (would be useful on smaller machines - > like 486'es - without X windows server) > * who wants to write this part? > > 3) socket interface specifications: > - should the server provide access to more than one client at a time?This is easy to do. But you can run out of resources on a small mmachine. You could use an environment variable to define the upper limit. Yes I think you need more then one or the only client that could run would be some kind of "do everything" supper client. It would defeat the purpose of using a server.
> - should we use datagrams or streams?My idea is to let the client specify. There could be two sockets, the client connects with the one it wants.
First thing any client must do after connecting is write on the socket a "request for pixel data." In the request would be the number of bits per pixel and wether it wants image, "edge" or both types of pixels. Maybe alllso the type od scaling to be performed.
Version one of the server could limit the options the client may ask for to say only one option. Later versions could offer clients more.
> - what sockaddr (socket address) should we bind to the socket? > - what data format should we exchange across the socket interface, i.e.: > how do we packetize the data we must exchange? > - here is what the server must provide to the clients: > * one image line (768 pixels of 4 bits) with its camera number (0, 1 or 2) > and line number since beginning of night (long int)Or 8-bit or 16-bit data as requested by the client.
> * top 3 lines of text > * focussing data (5 lines of text) > * contents of the Auxiliary Data In port > - here is what the clients must provide to the server: > * data for the Auxiliary Data Out port > * user commands to update the TEC and VCO ADC's, to choose one camera > for the focussing calculations, to start/stop focussing calculations, > to clear/ramp-up the CCDs, to start/stop storing raw data files, to > shut-down the server, etcAlong with the "request for pixel data" would be commands for the above as well.
> OK, gang, I'm waiting for comments / offers to write some client program / > suggestions for the socket interface specs / etc. There are quite a few > question marks here above, let's see some answers!Well I think the first think to write is a "dummy server" one that always serves the same line of pixels. The next program would be a "dummy client" that just reads the pixels and dumps them in the old bit bucket. These two programs would define the interface and the dummy client would serve as a model for other clients. In fact it would mmake writing a new client easy.
Tom Droege
droege@fnal.gov
Dec 11, 1996
> - why should the tm3get11 server not give direct access to the raw data > lines and why should it store them in the raw FITS files itself? > Quite simply, to guarantee complete uniformity of format for the raw FITS > files for all TASS installations. If that is not done by the server, > everybody and his sister will want to have its own client program storing > these raw files in his own favourite format (say, stripping the non-pixel > data off the file, for instance, or using a fancier header format with > angles specified - say - in radians or in hours instead of decimal > degrees). To put things bluntly: either I keep the format quite uniform > by providing the software that writes these files, or we will be unable > to exchange raw data files among ourselves in no time flat.I pretty much agree with Norman. He who writes the software gets to set the standard. I would rather live with a fairly good standard than a lot of very good ones! ;^) On the other hand you all have an opportunity to persuade Norman to your way of thinking. I am sure that if we all get together and decide that we really want thins done in one particular way that Norman will listen.
Norman also listed a number of things ending with:
* who wants to write this part?
I have enough parts here and on order to make a few more single cameras. I will try to get a camera to anyone who needs one to work on software.
Glenn Gombert
ggombert@mail.erinet.com
Dec 12, 1996
I have been doing some more experimenting with the Automated Telescope Tool Kit from the University of IOWA and have tried several of the tools out on several of the TASS images taken here in Dayton several weeks ago. One of the first calibration steps to using the ATF Tools is to calculate the "WCS" (World Coordinate System) values that are inserted into the FITS header of the images that it is desired to use the tool kit with. The WCS coordinates appear to be a very power set of information that allows many sophisticated measurements to be made from sets of images that have the WCS coordinates added to the FITS image headers. The set of programs that I downloaded over the weekend operate under Windows95/NT . I am in the process of installing the same set of tools that operate under Linux.
The procedure that I used to get the RA and DEC of the images that had the WCS information added to the header was from Michael Richmonds automated astrometry server. Once the approximate RA and DEC of the each TASS image field center was entered into the header the "WCS" program in the AFTTool kit which detected (automatically) the stars in the FITS image and then extracted reference stares from the HGSC CDROM selected the 25 brightest stars from each image and performed a pattern match on the selected stars from the reference image and the HGSC. Once this was accomplished the WCS FITS header parameters were calculated and added to the FITS image header.
Adding WCS parameters to FITS images allow some sophisticated image analysis to be performed in an automated fashion, such as specifying certain variable stars which to produce light curves for and checking images taken during different nights to check for new stars that were present in one image and not another (SuperNova Searches, etc.). The reset of the tools in the "tool kit" depend on the WCS coordinates being added to the FITS image headers.
Also over the weekend I downloaded and installed the immatch (image matching package) that is an add-on to IRAF. The immatch package also has a number of functions to use and manipulate FITS format images that have the correct WCS parameters setup in the header. In short the "WCS parameters " appear to be very useful information to have in TASS images in order to automate the data reduction process that confronts the mountain of data that results from just one evenings operation (120 megabytes collected Saturday night / Sunday morning).
Maybe Michael Richmond can expound on the "philosophy" behind the World Coordinate System fields in FITS format images and how they might be fully utilized to enhance the automated data analysis of TASS images that we will want to pursue in the future. It might be a good thing to add to a future version of "tm3get" in the future to calculate and insert these parameters into FITS header images. This could also greatly help to compare images taken at different TASS locations / camera's as more and more data is collected once these parameters are added to all image headers.
Glenn Gombert
ggombert@mail.erinet.com
Dec 12, 1996
While surfing the WEB looking up "info" on the World Coordinate System I can across a very interesting program developed by the European Southern Observatory called "SkyCat" which has some very interesting features AND it has been ported to Linux! Below are some of the features that I copied from the SkyCat Home Page.
I am too tired tonight to download and install it but it looks VERY interesting...
------------------------------------------------
SkyCat is a tool that combines visualization of images and access to catalogs and archive data for astronomy.
See the SkyCat FAQ for more information.
SkyCat features
Open and visualize a variety of FITS images including support for World
Coordinate System (WCS),
interactive measurement of offsets and other standard visualization
functions (SAOimage-like);
Overlay and edit color graphic objects on the image, like `tagging'
sources with text, arrows, circles or other
graphic elements such as masks;
Display a world coordinates based grid over the image with the
resolution given in arcseconds.
Display a compass indicating where north and east are in the image,
based on the world coordinates
information.
PostScript color printing of the display (image + graphics);
Access and load an image from a network server of the Digitized Sky
Survey scans;
Access and load catalog information from a number of popular
astronomical catalogs like the HST Guide
Star Catalog and others;
Access local user catalogs.
Save catalog data locally.
Overlay catalog sources on an image;
Interact with Netscape to display more object information when
available (support for Mosaic may be
addded in a future release);
Access the observations catalog from the NTT, HST and CFHT Science
Archives;
Access to SIMBAD and NED both as name resolvers as well as for
information on known objects;
Retrieve preview and other compressed images and decompress them on the
fly.
Retrieve and plot tabular preview data for a selected object as an X/Y
graph.
Calculate, display and plot the center position, FWHM, angle and other
information for a selected
star/object.
Load compressed images in hcompress, gzip or unix compress format.
Access SkyCat features from a remote process via socket interface or Tk
"send".
Access image header and data (FITS format) via SysV shared memory or mmap.
Interact with WWW browser to access catalog documentation and other
documents.
Things we are working on
Save canvas graphics possibly as a fits extension on the image file.
User customizable symbols to "label" objects on the canvas.
Load lists of catalogs from sites (ESO, CADC, CDS, local) and and allow
user to select his/her preferred
default catalog list.
Save user preferences for look-up tables and other settings.
Add or edit WCS parameters for a given image.
Caveats
Astrometry: SkyCat handles the astrometric positions by translating pixel positions into equatorial coordinates (RA, DEC). This translation is based on WCS (World Coordinate System) FITS keywords that are included in the image header and that give the astrometric solution for the image. The accuracy of the solution varies because, in many cases, an approximation is used. Colors: If you are running netscape or some other color intensive applications before you start SkyCat, you may get some color flashing when you move the mouse in and out of the SkyCat image window. This is because SkyCat is using a private colormap to get enough colors to display the image. You can get around this problem by starting netscape with the "-ncols" option: for example:
netscape -ncols 60 &
You can also control how many colors SkyCat uses via the color dialog in the View menu.
About the software
SkyCat was developed by ESO's Data Management and Very Large Telescope (VLT) Project divisions with contributions from the Canadian Astronomical Data Center (CADC).
The tool was originally conceived as a demo of the capabilities of the class library that we are developing for the VLT.
This library consists so far of two major subsystems:
The Real-time Display (RTD) image classes with the RTD widget
(see The Messenger, 81, 1995), and
the Catalog Interface classes with the CatSelect widget.
These constitute the basic building blocks of SkyCat. Early demonstrations of the tool to users were received so enthusiastically that we decided to polish things up and produce the current version.
Java vs Tcl/Tk
Some users might wonder why we did this development in the Tcl/Tk environment rather than in Java. The main reason is the pragmatic need to get the functionality implemented in time for when the VLT comes on-line (mid 1997). However, great care has been taken to develop as much as possible in the short time with object oriented languages (C++, [incr Tcl]), having in mind that the future lies in tele-scripting rather in the distribution of binary code.
SkyCat mailing list
A mailing list has been setup to support a wide collaboration on the SkyCat/RTD/CatLib project. Many people have shown insterest in participating in such a venture. You can sign up with that list here. A hypermail archive is also available.
Distribution and support
The SkyCat binaries are freely available to any users who want to download and use the software at their own risk. Users to wish to modify the source code are should contact malbrech@eso.org.
SkyCat is available as an executable for these platforms:
Solaris-2.5 (SunOS-5.5) Solaris-2.4 (SunOS-5.4) SunOS-4.1.3 HP-UX-B.10.10 HP-UX-A.09.05 Linux (courtesy of S. Sani)
ESO will maintain the Sun Solaris and HP versions in the longer term. They are the platforms on which VLT software will run. ESO does not have the resources to port to and maintain SkyCat on any other platforms. We will be glad to redistribute any port that other people or groups may support but decline any responsibility for them.
Chris Albertson
chrisa@wavenet.com
Dec 13, 1996
As I said, I'd offer to write a client server socket interface for the Linux version of tm3get. I have started work on a prototype. The prototype just shows that all the required system calls work under Linux and do what is expected.
The prototype consists of two demo programs. Nothing TASS specific yet. My point was just to see how much effort it would take. The basic setup took only a few hours.
I have a stand alone server and a matching client. All they do is connect then send each other messages then quit.
The client looks for an environment variable called TM3GET_HOST. If present it will read the value associated with TM3GET_HOST and connect with a server running on that machine. If the environment variable is not present, the client will connect with a server running on the local machine.
Both server and client look (by use of getservbyname) at either the local /etc/services file to get the proper port number for the "tm3get" service or if NIS is set up and has a services map getservbyname() will look there.
The system uses Internet Domain sockets bound to a port defined in /etc/services. After doing some research I think we realy should use the TCP protocol. It would not be a lot of work to add UDP capabilty later. The problem with UDP that error checking, re-tries, and timeouts must all be handled be the client and servers. TCP makes this the responceability of the system. UDP is something we can add later (with no need to remove TCP) if required and the change will be transparent.
The above all seems to work. I need to add better error checking. Currently the software checks for an error after every system call but just prints the error and quits. Not what I call "production quality" I still need to test odd things like a connection between a Sun Workstation client and a Linux server accross the Internet but this "should" work. Yes it should be possible to control a TASS camera located anywere in the world from your home computer given enought bandwidth.
Still to be determined is the format of the data passed between client and server. The prototype simply sends a big binary buffer with an ASCII test message like "Hello server" or "Hello client" stuffed into it.
My proposel is that:
Code zero is reserved to indicate so-called "out of band" data where the scan line actually contains some kind of message of command from server to client. I am sure we will think of a use for this.
I think this is simple but allows for expansion later. At first the server would offer only one or two differnt choices of scan line type.
Chris Albertson
chrisa@wavenet.com
Dec 13, 1996
As they used to say on Monty Python, "And now for something completely different."
Please don't take this too seriously. This is intended as pure entertainment. After hearing about all the pros and cons of various proposed camera designs I think I have thought of a new type. Just wondering, is this a new idea and would it work?
A while back someone on this list described a drift scan camera he had built that used a mirror. The CCD camera was faceing backward to the mirror but offset from the mirror's optical path so as not to obstruct the incomming light. There was some optical distortion but nothing that software could not iron out later.
My idea is to extend this. Make a _ring_ of CCD chips a little larger then the mirror and one focal lenght away. An 18 or 24 inch mirror is expensive but if it could be shared by 20 of so CCD chips the total cost could be lower. With a mirror that large there is room for two concentric rings.
So here is a way to use 50 or so cheap CCD detectors to make the equivelent of that 2 square inch, multi-megapixel CCD we can not afford.
I think this design could be improved at little cost: Double the focal lenth of the mirror then place a small positive lens just in front of the CCD to bring the _efective_ focal lenght back down to what it was. This reduces the offset angle from the mirror's axis and (we hope) optical distortion. Lens tilt and off-axis shift could re-align the plane of focus back onto the chip's plane. (Those of you who do large format photography should be able to visulize this better.) Now one more improvement, don't double the mirror's focal lenght but keep the lenses. Now you can have a large diameter f/2.0 optical system made with a standard issue dobsonian mirror.
I got this idea while reading about "non imaging optics" It turns out there is a whole class of optics that compute functions other then the one to one mapping that camera and telescope optics perform. Now I was thinking that if no one were ever to look at an image but were to simply dump it into a computer for analysis there would be no need to make such nice distortion free images. The above is a very mild example. I think all you realy need to do is channel the light flux from the sky onto the detector in some understandable way.
John Cooper
JBCooper3@aol.com
Dec 13, 1996
> I think this design could be improved at little cost: Double > the focal lenth of the mirror then place a small positive lens > just in front of the CCD to bring the _efective_ focal lenght > back down to what it was. This reduces the offset angle from > the mirror's axis and (we hope) optical distortion. Lens tilt > and off-axis shift could re-align the plane of focus back onto > the chip's plane. (Those of you who do large format photography > should be able to visulize this better.) Now one more improvement, > don't double the mirror's focal lenght but keep the lenses. Now > you can have a large diameter f/2.0 optical system made with a > standard issue dobsonian mirror.It's always easier to gun down someone than to come up with an original idea. That said, here goes (out comes six-shooter)... Two main concerns (one of which can be eliminated by throwing money at the problem) are that the usable FOV may not be large enough to fit more that a few chips in (inlcuding packaging constraints); and the auxiliary lens proposal doesn't give f/2-type images, since the cone of light that each chip uses is smaller than f/2.
Gunning me down is welcome, I don't mind being proven incorrect at all (especially if it betters The Cause!)
Joe Ronchetti
jronchetti@earthlink.net
Dec 13, 1996
Has anyone named one star on any tass image? If so send me the image and identify the star.
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 13, 1996
> Please don't take this too seriously. This is intended as pure > entertainment. After hearing about all the pros and cons of > various proposed camera designs I think I have thought of a > new type. Just wondering, is this a new idea and would it work?Some details should be ironed out of your design, but it is indeed a workable concept, and an interesting one at that - it would be perfect, for instance, for an amateur space telescope. It would also be great for an automated amateur telescope.
> My idea is to extend this. Make a _ring_ of CCD chips a little > larger then the mirror and one focal lenght away. An 18 or 24 > inch mirror is expensive but if it could be shared by 20 of so > CCD chips the total cost could be lower. With a mirror that large > there is room for two concentric rings.Well, it depends. The distortion on the outer ring would be much larger than on the inner ring, unless one used a slightly hyperbolic mirror - and these are more difficult to make (meaning more expensive) than parabolic ones or (cheaper still) spherical ones. Now, using a spherical mirror of f/16 or lesser aperture would still give an excellent image all around the ring, with quite acceptable distortion.
> So here is a way to use 50 or so cheap CCD detectors to make the > equivelent of that 2 square inch, multi-megapixel CCD we can not afford.It is indeed feasible, but there is a drawback: such a system would image a ring in the sky, with a huge blind spot at the center of the ring. When you want to image something, you have to offset the telescope so that what you want to see falls on one of the CCDs.
> I think this design could be improved at little cost: Double > the focal lenth of the mirror then place a small positive lens > just in front of the CCD to bring the _efective_ focal lenght > back down to what it was. This reduces the offset angle from > the mirror's axis and (we hope) optical distortion. Lens tilt > and off-axis shift could re-align the plane of focus back onto > the chip's plane. (Those of you who do large format photography > should be able to visulize this better.)This would also effectively multiply the price of the system by a large number. a small positive lens would magnify the distortion unless it was adequately corrected, i.e.: a small triplet - With a triplet, one could indeed choose the lenses in such a way as to largely compensate the distortion caused by the mirror offset.
> don't double the mirror's focal lenght but keep the lenses. Now > you can have a large diameter f/2.0 optical system made with a > standard issue dobsonian mirror.No, no, definitively no. A standard issue dobs mirror (say an 18 inches f/4.5 first surface parabolic) is already a tad expensive (to put it mildly), but it is also a very highly distorting optical system when used off-axis. Lenses to correct the gross distortion and at the same time reduce the focal length would cost an arm and a leg - each CCD would have to be fitted with such lenses, each set costing at least as much as the standard issue mirror. Your off-center ring telescope cannot be inexpensive unless one uses very small apertures (f/16 or smaller) and no correcting optics.
> I got this idea while reading about "non imaging optics" It turns > out there is a whole class of optics that compute functions other > then the one to one mapping that camera and telescope optics perform.Right, but the one to many mappings provided by imperfect optics such as the one you propose are useless unless one has a lot of computer power to deconvolve the images thus obtained. Think in terms of one Pentium per CCD chip. The simpler the mapping (i.e.: the nearer to "one to one"), the cheaper and faster it gets to extract usable image data. The Hubble mirror saga is a good case in point.
> Now I was thinking that if no one were ever to look at an image but > were to simply dump it into a computer for analysis there would be > no need to make such nice distortion free images. The above is a > very mild example. I think all you realy need to do is channel the > light flux from the sky onto the detector in some understandable way.As I said, the simpler the mapping, the cheaper the calculations required to make the data usable.
Another drawback of your system is that it can only be used in stare mode, or in stare-and-skip mode, unless you have found a way to make sure that all the CCD chips are perfectly parallel to each other.
All in all, I think your idea is certainly worth investigating in more detail, but in order to keep its price down, one would have to restrict oneself to non-corrected single mirrors, preferably of small numeric aperture (an 18" f/16 mirror has a focal length of 288" or 24 feet (about 8 meters), talk of a long tube!
Finally, there is a small remark that ought to be stated: at best, one could build a third of a ring around the mirror, because the remainder of the ring would obstruct the optical path for the CCDs on the first third.
Joe Ronchetti
jronchetti@earthlink.net
Dec 13, 1996
Why place the chips in a ring? Instead a grid of chips collecting light from the center of the mirror would give a larger image of the selected sky. Some data would be lost between the chips but overall I think it would be acceptable.(Correct me if I'm missing something.)
Bob Goff
goffaxe@azstarnet.com
Dec 13, 1996
I'd like to throw my hat into the ring. How about an offset matrix like they have at the ccdti at kitt peak.? On axis, of course ;}} ( note the beard )
Chris Albertson
chrisa@wavenet.com
Dec 13, 1996
> > Please don't take this too seriously. This is intended as pure > > It is indeed feasible, but there is a drawback: such a system would image a > ring in the sky, with a huge blind spot at the center of the ring.I was thinking of a TASS style survey device. The goal was to maximize the square inches of both silicon and glass
> No, no, definitively no. A standard issue dobs mirror (say an 18 inches > f/4.5 first surface parabolic) is already a tad expensive (to put it mildly),Realy, I figured just a few hundred dollars. Anyway I'll make a one CCD version someday out of my CB245 and a maybe 6" mirror. It would only cost me the price of a mirror. With luck we'll have analysis software long before I set up the hardware.
> Right, but the one to many mappings provided by imperfect optics such as > the one you propose are useless unless one has a lot of computer power to > deconvolve the images thus obtained. Think in terms of one Pentium per CCD > chip.We have huge amounts of computting power going to waste at my office. There are multi-CPU UltraSPARC and SGI boxes sitting idle 12 hours a day (or night) Just one of those UltraSPARC CPUs is like 5 or 8 Pentiums. Has 1/2 a gig of RAM too. 6 foot tall DEC Alpha servers upstairs along with VAXes. Well you get the point. Hundreds of mega- FLOPS setting Idle all over the country and mostly at night. Yes I know, way hard to put this to use.
> As I said, the simpler the mapping, the cheaper the calculations required to > make the data usable. >I read about one optical system that projects a Fourier transform. The math was way over my head and I don't remember the details. It was part of a device used by the Air Force to process Synthetic Apiture Radar data. We were building a digital replacement for it.
> Finally, there is a small remark that ought to be stated: at best, one could > build a third of a ring around the mirror, because the remainder of the ring > would obstruct the optical path for the CCDs on the first third.Yah, I see no way out of that.
Norman Molhant
molhant@ERE.UMontreal.CA
Dec 14, 1996
I wrote:
>> It is indeed feasible, but there is a drawback: such a system would image a >> ring in the sky, with a huge blind spot at the center of the ring.And you replied:
> I was thinking of a TASS style survey device. The goal was to maximize the > square inches of both silicon and glassRight. Taking into account the unavoidable optical path obstruction caused by an off-axis ring, as explained in my previous message, such a device could image one third of a ring in the sky. If all CCDs are kept parallel to each other, one could indeed use drift scan to gether data along a relatively large swath in the sky. There would be one problem with that, though: your system would require a very small aperture, as explained in my previous message, something like f/16 or fainter, hence it would have a long focal length, hence the sky would drift really fast in its field of view, hence your system would gather only very little light in drift scan mode. I think it would be more useful in stare and skip mode.
>> No, no, definitively no. A standard issue dobs mirror (say an 18 inches >> f/4.5 first surface parabolic) is already a tad expensive (to put it mildly) > Realy, I figured just a few hundred dollars. Anyway I'll make a one CCD > version someday out of my CB245 and a maybe 6" mirror. It would only > cost me the price of a mirror. With luck we'll have analysis software > long before I set up the hardware.Looking at the ads in a 6 months old Sky and Telescope, I see that Nova Optical Systems sells an 18 inches parabolic between f/4 and f/10 for US$ 1800, for instance. I don't call that cheap. Their 24 inches was US$ 3650. Matter of fact, anything above 13 inches is quite expensive in their price list.
>> Right, but the one to many mappings provided by imperfect optics such as >> the one you propose are useless unless one has a lot of computer power to >> deconvolve the images thus obtained. Think in terms of one Pentium per CCD >> chip. > We have huge amounts of computting power going to waste at my office. > There are multi-CPU UltraSPARC and SGI boxes sitting idle 12 hours a day > (or night) Just one of those UltraSPARC CPUs is like 5 or 8 Pentiums. > Has 1/2 a gig of RAM too. 6 foot tall DEC Alpha servers upstairs along > with VAXes. Well you get the point. Hundreds of megaFLOPS setting Idle > all over the country and mostly at night. > Yes I know, way hard to put this to use.The difficulty resides in the fact that solving an inverse optical problem (that's what you are thinking about) is relatively easy when the light sources are point sources (stars, for instance) and the PDF is constant over the field of view, but becomes much more difficult when the light sources are extended (galaxies) or the PDF varies with position in the field of view (which would be the case in your off-axis third-of-a-ring camera). The difficulty reaches its worst when both conditions are met.
>> As I said, the simpler the mapping, the cheaper the calculations required to >> make the data usable. > I read about one optical system that projects a Fourier transform. The > math was way over my head and I don't remember the details. It was part > of a device used by the Air Force to process Synthetic Apiture Radar > data. We were building a digital replacement for it.Right. I have used such optical Fourier transformers for crystallographic work ('t was 23 years ago ;^). I know these kinds of setup quite well. What we used then was this: a point source (a laser is better, due to its better coherency) is set at the front focus of a first lens, thus projecting the image of the point source to the infinite. At the back focus of that lens, we place the image (a transparency) to be Fourier transformed. At twice the same distance, i.e. still further away from the first lens, we place a second lens with the same focal length as the third lens, so that our transparency is at its front focus and its image is projected to the infinite. At the back focus of that lens, we have (superimposed on each other) both an image of the point source (easy to block with a single black dot on a transparent mask) and the Fourier transform of our transparency. Matter of fact, the image of the point source is just the DC (zero frequency) component of that Fourier transform. Naturally enough, the system worked better with monochromatic light and well corrected lenses, still better with a laser. One important point: the more extended the point source, the worse the result, because an extended source causes a spreading of the frequencies in the Fourier transform, hence a blurring of that transform. It is not easy to see how to apply a similar process to starlight, I'm afraid.
Glenn Gombert
ggombert@mail.erinet.com
Dec 15, 1996
Proposal For a TASS Automated Variable Star Monitoring Project
Using several of the programs in the WCS "ToolKit" it should be fairly easy to implement an elementary variable star observing project with some minor modifications of existing stuff that should get us down the road to "automated" data reduction of TASS image data, and also set the ground work for more advance projects in the near future. The procedure that I would like to propose is outlined in two parts below. I invite comments questions/ criticism of the approach and would like to see some more concrete use made of the TASS data that is being collected here in Dayton as well as other places.
The first step would be to calculate a general astrometric solution to each TASS image that could is collected . The program WCS.exe from the ATFTool Kit looks like it just might do the job with some additions to the FIT header file that is written out by "tm3get".
The WCS.exe program basically does three things:
For the above to work properly it needs to know the approximate center of the image and several other parameters such as the pixel scale in arc seconds (for TASS images this would be 13.8 arc seconds/pixel) in both the X&Y directions. Norman's program "tm3get" calculates the value of RA based upon the time and the fact that the middle camera is pointed directly south. The RA and DEC values might have to be "fine tuned", perhaps with and offset factor that is entered in for RA and DEC that would have to be determined by each camera location and used to correct for misalignment etc. The center of the filed does not have to be "exact" for the pattern matching algorithm in WCS.exe to work properly. The fact that TASS images are "wide-field" images as compared to say ST-6 images taken at 72 inch focal length should help a lot. There need to be enough of an overlap region between the TASS image and the field center for the pattern matching algorithm to work properly.
I have tried the "WCS.exe" program out over the last few days and it appears to work properly with several TASS images that were centered using Michael Richmonds astrometry server as well as several ST-6 images taken through a 16 inch F/4.5 Newtonian telescope. The pattern matching algorithm appeared to work properly on such popular objects a NGC891 and NGC4565 when the center of the galaxy was used as the center of the field, often times when the object itself was not really centered in the field. The nice thing about the program is that it can (automatically) detect stars in the image, select a region from the HGSC CD-ROM and then do a pattern matching operation between the two to calculate the necessary rotation parameters for the WCS FITS header file variables.
Once this has been accomplished it would then be possible to use the "VSMON.exe" Variable Star Monitoring Program to monitor specific variable stars along the 0 degree DEC path by specifying which objects that we were interested in the RA & DEC parameters and a "reference" star in the same color band located somewhere in the image that has had the WCS parameters calculated for it. The input reference file that the VSMON.exe program uses as input is just a text file with the "target" and "reference" stars defined for each observation that it is desired to make. The program then calculates the differential magnitude ( in the defined color band) and prints out one line of output for each variable on the input list that is found in the image.
The list could star out small with 15-20 stars that it is desired to make light curves of and then expand the list as the first few measurements that are made automatically are verified. This is not as sophisticated as some of the things that we would like to do with TASS image data but would be good first step to possibly providing a general astrometric solution to each TASS image as well as automating some of the photometric solution to measure some variables that are to be present along the path of zero degree declination.
The above program outlined above should be doable with some modifications to the programs and hardware that we currently have with out having to undertake major new development efforts. This would require some careful calibration of each TASS camera site to ensure that the calculated center of the field of each TASS image is as accurate as possible but this seems that it would be possible with some calibration of each site. Hopefully a project like this (less ambitious than making measurements on every star found in a TASS image) would start the flow of good measurements made from the TASS installations that are collecting data at a good rate.
I will try and put together a TASS Applications Note on how the WCS parameters are actually calculated from TASS images. I invite comments, questions, suggestions, etc on the above..
Tom Droege
droege@wwa.com
Dec 14, 1996
When I first started building cameras, I burned out the CCDs. Now I seem to be able to move a CCD assembly from one camera to another and have not hurt one in some time. This is encouraging. Now there are other things that break. Slowly, I am learning this art.
Nick Barnes
nickb@harlequin.co.uk
Dec 16, 1996
> We have huge amounts of computting power going to waste at my office. > There are multi-CPU UltraSPARC and SGI boxes sitting idle 12 hours a > day (or night) Just one of those UltraSPARC CPUs is like 5 or 8 > Pentiums. Has 1/2 a gig of RAM too. 6 foot tall DEC Alpha servers > upstairs along with VAXes. Well you get the point. Hundreds of mega- > FLOPS setting Idle all over the country and mostly at night. > Yes I know, way hard to put this to use.Actually, not too hard to put at least some of it to use. Look at the way the crypto people get millions of MIPS-years together for crypto challenges. For a start, I suspect that on the TASS list alone there are people with access to MFLOP-years per night. On my own I could probably put together a GFLOP-hour per night, most nights. Give me code, data, and a motivation to use those idle cycles....
Joe Ronchetti
jronchetti@earthlink.net
Dec 16, 1996
> So far we have not worried about declination. Most of us get it to > 1/2 degree or so. Plenty accurate if you are using an atomatic > star finding program. Note that we probably never be accurate > enough to try to locate stars by time and camera position. The local > earth moves around too much.I have written the majority of a program that displays raw tass images, allows you to process the image to expose the faintest stars, shows the right ascention and declination of the visible star and when clicked on will identify the star and pertinant information.
Programatically it works but without accurate header information the co-ordinates will not be correct and the identity of the star will not be correct.
If all agree with Tom, then there is no need to continue with the program. If anyone believes that header info and the alignment of the camera can be improved and you wish to work with me to accomplish the task then let me know. I don't have a camera so I need someone with a camera to be willing to work on these issues and make the images available to me for test purposes.
Nick Beser
beser@aplcomm.jhuapl.edU
Dec 16, 1996
The APL TASS observatory will try to have the headers correctly identified. I am also working up image alignment software to calibrated images, so I hope to have very accurate pointing information in the header. If you have some software that can help with that, release it to the group. It may not be used as is, but could be incorporated with other analysis software.
Chris Albertson
chrisa@wavenet.com
Dec 16, 1996
I think Joe Ronchetti in all his recent postings brings up a VERY important point: We (collectively) have working cameras but when you look at the data it is hard to know what part of the sky the camera was looking at. It will be very hard to do any kind of real science if we are unable to attach names to the stars we see. So far it appears that not one person on this mail list is able to do this for the recent crop of FITS format images.
That sounds pretty bad but I suspect an error in the tm3get software.
About pointing accuracy requirements:
I think plus or minus 1/2 degree in DEC and RA is about as good as we can hope for in the raw FITS files. A camera operator may notice how far off his aim point is and simply correct the tass.rc file rather then move the camera. So I think over time the raw header files will improve in accuracy but never perfect as the true aim point moves around based on whatever is supporting the camera. Still any data processing software should allow for a big error box on the aim point.
It would appear however that some posted FIT files have gross errors the the header data. (The ones I put up on ftp.logicon.com for example) These errors are much greater then a 1/2 degree pointing uncertainty. It would be good to locate the source of these errors. Could it be inaccurate values in the TASS.RC file or an error in the TM3GETxx program? The values in these files are what led me to suspect an error in the program.
So, clicking on a star image in a raw FITS file will never give perfect possition. This should be expected just as you cannot ma