[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
RE: Bad pixel determination with better software
Thanks Chris,
With your thoughts, I can write some more software. Although this might
illuminate errors in my flat field creation. Sounds better already...
Cheers,
Rob
Robert Creager
Senior Software Engineer
Client Server Library
303.673.2365 V
303.661.5379 F
888.912.4458 P
StorageTek
INFORMATION made POWERFUL
> -----Original Message-----
> From: Albertson, Chris [mailto:CAlbertson@primeadvantage.com]
> Sent: Wednesday, March 14, 2001 4:27 PM
> To: 'Tom Droege'; Creager, Robert S; tass@listserv.wwa.com
> Subject: RE: Bad pixel determination with better software
>
>
>
> To find bad pixels, I think you'd be better off using
> flat field data then using dark frames. The (well, my)
> definition of a bad pixel is one that has a hugely non-linear
> response to light. In other words it stays low even in bright
> light or hot in the dark. It's the fact that the pixel's ADU
> value is not in proportion to it's illumination that makes it
> bad not the absolute ADU value. So, even if it is stuck at the
> mid point of the range it is still stuck and "bad". Using
> only dark frames gives you only one illumination
> level to test linearity with.
>
> I figure you can determine the general level of illumination
> by the average ADU value of the surrounding couple hundred pixels.
> If you find a pixel that does not respond to changing light then
> Yes, I'd put it in the bad pixel mask.
>
> Dark values are repeatable and can be subtracted out.
>
> All that said, it should be clear that a pixel stays at one end
> of the ADU range in every dark fram is likely "bad". But I think
> these are just the easy to find bad pixels.
>
>
> > -----Original Message-----
> > From: Tom Droege [mailto:tdroege@veriomail.com]
> > Sent: Thursday, December 07, 2000 8:37 PM
> > To: Creager, Robert S; tass@listserv.wwa.com
> > Subject: Re: Bad pixel determination with better software
> >
> >
> > Rob and all,
> >
> > To look for bad pixels, I would use a stack of dark frames
> > from the same
> > detector. I would take as many dark frames as I could get
> > (that is why you
> > have so many disks), sort them by mean value in the middle of
> > the frame,
> > say x 500-1500, y 500 -1500. Then I would take the mean of a
> > bunch where
> > the variation in mean value was small compared to the sigma
> > of the area.
> >
> > Now I would take the stack of comparable images - the above
> > process got you
> > a stack taken at roughly the same temperature - and take the
> > mean of the
> > full frames. Now you can take the histogram as below. This
> > should show up
> > any "hot" pixels. Now you have to go through and "find" the
> > hot pixels by
> > x-y coordinate.
> >
> > OK, the experts will probably jump all over me.
> >
> > Not that this is just the first step to get a dark frame and
> > to identify
> > pixels with high dark current. There are many more
> problems to come.
> >
> > I suspect that you will not find enough hot dark pixels to worry
> > about. But it is worth doing the experiment. However bad
> > they are, they
> > get much better with reduced temperature. The dark current
> > goes down by
> > half for a 5 C decrease in temperature. But it is really
> > much better than
> > that. There are hot pixels that tend to turn off at some
> > temperature. So
> > they go from a high current to near zero as their critical
> > temperature is
> > passed.
> >
> > I do not know of any mechanism that causes change of dark
> > current (at the
> > same temperature) over time. Or for a pixel to be lost.
> > Well, not quite
> > true. A lot of radiation will do it. I suppose that one
> > could get really
> > unlucky and get clobbered by a cosmic ray. But I think such
> > problems heal.
> >
> > For the most part, what you have gotten in the past at any
> specified
> > temperature is what you expect to get in the future. The
> > drift of the
> > electronics should also be small. I would expect the
> > pedestal (the mean
> > value of a dark frame) to change only about 3 counts per C of
> > the ambient
> > temperature. That is the electronics temperature. There is
> > a thermometer
> > that is relatively close to the electronics that are critical.
> >
> > Exercise for the student:
> >
> > 1) Measure the mean value of a bunch of dark frames. Use the
> > center area
> > as above.
> > 2) Plot the mean value of the above against the VCO
> > temperature. (Woops,
> > we are not recording the VCO temperature. So this cannot be done. )
> > 3) Plot the mean value of the above against the CCD
> > temperature. We have
> > been recording this.
> >
> > OK, there are two (main) things that can make the pedestal vary.
> > 1) Dark Current. This doubles for each 5C increas in
> > temperature. (roughly). This can be sorted out from the pedestal.
> > 2) Electronics drift.
> >
> > Lets take the electronics drift first. I expect about 3 ADU
> > per C. This
> > is based on a 100 ppm expected typical component value drift
> > and the 25000
> > ADU pedestal. It will depend on how all the errors add up.
> > While the mean
> > value of the pedestal will move around, the sigma should remain
> > constant. It is just the noise from the ADC and other stuff
> > on the printed
> > circuit board. So the mean value of the frames are expected
> > to move around
> > with the ambient temperature, but the noise due to this
> > movement will not
> > change (much).
> >
> > Now the dark current. The dark current changes the pedestal by
> > accumulating electrons in the pixel. The accumulation of
> > electrons causes
> > the pedestal to shift by 1/2.2 (roughly) ADU per electron.
> > The noise (as
> > expressed in electrons) should increase as the square root of
> > the number of
> > electrons. So pedestal change caused by electronic drift causes no
> > increase in noise, pedestal drift caused by dark current
> > causes the noise
> > to increase as the square root of the number of electrons.
> > OK, lets say we
> > saw the mean value of the pedestal to shift during a long
> > dark exposure by
> > 1000 adu. This means the mean value of the pixel changed by 2200
> > electrons. The square root of 2200 is 47 electrons. So we
> > expect a sigma
> > of 47 electrons. (To get this we compute the sigma of all 4
> > million pixel
> > values) But we measure in ADU which is roughly 2.2 electrons.
> > So we would
> > expect the sigma taken over many pixels to increase by 22
> > ADU. OK, this
> > is a random process. While the mean pixel has a leakage of
> > 2200 electrons,
> > this is *not* 2200 electrons for each and every pixel.
> Because it is
> > random, they will be distributed. Most pixels will be within
> > 47 electrons
> > of 2200. But we have a lot of pixels, so some will be 4 or 5
> > sigma away -
> > so it would not be surprising to find one with 2500 or 1900
> > electrons. Further there are other processes going on, so
> > while a 1800
> > electron pixel would be rare, one could easily have a
> 10,000 electron
> > pixel. (from a "hot" pixel - one where there was some
> impurity in the
> > silicon which acts as a source of electrons) The
> > distribution will be
> > skewed. In fact, if you look at dark frames, you will see
> > that at room
> > temperature the pixel distribution is quite skewed, but as
> > the temperature
> > is lowered the distribution becomes more and my symmetrical.
> >
> > OK, it is never as easy as this. There are lots of noise
> > sources and we
> > hope we can add them all up in quadrature. We have a
> "floor" in this
> > system somewhere around 15-20 electrons.
> >
> > OK, having introduced the topic, I will retire, and let the
> > experts beat me
> > up. ;^) I have tried in the discussion above to keep away
> > from terms for
> > which I do not know the precise definition. I am just trying
> > to outline
> > what happens. The dark current is probably the least of our
> > problems when
> > we can operate the CCDs below -20 C or so. By -30 C it is
> > pretty negligible.
> >
> > Tom Droege
> >
> >
> > At 05:59 PM 12/7/00 -0700, you wrote:
> >
> > >As some of you noticed, my previous histograms were
> > incorrect. Something
> > >about mixing the rows in the columns in CFITSIO routines...
> > Well anyhow,
> > >the question still stands, even though the data looks better
> > (I also removed
> > >7 lead in/out pixels on each row/column). Is there a
> > "standard" way of
> > >determine the locations of bad pixels in the image? Do you
> > use the dark
> > >frame, fabricated flat field? Or do you even not worry
> > about the pixels
> > >being 'bad' unless they are at the extremes (0 or 65535)
> > from a dark? Will
> > >CCD's loose a single pixel, or row/column over time?
> > >
> > >% ./histo h3r1836.524 21
> > > 0 to 3120 = 0
> > > 3120 to 6241 = 0
> > > 6241 to 9362 = 4102506
> > > 9362 to 12483 = 123
> > >12483 to 15603 = 7
> > >15603 to 18724 = 6
> > >18724 to 21845 = 0
> > >21845 to 24966 = 2
> > >24966 to 28086 = 1
> > >28086 to 31207 = 1
> > >31207 to 34328 = 0
> > >34328 to 37449 = 0
> > >37449 to 40569 = 0
> > >40569 to 43690 = 2
> > >43690 to 46811 = 0
> > >46811 to 49932 = 0
> > >49932 to 53052 = 0
> > >53052 to 56173 = 0
> > >56173 to 59294 = 0
> > >59294 to 62415 = 0
> > >62415 to 65536 = 0
> > >
> > >% ./histo h4r1836.524 21
> > > 0 to 3120 = 0
> > > 3120 to 6241 = 0
> > > 6241 to 9362 = 4101496
> > > 9362 to 12483 = 1086
> > >12483 to 15603 = 63
> > >15603 to 18724 = 1
> > >18724 to 21845 = 0
> > >21845 to 24966 = 0
> > >24966 to 28086 = 0
> > >28086 to 31207 = 0
> > >31207 to 34328 = 0
> > >34328 to 37449 = 2
> > >37449 to 40569 = 0
> > >40569 to 43690 = 0
> > >43690 to 46811 = 0
> > >46811 to 49932 = 0
> > >49932 to 53052 = 0
> > >53052 to 56173 = 0
> > >56173 to 59294 = 0
> > >59294 to 62415 = 0
> > >62415 to 65536 = 0
> > >
> > >
> > >Robert S. Creager
> > >Senior Embedded Software Development Engineer
> > >Multi Platform Tape Library Development
> > >Ph: 303-673-2365
> > >Fax: 303-661-5379
> > >Pager: 888-912-4458
> > >StorageTek
> > >INFORMATION made POWERFUL
> >
> >
>