Gee! The 37.5 KHz band isn't even included in this plot. It
was seen in only 3 images. The solid black squares are the
114 Hz signal mentioned above.Orders of Magnitude
Noise Bands
A Funny Coincidence
Revised 1999Mar15.
An image consists of lines of N1 samples [0..N1-1] digitized at interval t (about 10 microseconds). N2 lines [0..N2-1] are digitized at interval T. Pixel[I,J] is thus sampled at time I*T+J*t
Yes - I am associating I with N2 and J with N1. Why? Because I did it that way once and my brain is too rigid to even think of changing it.
A sinusoidal noise at frequency f might be
A*Cos(2*pi*f*(I*T+J*t))Processing this with a 2-D Fourier transform gives an output at
i = N2*f*T Mod N2 in the I or N2 direction
j = N1*f*t Mod N1 in the J or N1 direction
Note that so far the time gap between sampling the lines has not even been mentioned. The gap may be as long as one pleases! This follows from the assumption that we are looking for continuous sinusoidal noise. We are assuming that this continues unabated through the gaps when we are not measuring. We are thus replacing our complete ignorance by an assumption of complete knowledge of what happened during the gap.
Of course, one loses something by having gaps in one's data. If there is no gap, the sample rate in the N2 direction is exactly equal to the resolution in the N1 direction. Each alias (see folding below) in the N2 direction is separated by just one pixel in the N1 direction. If there are gaps in the data, the resolution in the N1 direction is poorer and the aliases are separated by less than one pixel. If the gaps are large, it becomes difficult to sort out the aliases. Even as it is, with quite short gaps, one can get confused if one is as careless as I am. The tiny 114 Hz sinewave was originally measured as 70 Hz, one alias out, and ascribed to a low-flicker monitor.
In the case of the MKIV images on the CD ROM
1/t = 100KHz
T = N1*t + Gap; around 23 msec.
1/T = 44 Hz; folding frequency 1/(2*T) = 22 Hz
Folding is just a little odd compared to the regular 1D spectrum. In one dimension, east and west going waves are indistinguishable. For a sample time of T, the spectrum folds itself onto 0..1/(2*T). In two dimensions one still can't tell which way a wave is going. One cannot distinguish a SE wave from a NW one. But one can tell a SW wave from a NW wave. Or equivalently, a NE wave from a NW one. The spectrum is thus folded once and this can be done in two different ways. I have chosen to fold mine in the I direction so my I or N2 frequencies run 0 to 22 Hz. The other way is not folded so the J or N1 frequencies run 0 to 100 KHz. Or 0 to 50 KHz plus -50 KHz to 0. It's all the same.
Well - nearly all the same.
The folding in the I direction involves replacing
N2*f*T Mod N2 with
N2 - (N2*f*T Mod N2)
when N2*f*T Mod N2 comes out between N2/2 and N2. Looking back at the original noise
A*Cos(2*pi*f*(I*T+J*t)),In order not to change the value of the expression, one must negate the J term as well as the I term when folding.this is equivalent to negating the I term.
This means, plotting I horizontally and J vertically with zero frequency at the bottom, that the point in the 2D spectrum of a sinusoid disappears over the fold at the bottom right as the frequency is increased and reappears at the top right.
Just a little disconcerting.
Some people fold their 2-D Spectra both ways to end up with both ways running from zero to the folding frequency. Fair enough if you really don't care about the difference between NE and NW. I have known people who really did care do it out of ignorance ...
In order to match the abilities of my available spectral analysis program, I have analysed 2016 x 2016 out of the available image area, dropping 8 pixels at each edge. For the quietest images, for example d150004, the rms noise over the whole image is about 6.5 digits (19.5 electrons), cosmic rays, frost crystals and all. The variance is 42 digit^2.
In a 1-D spectrum, this variance is distributed over 1008 bins, giving about 0.042 digit^2 per bin; one can easily spot a bin containing 0.3 digit^2. My trusty Cambridge Elementary Statistical Tables with a confidence level of 0.1% gives 13.81*0.042/2 or 0.029 digit^2, in suspiciously good agreement with my eyeball.
In the 2-D spectrum, the same variance is distributed over 2016 x 1008 bins. Wow! That's only 0.00002 digit^2/bin. No wonder I had to turn up the gain in my display program. Extrapolating the tables to a confidence level of 0.0001% to allow for the fact that there are rather a lot of possibly erroneous points gives a detection limit of 27.63*0.00002/2. or 0.00028 digit^2 (0.017 digits rms).You don't believe it?. Here are plots of a section of the d150004 spectrum where the 60 Hz noise should be. To one of them I have added 0.017 digits rms of synthetic 60 Hz. You be the judge.
The lighter pixels across the bottom are the zero frequency in the N1 direction - basically the coupling time-constants that we have decided to ignore. The bright blob near the middle is part of one of those noise bands variously ascribed to monitors, switching power supplies or Acts of God. I reckon that blob is at 60.4 Hz or thereabouts. The fainter smear above it is presumably another such, though I can't see much of a band in the full image.
Still can't find it? Hint: 60 Hz is aliased once so it ought to come out about one pixel up from the bottom. Um! Not too convincing, is it? If I got my statistics right, all those other points about as bright are signals not noise.
Anyway, even if you get to look at the whole image, there is nothing in that first row of pixels above the zero frequency row that comes close; there is no 60 Hz sinusoidal noise in the image even approaching 0.017 digits rms. There is no 120 Hz such as you would expect from full wave rectifiers. There are no higher harmonics of the mains. I have no idea how Tom Droege managed it!
Since reporting that I could finally see 60 Hz noise in the MKIV CDROM data at a level of 0.3 digit^2 in the expected place, using Tom's figures for sample rates, I have been looking more closely at the data using 2D spectra. I also reported noise signals at a variety of frequencies in the range 18.7 to 40.4 Khz, at least some of which might be monitors (but see COINCIDENCEbelow)
e.g. 800 x 600 VESA 37.8Khz/60Hz.
The 2D spectrum shows that these noise signals plot as bands, covering the full bandwidth in the N2 (E-W) direction but with concentrations at a variety of frequencies (correlated, in some cases, from one band to another). The most powerful concentration is at the alias of 60 HZ. Assuming the source is monitors, this would be the frame frequency.
About 34.5 KHz at the bottom, 45.6 KHz at the top.
13.2 Hz to the left, 19.7 Hz to the right. The lower, rather
broad band is at 37.5 KHz and the narrower brighter one just
above the centre is at 40.4 KHz. Or, of course, at aliases
of these frequencies. The bright
blobs are at 60.4 Hz, aliased; these are the source of
the "60 Hz" that I found in the 1-D spectrum. There is nothing
in the spectrum in the place one would expect to find a 60 Hz
sinusoid.
Interestingly, although there is no 60 Hz sinusoid, there is
detectable 114 Hz. I thought at first it was 70 Hz (the
next lower alias) and that Tom had bought a new monitor -
one of those fancy low flicker monitors that run at 70 Hz
frame frequency. Don't worry: it is about 0.03 digit^2.
The frequency drifts a bit. It is visible on all but 2 or
3 of the December 14th and 15th images but not on the
earlier ones except perhaps dark1123 (which has a number of
other funny things going on at this low level). This lack
of visibility may just be the result of the higher signal
level of the unfiltered images or perhaps something changed.
Could it be something to do with the new RA drive motor
that went in at that point?
Sample frequency 99.61 KHz
1 cycle per 32 pixels = 3.113 KHz
1.1 digit^2 at 18.7 KHz: 6 x 3.113 = 18.7 KHz 0.8 21.7 7 21.8 0.5 37.5 12 37.3 0.7 40.4 13 40.4
At first sight, this suggests there might be a 32-pixel
periodicity - for example in the pixel alignment of the CCD.
However, these bands on d150004 appear at different
frequencies (or not at all) on other images.
Whatever caused them, it is not a periodic flaw in the CCD.
Gee! The 37.5 KHz band isn't even included in this plot. It
was seen in only 3 images. The solid black squares are the
114 Hz signal mentioned above.
There is some evidence in the 2D spectra of 16-pixel
patterning - if you look hard enough for something, you are
likely to find it - but at a very low level.
It looks quite different in the spectrum from the banded
noise which is also shown. You are supposed to be looking at
the regularly spaced bright points on the left of the image
These are at zero E-W frequency; the irregularities, such
as there are, are constant across the CCD. They remain firmly
locked in place from image to image, as one would expect of
something cast in silicon. One day soon, I will get round to
lookining at the numbers, rather than just looking at the
pretty pictures, and find out just how tiny the effect is.
The bright pixels are located at multiples of 63; 2016/32 = 63 so the spatial pattern has a scale of 32. The full image is 2032 pixels which is not a multiple of 32. Details, details!
Note how well these bright pixels align with the broadband noise bands, which are the same ones shown above. Two other bands on this image align just as well. But only on this image, which happened to be the first one I looked at! Just imagine the odds against this - clearly proves alien intervention.