Dark Frames
Dark frames are frames which contain no image - only noise, and are typically shot with the lens cap in place. Exposure settings are left the same as for the images frames, and the resulting frames are simply subtracted from the raw image frames. The premise being:
Image + Noise - Noise = Image
Seems fairly straightforward, and as long as the noise is actually contained within both frames in equal measure, it works just that simply. The catch? The QC is wired such that the DC offset of the CCD output is not within the range of the converter at normal exposure settings, and therefore an exposure made without illumination does not have a background level higher than the noise. The exposed frame has the noise + light, and this brings the noise up to where it is digitized. Ooops. Dark frames do not in any way corespond to the noise in an image frame! The fact that hot pixels show up is not evidence of dark signal. The noise we want to capture is the linear background noise from pixels which can be corrected. The hot pixels will be dealt with separately.
Another point which bears mentioning is that the dark frames actually contain the dark signal. Dark noise is simply the uncertainty of the value of the dark signal at any given time. Subtraction of a median dark frame removes the dark signal, but leaves the dark noise. The dark noise has an amplitude equal to the dark signal, according to the manufacturer's data sheets.Averaging the frames in a stack removes the dark noise, along with any other correlated noise, that is noise which is related to a position on the detector, providing you have followed the recommendation under Framing. Keep in mind that this is a divergence from the way a real camera works, because with any real camera, the entire range of outputs of the CCD falls within the range of the ADC.
The artifacts of noise in our QC frames are many and varied. All of the noise of a normal camera is there, but it is completely swamped by thermal noise. There are hot pixels, which are ultra sensitive to light, have an offset from the majority of pixels, and are non-linear, there are pixels which have non-proportional amounts of dark signal, but are otherwise normal, and there are the majority of pixels, which exhiibit linear response to both light and thermal signals. There really isn't any way to sort them out, because at some exposure length, the ones which are prone to saturation will be saturated, and the rest will be somewhere below. All are in need of fixing, but the fixes come in two flavors.
First, the normal, more or less linear pixels can be normalized - that is the dark signal can be subtracted from them, and the result will be the image signal. These pixels are handled by normal dark subtraction, using a frame which accurately represents the dark signal in these pixels. We'll get to just what that is later.
Second, the hot pixels must be removed. There is no image data there. The converter failed to digitize the data because the noise was higher than the ADC reference, or because the pixel was incapable of recording the image data. In either case, these pixels contain no information, and removing them is required. These pixels are called "correlated", because they are in a fixed position on the detector. As mentioned in "Framing", if you don't move the image on the detector from time to time, these pixels are also correlated to the object. A pixel value equal to the maximum value allowed does not in itself indicate this condition, though. A look at the dark frame is required. If the corresponding pixel in this frame is at the maximum value, the pixel in the image frame cannot be salvaged, and must be removed.
Removing a pixel may consist of just "punching" it out of the frame, "weaving" the data around it, or even just blurring the data from a previous pixel into the hole. In some cases it may be preferrable to use a smart algorithm which attempts to determine the nature of the saturated pixel, to decide if it should be left alone, or modified in some way. For example, a saturated pixel in the center of a star, if treated like any other saturated pixel, will leave a black hole in the star. Stacking will fix this. A similar pixel in the core of a galaxy will result in a trail of averaged pixels across the face of the galaxy, which follow any tracking errors in your drive. DarkGen provides four different ways to deal with this noise. Experimentation is always required. I rarely pick the right one on the first try. The object itself generally determines the type of subtraction required. My M42 was done with a mapping algorithm, which blurs the hot pixels out horizontally, while just subtracting the dark from the normal pixels. This same algorithm wiped the Horsehead Nebula out, leaving nothing to stack. A new mapping algorithm, which replaces hot pixels with the average of the eight surrounding pixels had to be made, and then an erosion function to pick off the leftovers, which are artifacts of smearing in the horizontal register, and are always to the right of a saturated pixel (with my setup).
So where does the noise frame come from? I showed you how the data contained in a normal dark frame contains no useful data. Fortunately most readers will have the advantage of living on the bottom side of the atmosphere, and this is our saving grace. The image of the galaxy contains light from the galaxy, noise from our detector, and a nice uniform (more or less) wash of skyglow. This skyglow is not selective - it increases the brightness of the galaxy as well as the space around it. Most of our image is of the space around it. This would be black, were it not for the skyglow. If we subtract the skyglow from the image, we should wind up with exactly that - a black background. Luckily for us, when we take an exposure of this skyglow, it also suffers from dark signal, and when we subtract it from our image frames, we lose both the skyglow and the noise!
The problem is that this skyglow also has our image superimposed on it. The solution is to move the scope off of the image, and shoot an odd number of frames, moving the scope between exposures. We then median these frames, and use the resulting frame as a dark frame. This frame should contain, in perfect proportions, everything we don't want in our image. Those using real cameras would reel at this, but then, they don't have the kind of noise we have. The sky background contains gradients, which with a 12-bit converter may be quantitized. Here, our 6-bit converter can't generally digitize this subtle feature.