Live Stack Dark/Flat correction - too much "salt" and "pepper"?
Posted: Fri Oct 15, 2021 12:27 am
Hi Robin,
Been endeavouring to do some good Darks and Flats (w/ Dark Flats) to better work with Live Stack, which is a great feature. First attempt saw the Triffid Nebula "emerge from the mists", which was great fun. But that night PHD2 was dropping the guide camera, the full moon was coming up and I forgot the filter! So this is all about better preparation for when the rain stops and the clouds clear ...
I used an iPad as a flat light source. Started at low gain, but ended up at the common gain=120 at -5 C for the ZWO 294MC Pro I use, to do Darks, Flats and Dark Flats (as part of SC 4.0's Flat capture routine). After the Flat capture the histograms for RGB all collapse onto a single peak.
Now I also select Dark subtraction in Camera Preprocessing and get this second histogram with these "side bands" - a few pixels showing either very low or very high values away from the main peak ...
Capturing a frame, debayering in PIPP and loading this into IDL, and zooming in on a small area in the top-left of frame, I see this in the Red channel, with one high and one low pixel ...
I wanted to understand where these divergent pixel values were coming from, so I looked at the same area in the Red channel of the Raw data, Dark and Flat frames. (Note: this "Raw data" is a new capture, with no Dark or Flat correction.) All maps use full histogram stretch ...
Raw data (for small area) ... Dark frame (same small area) ... Flat frame (same small area) ...
Looking at the values across the whole frame (and in this small area), it is not clear how we could get such divergent values ("SC" means SharpCap frame with Dark subtraction and Flat correction). The Dark values have small values and a small range, so don't impact greatly, and the Flat shows a small range of values.
Raw min/max= 27036 58284
Dark min/max= 264 3282
Flat min/max= 29156 59375
SC min/max= 20280 61931
For just this small view ...
Raw min/max= 29316 32272
Dark min/max= 308 338
Flat min/max= 31473 32901
SC min/max= 22005 55524
Tried an IDL program to perform: new = long((raw - dark) / (flat / mean(flat))) as a naive shot at Dark and Flat correction, and this seemed to work OK (no doubt you do some clever digital filters as well) and removed the gross variation in the Flat and brought R/G/B together. Just continuing looking at Red, I see this, which does not have any divergent pixels. Perhaps not surprizing given the ranges of values shown above.
So, that still leaves me wondering what is happening in SC 4.0.8244.0. Have you seen these telltale "side bands" in the histogram plots before? Is it something I have done wrong somewhere? Any tips?
If it is something awry in SC, let me know what else I might dig out ...
Cheers, Chris.
Been endeavouring to do some good Darks and Flats (w/ Dark Flats) to better work with Live Stack, which is a great feature. First attempt saw the Triffid Nebula "emerge from the mists", which was great fun. But that night PHD2 was dropping the guide camera, the full moon was coming up and I forgot the filter! So this is all about better preparation for when the rain stops and the clouds clear ...
I used an iPad as a flat light source. Started at low gain, but ended up at the common gain=120 at -5 C for the ZWO 294MC Pro I use, to do Darks, Flats and Dark Flats (as part of SC 4.0's Flat capture routine). After the Flat capture the histograms for RGB all collapse onto a single peak.
Now I also select Dark subtraction in Camera Preprocessing and get this second histogram with these "side bands" - a few pixels showing either very low or very high values away from the main peak ...
Capturing a frame, debayering in PIPP and loading this into IDL, and zooming in on a small area in the top-left of frame, I see this in the Red channel, with one high and one low pixel ...
I wanted to understand where these divergent pixel values were coming from, so I looked at the same area in the Red channel of the Raw data, Dark and Flat frames. (Note: this "Raw data" is a new capture, with no Dark or Flat correction.) All maps use full histogram stretch ...
Raw data (for small area) ... Dark frame (same small area) ... Flat frame (same small area) ...
Looking at the values across the whole frame (and in this small area), it is not clear how we could get such divergent values ("SC" means SharpCap frame with Dark subtraction and Flat correction). The Dark values have small values and a small range, so don't impact greatly, and the Flat shows a small range of values.
Raw min/max= 27036 58284
Dark min/max= 264 3282
Flat min/max= 29156 59375
SC min/max= 20280 61931
For just this small view ...
Raw min/max= 29316 32272
Dark min/max= 308 338
Flat min/max= 31473 32901
SC min/max= 22005 55524
Tried an IDL program to perform: new = long((raw - dark) / (flat / mean(flat))) as a naive shot at Dark and Flat correction, and this seemed to work OK (no doubt you do some clever digital filters as well) and removed the gross variation in the Flat and brought R/G/B together. Just continuing looking at Red, I see this, which does not have any divergent pixels. Perhaps not surprizing given the ranges of values shown above.
So, that still leaves me wondering what is happening in SC 4.0.8244.0. Have you seen these telltale "side bands" in the histogram plots before? Is it something I have done wrong somewhere? Any tips?
If it is something awry in SC, let me know what else I might dig out ...
Cheers, Chris.