Exposure's Effect On Color Quality

Somewhere to share your expertise in using SharpCap
User avatar
oopfan
Posts: 996
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Exposure's Effect On Color Quality

Post by oopfan »

This Galaxy Season I've been on a quest to spend only one session per galaxy. In the past, most notably M81 last year, I devoted four sessions and 11.6 hours. No more.

I restrict myself to exposure times of 90 seconds or less due to having no ability to auto-guide with my 50-year-old GEM. And because of that, I limit myself to 60-second exposures which guarantee good tracking.

Two nights ago, I first attempted M51 using 60-second exposures. The results were underwhelming (see left-side image). I mean, the detail is OK, and the stars are (nearly) round, but it is essentially monochromatic with only a hint of blue.

I've seen this before, having to deal with a "muddy brown" image. My gut reaction in the past was to capture more data to lower the noise content, but that never solved the problem. It only led to a noise-free, muddy brown image. So now, I was back to square one. How to solve this problem?

At that very moment, while I was grasping for hints, a friend contacted me asking for help with some processing. He had acquired an image of M51 with a 10-inch Dall-Kirkham. The detail was amazing.

I decided to analyze his exposure settings. I was blown away by the signal-to-noise ratio: Total SNR 52, so it was essentially noise-free. But what impressed me more was the signal-to-noise ratio of each sub: SNR 6.33 in the red channel. I compared that to my SNR in the red channel: SNR 0.23. That's what I get for using a mere 60-second exposure! According to my analysis, a 300-second exposure in the red channel would get me SNR 1.12.

So the following night, it was clear again here. I don't have auto-guiding so I decided to manually guide out the imperfections in the PEC. It wasn't as hard as I thought, but it was tiresome over 2.5 hours.

The right-hand image uses 300-second exposures in the color channels. I was careful to adjust the frame counts so that the total SNR of the image was the same as the first image. Now I could see, in perfect isolation, what happens when I increase exposure.

I forwarded the image to a friend. He replied, "There is better definition in the latest image." That is true, but the perception of greater detail comes from better color, nothing else. I didn't use any sharpening in post-processing.

Of course, the image is not perfect. The red SNR per sub is only 1.12. I wonder how much better it would look if I increased it to SNR 2? Now, we are pushing the limits of my ability to manually guide. A case can be made for switching from Bin 1 to Bin 2 at a lower exposure. But then I give up the spatial resolution of Bin 1. Perhaps that isn't so bad since my guiding is less than optimal anyhow.

I know that a lot of SharpCap members are passionate about short exposures. I am not trying to persuade to change. I am just saying that if your colors are close to monochrome without a lot of punch or pizazz, then consider experimenting with longer exposures.

Brian
Attachments
M51 300s vs 60s.jpg
M51 300s vs 60s.jpg (215.21 KiB) Viewed 610 times
timh
Posts: 207
Joined: Mon Aug 26, 2019 5:50 pm

Re: Exposure's Effect On Color Quality

Post by timh »

Hi Brian. Your observation chimes closely with my own recent experience.

I own a 10 inch dobsonian that served me well over the years for visual work. However - when first starting EEA last year I got a bit obsessed with the goal of seeing how I could extend it to astrophotography and delved into all sorts of things- field rotation etc- but basically although I could sometimes get up to quite long total exposure stacks they inevitably always comprised lots of short frames ~typically 3-8 s at F 4.2. The upshot was - exactly as you say - pictures of the brighter nebulae with sometimes quite good resolution but lacking in both faint detail and colour.

So I bit the bullet and got an EQ mount plus a similarly fast newtonian (F 4.5-5) to go with it. Far more colour and faint detail with the longer exposures now possible - even at quite similar total exposure times as with the Dob. Of course the faint detail is ultimately anyway limited by the background light levels. For emission nebulae the best solution to that problem for me has been to also image in HA and then mix that into the luminance of the RGB image. But of course that doesn't work for galaxies which is what we have on the menu at the moment.

TimH
timh
Posts: 207
Joined: Mon Aug 26, 2019 5:50 pm

Re: Exposure's Effect On Color Quality

Post by timh »

Just to add some further data to your point - 'Nullius in verba' as the Royal Society would say.

The left hand image of IC805 and the running dog nebula is a stack of 70 min of 5s exposures at gain 200 taken at F 4.2 using a 10 inch Dobsonian. The right hand image is a 69 min total exposure time at F4.5 using an 8 inch Newtonian comprising mixed 40 and 70s exposures at gain 124 (~ unity)

While all of these exposures are shorter than optimal - and particularly given they were all taken with a UHC filter in place - the longer exposure detects a lot more colour and nebula even notwithstanding the slightly lower gain.

The Dobsonian exposures are obviously way shorter then optimum out of necessity but I still think that it is quite a nice sharp image of the star cluster itself.

I wonder whether colour might be a particular problem for OSC cameras at low exposure times given their lower QE and that only 25% of the pixels directly detect R and B? Suburban skies also have a lot of background colour as well as luminance which probably doesn't help when each short frame has low SNR....plus all the added read noise.
TimH
Attachments
IC405exp_compare.JPG
IC405exp_compare.JPG (168.39 KiB) Viewed 568 times
User avatar
oopfan
Posts: 996
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Exposure's Effect On Color Quality

Post by oopfan »

Hi Tim,

I've been trying to wrap my head around this phenomenon. There are two sets of equations acting on the image. One set for the sub, and one for the stack. The stack is easy: the square root of the frame count. However, the sub is more involved. Each pixel experiences the signal differently. Those that receive the light from a bright part of the nebula benefit from a flood of electrons, quickly overwhelming the sources of noise. However, the pixels in a faint part of the nebula will struggle with a weak signal, but will succumb to the noise if the exposure is cut short. It would be a different story if there was no noise. Read Noise is just one component. There is also Dark Noise and Sky Noise. If we could eliminate those sources of noise (good luck!) then we could get closer to a linear relationship (we still have Shot Noise to deal with.) All of those noise sources conspire to harm weak signals. The best way I know of to combat this is to lengthen the exposure, therefore giving the weak signal more time to build until the signal-to-noise ratio is greater than one, the higher the better. So, I believe that sub exposure makes or breaks an image. I am not saying that every DSO demands long subs. But it does depend on what you are after. If you want to see nebula at the limits of detection then you need a long exposure to fight against all of that noise. There is no amount of stacking that will deliver what you want. In other words, your choice of sub-exposure determines your success or failure.

Brian
timh
Posts: 207
Joined: Mon Aug 26, 2019 5:50 pm

Re: Exposure's Effect On Color Quality

Post by timh »

It is a very interesting area of discussion Brian. I think that I agree

Argument by Reducto ad absurdum says that there has to be a minimum exposure time below which the signal at a point within any given frame is almost always completely lost simply due to quantization --so never makes to the ADU - and thus neither can any signal emerge from stacking however many frames.

The same logic says that this minimum usable exposure time must be longer for very faint parts of an image than for brighter parts.

There is an interesting polemic/ paper on this topic here (and doubtless elsewhere) https://www.cloudynights.com/articles/c ... sure-r1571 that you may have seen?

I only wish I fully understood it :-) ...Maybe one day after reading it through slowly . Basically the author takes the same approach as in Robin's Brain algorithm calculating a mimimum exposure length based on light pollution and restricting read noise to (say) 5% of the total (EQ 9) BUT at the same time combines and contrasts this with a target-based approach (EQ 14) which says that individual exposures must also exceed a certain level (say 15 ADU) to achieve detectability of certain features.

Anyway it seems to me to be probably about right balance of approach- and also fits with my own practical experience thus far. Stacking really helps but for a given target you have to start with a frame length long enough that the detail you want to collect is at a level that the ADU can 'see'.

TimH

(below is an extract --- it is not an easy read but it gives an idea)

"At this point you may wondering what exactly is the difference between the light pollution limited exposure given in Equation 9 and the optimal sub-frame exposure based on the target and light pollution combination in Equation 14 so let’s take a detailed look. Equation 9 sets the exposure length and then reduces noise by the square root of the number of frames stacked. If you want to double the SNR, stack 4 times as many frames. Low-level signals will be seen after the noise is driven below them. Eventually the noise will be driven down to 2 RMS or even <1 RMS. Equation 14 takes a completely different approach. It assumes that signals below ?(say ?=15) ADU will not be easily discriminated and computes the exposure length and number of frames to stack for TTotal so the final photo will have exactly ?/2 RMS error, no more and no less. If you want to double the SNR, you must shoot twice as long (doubles signal and increases noise by SQRT2 ) for twice as many frames (decreases noise by SQRT 2). The problem with the first approach is that low-level signals do not increase their SNR by SQRT (number of frames) but something less than that. Also the law of diminishing returns kicks in when driving the noise error so low. Once the error term gets down to around 2, quantization error reduces the effectiveness of stacking. By keeping the final noise at ?/2, the optimal sub-frame exposure is always in the range where noise is reduced by the SQRT (number of frames) .

Examine how the two approaches make you think about the problem of noise. Limiting exposure because of light pollution makes you think that 1) you can simply stack your way to any level of detail or 2) light pollution greatly limits the detail. Both are false. The optimal exposure makes you think in terms of length of exposure required to obtain the desired level of detail in the target and that light pollution determines how many frames to stack to drive the noise error down to ?/2. Light pollution becomes the enemy of total time and could even prevent the capture due to light pollution saturation. Look at Table 3 for a 10 minute exposure. A location that has strong light pollution of ETotal=17 requires about 3 times as many frames to stack to drive the noise to ?/2 and generate the same end result as it does at a dark site where ETotal=10. Over 8.5 hours versus 3 hours because 10 minute exposures at ETotal=17 have a noise of ~54 RMS and at ETotal=10 they have a noise of ~31.8 RMS. Image details are not limited by noise; they are limited by the length of exposure. You can always stack whatever number of frames necessary to drive the error to ?/2 but stacking a vast number of frames will not allow you to see signals you never sufficiently captured."
User avatar
oopfan
Posts: 996
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Exposure's Effect On Color Quality

Post by oopfan »

Hi Tim,

No I have not seen that article, but I'll certainly give it a thorough read. If my initial reading of it is correct, it is saying that there exists a maximum sub-exposure that meets our requirements for detail in the faintest regions of a nebula. If that sub-exposure happens to be 2 minutes, then we switch over to stacking for the remainder of our total exposure budget. For example, if we are wiling to invest a total of 2 hours, then it is optimal to stack 60x120s subs.

In my initial post, I said this:
Of course, the image is not perfect. The red SNR per sub is only 1.12. I wonder how much better it would look if I increased it to SNR 2?
Now I see that I failed to consider an upper limit on detail. Honestly, I've never owned a scope that collected too many photons!

Thanks,
Brian
timh
Posts: 207
Joined: Mon Aug 26, 2019 5:50 pm

Re: Exposure's Effect On Color Quality

Post by timh »

Hi Brian,

When you read it through I think that you will see that it is proposing a physical and math explanation for your observation that stacking more short exposures does not always provide the equivalent result to stacking fewer long ones for the same total length of time.

i.e. it is not proposing any maximum exposure length but only that you need to be above a minimum exposure length that provides above a certain ADU signal -- which is exactly in accord with your intuitive SNR argument (i.e an individual frame needs to be above a certain SNR) .


The core of the argument seems quite simple. The author says that below a certain critical level signal increasingly gets lost in quantization and the probability of detection is no longer normally distributed. This matters because the concept of stacking frames / exposure doubling to reduce noise and increase SNR is based on the assumption that signals are normally distributed and the error therefore being ~ SQRT signal.

The author then somewhat arbitrarily sets the threshold of this critical signal level at an ADU of 15. Above this level he deems that increased stacking works fine to improve SNR --- but if you go below this level signal will be lost - therefore this criterion should set your minimum exposure level.

I am still struggling though to see how I can make his equations translate to my background light levels, specific camera and specific targets.



TimH
User avatar
oopfan
Posts: 996
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Exposure's Effect On Color Quality

Post by oopfan »

Hi Tim,

Very interesting indeed. These past couple years I've been improving upon a calculator for determining LRGB exposures. My first version required that I take a test exposure with a luminance filter to discover the exposure that lifts the galaxy out of the noise. I admit, it is a totally subjective, but it works.

M81 was my first test case from last year. I determined that a 90s exposure in luminance at Bin 1 was adequate. Then I plugged 90s into my calculator, and it told me what the RGB exposures needed to be. I don't auto-guide so my choice was Bin 2 for the color filters. This is what I got after 11.6 hours of integration:
M81_L250x90s_R90x85.3s_G90x50.2s_B90x77.8s_Rx0.83_Gx0.90_Bx1.00_S15-3-25_SA25-25_G92_SGB2_SH70_2020-02-24_AFF.jpg
M81_L250x90s_R90x85.3s_G90x50.2s_B90x77.8s_Rx0.83_Gx0.90_Bx1.00_S15-3-25_SA25-25_G92_SGB2_SH70_2020-02-24_AFF.jpg (752.25 KiB) Viewed 512 times
Not bad at all except for those fat blue stars. (That's a different topic altogether.)

Just this morning, I decided to plug those exposures into my next generation calculator. This is what I got:
M81 LRGB exposure analysis.jpg
M81 LRGB exposure analysis.jpg (61.11 KiB) Viewed 512 times
Wow, look at that. The "SNR per sub", in the red box, is a very respectable SNR 1.28, and the "Signal per sub" is 13.4 electrons. When I convert that to ADU counts, I get 64 levels. That is 6 bits of dynamic range. Old school CRT monitors give you 8 bits, so there is room for improvement there.

Then I decided to reconfigure my exposure settings for M51 for the next opportunity to image it. I decided to target "SNR per sub" of 1.5. Doing that yielded about 6.5 bits of dynamic range. Here are the exposure settings I will use:

W12: 30x200s bin 1
R: 15x120s bin 2
G: 15x70s bin 2

Those frame counts fit within one session, approximately 2.5 hours. The Total SNR is 12. That is only half of what my M81 is. So if I want to get similar quality, I would need 4 sessions total. I'm not going to do that. My objective is "one galaxy/one session" and let the chips fall where they may.

The good thing about this new generation calculator is that it allows me to plan my imaging sessions without requiring a test exposure. Also, it alerts me to galaxies that are beyond my ability. For example, I've decided that I will not accept anything below 1.0 for "SNR per sub".

Brian
timh
Posts: 207
Joined: Mon Aug 26, 2019 5:50 pm

Re: Exposure's Effect On Color Quality

Post by timh »

Hi Brian,

I do like the idea of a semi-empirical model like that. It is a very practical approach. Must by now have loads of frames on loads of objects and I might just start digging into them to see what sort of SNR I have been getting on what --and the all important qualitative judgement "Is it showing me what I want to see?" ...and then have a go at calibrating and setting up a model of my own in Excel. It is a good discipline anyway because it forces one to better think through and understand what you are trying to achieve.

I think that I can probably also directly translate some things from your data? You are at about F 5.9 so at F4.5 similarly effective exposures are likely to be around 2/3 as long.

Nice pictures of M81 by the way ...but now I also understand why and how you got into the stuff around filtering out the blue

Anyway a project to have a go at - much more interesting (but according to some - less useful) than decorating the house.
thanks
Tim
User avatar
oopfan
Posts: 996
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Exposure's Effect On Color Quality

Post by oopfan »

Tim,

I can give you access to the website, if you are interested. I have to be careful not to allow too many users. I am on a free tier at the cloud database provider. The company used to have a very generous free tier of 100,000 read operations per day, and then they suddenly changed the pricing structure to 100,000 read operations per month with a hard limit, meaning that if I hit the limit 15 days into the month, then database operations fail until the 1st of the next month. I'm retired, so shelling out $25 per month is something I am trying to avoid. The best solution would be to try to monetize it.

The calculator needs quite a lot of information: your telescope (aperture, focal length, etc.); your camera (pixel size, read noise, etc.); quantum efficiency in the red, green, and blue bands; binning; and the sky brightness at your site. It also needs to know the surface brightness of the DSO. I have a long list of targets, but you can create your own. You can create "projects" and save them to folders for easy recall and modification.

PM me if you are interested. BTW, this is for the next-gen calculator where you don't need to take a test exposure.

Brian
Post Reply