Understanding Binning
Forum rules
One tutorial per topic please.
Please only post content that you have created in this form.
Tutorials should be either in PDF format as an attachment or written in the first post of the topic. Links to video tutorials you created are also acceptable.
One tutorial per topic please.
Please only post content that you have created in this form.
Tutorials should be either in PDF format as an attachment or written in the first post of the topic. Links to video tutorials you created are also acceptable.
- admin
- Site Admin
- Posts: 14358
- Joined: Sat Feb 11, 2017 3:52 pm
- Location: Vale of the White Horse, UK
- Contact:
Understanding Binning
There's a lot of misunderstanding about how binning improves images and whether software binning (as used by CMOS cameras) is worth using or not. Sadly the situation isn't really helped by some people who don't really understand the situation claiming that they do...
So, let's talk about binning and how it works, but to do that we need to talk about something else first - Noise...
The noise we are talking about here is random variations in pixel brightness value from frame to frame and pixel to pixel - you can see the effect of noise in the images below :
Low Noise (800ms exposures, minimum gain) High Noise (8ms exposures, maximum gain) (These are images taken of a picture of Jupiter for example purposes!)
The really important measure of how noisy an image is isn't just the size of the noise, it's the size of the noise relative to the brightness of the image itself. This is what we call the 'signal to noise ratio' or SNR for short. To give an example, if an image mostly had a brightness of 200 (out of 255) and the noise meant that the values were spread between 195 and 205 (+/- 5) we might say we have a SNR of 40 (40 = 200 / 5).
So, where does the noise come from in an image - there are three main sources that we need to consider
1) Shot Noise - this type of noise is caused by the number of photons that each pixel detects varying randomly from frame to frame and from pixel to pixel. Imagine that you had a way to count raindrops as they fell into a bucket while it was raining. If you counted the number of raindrops going into the bucket every second the number you get wouldn't be the same for all the seconds you measured even if the rain was steady - in some seconds you might get 20, in others 21, 22, 19, 18, 17, 23, etc. This natural variation is shot noise. Interestingly the amount of noise is equal to the square root of the number of photons detected.
2) Read Noise - this type of noise is caused by the process of converting the charge accumulated on the pixel during the exposure (a count of electrons) to a digital value not being exact. Read noise is a property of the imaging sensor itself, so some sensors have higher read noise values than others. In particular modern CMOS sensors tend to have much lower read noise than CCD sensors.
3) Thermal Noise - thermal noise is caused by the heat in the sensor sometimes being able to release electrons in just the same way that an incoming photon can. The sensor detects these electrons as light, adding noise to the image. Read noise tends to build up during longer exposures as there is more time for the thermal electrons to be released and can be ignored for short exposures. Cooling helps to reduce thermal noise dramatically.
Let's put some concrete number behind these as an example... Take a sensor like the IMX178 and set it at a fairly low (but not lowest) gain setting. At this setting we might have the following values
Gain (e/ADU) : 0.25
Read Noise (e-RMS) : 1.7
Exposure : 0.1s
If we have a pixel value (ADU) of 200, that implies that we have detected 50 electrons (multiply the ADU by the gain).
Detecting 50 electrons gives us a shot noise value of Sqrt(50) or roughly 7 electrons.
The exposure is short, so the thermal noise will be effectively zero.
Read noise is 1.7 Electrons
So, total noise = 7e (shot ) + 0e (thermal) + 1.7e (read)
Interestingly, this does *NOT* add up to 8.7e! When adding noise values together you don't just add the values, you take each value, square it, add the squares and then take the square root of the result [unless the sources of noise are correlated with each other, which they are not in this case]. This is called 'adding in quadrature'.
So...
Sqrt ( 7*7 + 1.7*1.7) = Sqrt( 49 + 3) = Sqrt(52) = (approx) 7.2e
Now all we have to do is scale back to ADU using our gain value and we find a noise of about 29ADU.
Let's note a few interesting things about that noise figure before taking a break...
1) The noise is dominated by the shot noise - the read noise has almost no effect because it is smaller to start with and the adding in quadrature ensures that the largest individual source of noise dominates and smaller ones have relatively even less effect.
2) If we had a bigger read noise (10e for instance) then that would dominate the total noise - our sum would give us a total noise of approximately 12.2e. CCD cameras with read noise of 10e are apparently not uncommon...
So, let's talk about binning and how it works, but to do that we need to talk about something else first - Noise...
The noise we are talking about here is random variations in pixel brightness value from frame to frame and pixel to pixel - you can see the effect of noise in the images below :
Low Noise (800ms exposures, minimum gain) High Noise (8ms exposures, maximum gain) (These are images taken of a picture of Jupiter for example purposes!)
The really important measure of how noisy an image is isn't just the size of the noise, it's the size of the noise relative to the brightness of the image itself. This is what we call the 'signal to noise ratio' or SNR for short. To give an example, if an image mostly had a brightness of 200 (out of 255) and the noise meant that the values were spread between 195 and 205 (+/- 5) we might say we have a SNR of 40 (40 = 200 / 5).
So, where does the noise come from in an image - there are three main sources that we need to consider
1) Shot Noise - this type of noise is caused by the number of photons that each pixel detects varying randomly from frame to frame and from pixel to pixel. Imagine that you had a way to count raindrops as they fell into a bucket while it was raining. If you counted the number of raindrops going into the bucket every second the number you get wouldn't be the same for all the seconds you measured even if the rain was steady - in some seconds you might get 20, in others 21, 22, 19, 18, 17, 23, etc. This natural variation is shot noise. Interestingly the amount of noise is equal to the square root of the number of photons detected.
2) Read Noise - this type of noise is caused by the process of converting the charge accumulated on the pixel during the exposure (a count of electrons) to a digital value not being exact. Read noise is a property of the imaging sensor itself, so some sensors have higher read noise values than others. In particular modern CMOS sensors tend to have much lower read noise than CCD sensors.
3) Thermal Noise - thermal noise is caused by the heat in the sensor sometimes being able to release electrons in just the same way that an incoming photon can. The sensor detects these electrons as light, adding noise to the image. Read noise tends to build up during longer exposures as there is more time for the thermal electrons to be released and can be ignored for short exposures. Cooling helps to reduce thermal noise dramatically.
Let's put some concrete number behind these as an example... Take a sensor like the IMX178 and set it at a fairly low (but not lowest) gain setting. At this setting we might have the following values
Gain (e/ADU) : 0.25
Read Noise (e-RMS) : 1.7
Exposure : 0.1s
If we have a pixel value (ADU) of 200, that implies that we have detected 50 electrons (multiply the ADU by the gain).
Detecting 50 electrons gives us a shot noise value of Sqrt(50) or roughly 7 electrons.
The exposure is short, so the thermal noise will be effectively zero.
Read noise is 1.7 Electrons
So, total noise = 7e (shot ) + 0e (thermal) + 1.7e (read)
Interestingly, this does *NOT* add up to 8.7e! When adding noise values together you don't just add the values, you take each value, square it, add the squares and then take the square root of the result [unless the sources of noise are correlated with each other, which they are not in this case]. This is called 'adding in quadrature'.
So...
Sqrt ( 7*7 + 1.7*1.7) = Sqrt( 49 + 3) = Sqrt(52) = (approx) 7.2e
Now all we have to do is scale back to ADU using our gain value and we find a noise of about 29ADU.
Let's note a few interesting things about that noise figure before taking a break...
1) The noise is dominated by the shot noise - the read noise has almost no effect because it is smaller to start with and the adding in quadrature ensures that the largest individual source of noise dominates and smaller ones have relatively even less effect.
2) If we had a bigger read noise (10e for instance) then that would dominate the total noise - our sum would give us a total noise of approximately 12.2e. CCD cameras with read noise of 10e are apparently not uncommon...
You do not have the required permissions to view the files attached to this post.
- admin
- Site Admin
- Posts: 14358
- Joined: Sat Feb 11, 2017 3:52 pm
- Location: Vale of the White Horse, UK
- Contact:
Re: Understanding Binning
Ok, so what we've arrived at so far is that each pixel has a certain amount of noise and it comes from a mixture of 3 sources. Let's go back to our example with our pixel value of 200 where we calculated a noise of 29ADU. Going back to our concept of signal-to-noise-ratio, we get an SNR of about 6.89 in that case. Remember that SNR of ~6.9 - let's see what happens to it when we start binning...
Imagine that we now apply 2x2 binning, so we will take 4 pixels in a little box, each of which has an ADU value of about 200 and a noise of 29 and we add their values together to make a single binned pixel
Binned value = 200+200+200+200 = 800
But what about the noise on that binned pixel value? Well, remember back to adding noise and recall that we have to add noise 'in quadrature' if the different sources of noise are not correlated - which means squaring the noise, adding the squares, then taking the square root, so:
Binned noise = SQRT( 29*29 + 29*29 + 29*29 + 29*29) = SQRT( 4*841) = 58
Now we notice something really interesting - the pixel value has gone up by a value of 4, but the noise has only gone up by a factor of 2. Let's calculate our SNR for the binned pixel
SNR = Value / Noise = 800 / 58 = approx 13.8
The binning has *doubled* our SNR from ~6.9 to ~13.8. That's about 90% of the secret of binning - it increases the SNR because the pixel values are added normally but the noise is added via the square, add and square root approach.
Before moving on, let's try to understand how that works without the mathematics... When we add the pixel values for each of the four binned pixels, we end up adding the true values for each pixel (the 200 in our example) and the noise. Now, sometimes the noise on all four pixels will be in the same direction (ie all 4 will be brighter than average), but that only happens occasionally - far more often some of the pixels will be brighter than average and some will be dimmer and when you add them up the noise will at least partly cancel out. This is what causes the improvement in SNR when binning.
Now, I did say that this cancelling of noise was 90% of the secret of binning. The other 10% applies to CCD cameras only. CCD cameras can add up the charge values for binned pixels before reading the charge and turning it into an ADU (digital) value. This helps because you only get one lot of read noise when doing this rather than a separate read noise contribution from each of the binned pixels. That sounds like a good thing, but think back to our example for a CMOS sensor where the read noise of 1.7e only increased the total noise from 7e to 7.2e - in other words for CMOS sensor the read noise is so low that it often has little effect. Of course for CCD sensors with their higher read noise this is much more important.
Just for completeness, in the next post I'll work through a CCD example to show the increase in SNR.
Imagine that we now apply 2x2 binning, so we will take 4 pixels in a little box, each of which has an ADU value of about 200 and a noise of 29 and we add their values together to make a single binned pixel
Binned value = 200+200+200+200 = 800
But what about the noise on that binned pixel value? Well, remember back to adding noise and recall that we have to add noise 'in quadrature' if the different sources of noise are not correlated - which means squaring the noise, adding the squares, then taking the square root, so:
Binned noise = SQRT( 29*29 + 29*29 + 29*29 + 29*29) = SQRT( 4*841) = 58
Now we notice something really interesting - the pixel value has gone up by a value of 4, but the noise has only gone up by a factor of 2. Let's calculate our SNR for the binned pixel
SNR = Value / Noise = 800 / 58 = approx 13.8
The binning has *doubled* our SNR from ~6.9 to ~13.8. That's about 90% of the secret of binning - it increases the SNR because the pixel values are added normally but the noise is added via the square, add and square root approach.
Before moving on, let's try to understand how that works without the mathematics... When we add the pixel values for each of the four binned pixels, we end up adding the true values for each pixel (the 200 in our example) and the noise. Now, sometimes the noise on all four pixels will be in the same direction (ie all 4 will be brighter than average), but that only happens occasionally - far more often some of the pixels will be brighter than average and some will be dimmer and when you add them up the noise will at least partly cancel out. This is what causes the improvement in SNR when binning.
Now, I did say that this cancelling of noise was 90% of the secret of binning. The other 10% applies to CCD cameras only. CCD cameras can add up the charge values for binned pixels before reading the charge and turning it into an ADU (digital) value. This helps because you only get one lot of read noise when doing this rather than a separate read noise contribution from each of the binned pixels. That sounds like a good thing, but think back to our example for a CMOS sensor where the read noise of 1.7e only increased the total noise from 7e to 7.2e - in other words for CMOS sensor the read noise is so low that it often has little effect. Of course for CCD sensors with their higher read noise this is much more important.
Just for completeness, in the next post I'll work through a CCD example to show the increase in SNR.
- admin
- Site Admin
- Posts: 14358
- Joined: Sat Feb 11, 2017 3:52 pm
- Location: Vale of the White Horse, UK
- Contact:
Re: Understanding Binning
So, let's work through a CCD camera example with a read noise of 10e but otherwise unchanged.
With no binning, our total noise is SQRT(7*7+10*10) = SQRT(150) = 12.2e or ~49 ADU, giving an SNR of about 4.
If we perform 2x2 binning the CCD way of adding the charges then reading out, we find that our 2x2 binned pixel has 200e now so the shot noise is SQRT(200) or about 14.1e. When we combine this in quadrature with our 10e read noise we get a total noise of 17.3e, which gives us an SNR of about 11.5 (=200/17.3) in the binned case.
So, in this example CCD style binning gives us an even bigger improvement in SNR of nearly a factor of 3, but look at the SNR figures again...
CMOS No binning: 6.9
CMOS 2x2 binning: 13.8
CCD No binning: 4
CCD 2x2 binning: 11.5
The CCD shows a bigger improvement with binning, but that's largely because it started out so far behind due to the higher read noise. With binning that read noise becomes less of a problem, but even at 2x2 binning it's still causing a lower SNR than the CMOS sensor with lower read noise.
Ok, just for fun, let's try the figures for 3x3 binning to see how they work out for both cases.
CMOS is easy - the SNR for 3x3 is just going to be 3x the SNR for no binning or 20.7.
For CCD we need to add 9 pixels of 50 electrons each to get 450 electrons, then take the square root of that to get a shot noise of 21.2 electrons, then add that in quadrature with the read noise of 10e to get a total noise of 23.5 electrons (out of a signal of 450), giving an SNR of 19.1.
Closer again, but the CCD is still behind due to the higher read noise.
It's worth noting that I've assumed for these calculations that the only differences between CCD and CMOS are the read noise and whether they can do charge level binning or not. That's not really true, but it's a nice simple assumption to allow us to do some calculations. In real life there are other differences between sensors like different quantum efficiencies (QE, the fraction of incoming photons that get detected) and different gains, making this sort of calculation more complex.
With no binning, our total noise is SQRT(7*7+10*10) = SQRT(150) = 12.2e or ~49 ADU, giving an SNR of about 4.
If we perform 2x2 binning the CCD way of adding the charges then reading out, we find that our 2x2 binned pixel has 200e now so the shot noise is SQRT(200) or about 14.1e. When we combine this in quadrature with our 10e read noise we get a total noise of 17.3e, which gives us an SNR of about 11.5 (=200/17.3) in the binned case.
So, in this example CCD style binning gives us an even bigger improvement in SNR of nearly a factor of 3, but look at the SNR figures again...
CMOS No binning: 6.9
CMOS 2x2 binning: 13.8
CCD No binning: 4
CCD 2x2 binning: 11.5
The CCD shows a bigger improvement with binning, but that's largely because it started out so far behind due to the higher read noise. With binning that read noise becomes less of a problem, but even at 2x2 binning it's still causing a lower SNR than the CMOS sensor with lower read noise.
Ok, just for fun, let's try the figures for 3x3 binning to see how they work out for both cases.
CMOS is easy - the SNR for 3x3 is just going to be 3x the SNR for no binning or 20.7.
For CCD we need to add 9 pixels of 50 electrons each to get 450 electrons, then take the square root of that to get a shot noise of 21.2 electrons, then add that in quadrature with the read noise of 10e to get a total noise of 23.5 electrons (out of a signal of 450), giving an SNR of 19.1.
Closer again, but the CCD is still behind due to the higher read noise.
It's worth noting that I've assumed for these calculations that the only differences between CCD and CMOS are the read noise and whether they can do charge level binning or not. That's not really true, but it's a nice simple assumption to allow us to do some calculations. In real life there are other differences between sensors like different quantum efficiencies (QE, the fraction of incoming photons that get detected) and different gains, making this sort of calculation more complex.
- admin
- Site Admin
- Posts: 14358
- Joined: Sat Feb 11, 2017 3:52 pm
- Location: Vale of the White Horse, UK
- Contact:
Re: Understanding Binning
Binning : Adding or Averaging
In the second post where we did our first binning calculation, we added the four pixel values to get a new binned value of 800. This is quite a common binning technique and it has the desirable property of making the image immediately four times brighter to show faint detail. One of the problems with this technique though is if our maximum pixel value is still 255 (8 bit imaging). We won't get a value of 800, we'll get a value of 255, in fact we'll get a value of 255 for any areas where the original unbinned pixel values were 64 or more...
Other cameras perform their software binning by *averaging* the values of the four pixels instead of adding them. Let's see how this affects SNR...
We'll do our averaging in two stages, add the values first then divide by the number of items we've added together.
The adding step proceeds exactly as we've already calculated above
Sum = 200+200+200+200 = 800
Noise on Sum = SQRT( 29*29 + 29*29 + 29*29 + 29*29) = SQRT( 4*841) = 58
Now we have to divide by 4...
Average = 800/4 = 200
Noise on Average = 58/4 = 14.5
I'll just justify the noise being divided by four - if the noise on the total is 58 then the typical range for the total including noise would be 742 to 858. When we divide those by four we see that the typical range of the average including noise would be 185.5 to 214.5 - ie the noise definitely needs to be divided by four when we divide the value by four.
SNR for Average = 13.8
So, we see that it doesn't matter whether we use adding when binning or averaging - in both cases the improvement in image quality in terms of SNR is just the same. Of course using averaging means that the image doesn't immediately get brighter in the way it does when using adding, but the image noise does drop by a factor of two instead to maintain the same SNR gain.
Averaged binning doesn't have the same problem with clipping at the maximum pixel value (255) that plagues additive binning, and if you do want a brighter image then you always have the option of turning up the gain! (Changing the gain while not changing any other parameters like exposure does not significantly affect the SNR - at least it only affects it via a change to the read noise with the change in gain value and only then if the read noise is significantly more than the shot noise).
So, in summary we've seen that
1) binning increases the signal-to-noise-ratio (SNR) of an image
2) CMOS style software binning increases the SNR in proportion to the N (of NxN binning), so 2x2 doubles the SNR, 3x3 triples it, etc
3) CCD style hardware binning gives an even bigger improvement, but only because the unbinned case has such a low SNR due to the high read noise
4) Adding and Averaging software binning both give the same SNR improvement
As a final note, if you want to work through the calculations you'd find that if you had a low read noise sensor (say 1.7e) that could do hardware binning like a CCD then the SNR gain for hardware binning would only be fractionally better than for software binning. That's because the if the low read noise is being swamped by the higher shot noise then it doesn't matter much whether you have 4 lots of read noise or only 1!
In the second post where we did our first binning calculation, we added the four pixel values to get a new binned value of 800. This is quite a common binning technique and it has the desirable property of making the image immediately four times brighter to show faint detail. One of the problems with this technique though is if our maximum pixel value is still 255 (8 bit imaging). We won't get a value of 800, we'll get a value of 255, in fact we'll get a value of 255 for any areas where the original unbinned pixel values were 64 or more...
Other cameras perform their software binning by *averaging* the values of the four pixels instead of adding them. Let's see how this affects SNR...
We'll do our averaging in two stages, add the values first then divide by the number of items we've added together.
The adding step proceeds exactly as we've already calculated above
Sum = 200+200+200+200 = 800
Noise on Sum = SQRT( 29*29 + 29*29 + 29*29 + 29*29) = SQRT( 4*841) = 58
Now we have to divide by 4...
Average = 800/4 = 200
Noise on Average = 58/4 = 14.5
I'll just justify the noise being divided by four - if the noise on the total is 58 then the typical range for the total including noise would be 742 to 858. When we divide those by four we see that the typical range of the average including noise would be 185.5 to 214.5 - ie the noise definitely needs to be divided by four when we divide the value by four.
SNR for Average = 13.8
So, we see that it doesn't matter whether we use adding when binning or averaging - in both cases the improvement in image quality in terms of SNR is just the same. Of course using averaging means that the image doesn't immediately get brighter in the way it does when using adding, but the image noise does drop by a factor of two instead to maintain the same SNR gain.
Averaged binning doesn't have the same problem with clipping at the maximum pixel value (255) that plagues additive binning, and if you do want a brighter image then you always have the option of turning up the gain! (Changing the gain while not changing any other parameters like exposure does not significantly affect the SNR - at least it only affects it via a change to the read noise with the change in gain value and only then if the read noise is significantly more than the shot noise).
So, in summary we've seen that
1) binning increases the signal-to-noise-ratio (SNR) of an image
2) CMOS style software binning increases the SNR in proportion to the N (of NxN binning), so 2x2 doubles the SNR, 3x3 triples it, etc
3) CCD style hardware binning gives an even bigger improvement, but only because the unbinned case has such a low SNR due to the high read noise
4) Adding and Averaging software binning both give the same SNR improvement
As a final note, if you want to work through the calculations you'd find that if you had a low read noise sensor (say 1.7e) that could do hardware binning like a CCD then the SNR gain for hardware binning would only be fractionally better than for software binning. That's because the if the low read noise is being swamped by the higher shot noise then it doesn't matter much whether you have 4 lots of read noise or only 1!
Re: Understanding Binning
Thanks posting this excellent tutorial!
Roel
Celestron Nexstar Evolution 8 - Celestron Nexstar 6 SE - StarSense
ZWO ASI533MC-Pro -- ZWO ASI533MM-Pro
Celestron Nexstar Evolution 8 - Celestron Nexstar 6 SE - StarSense
ZWO ASI533MC-Pro -- ZWO ASI533MM-Pro
Re: Understanding Binning
Yes, this is an excellent account of noise and binning for exposures of 0.1s. I would like to be able to repeat the comparison between CCD and CMOS for slightly longer exposures, say 1s. Unfortunately I can't find numbers for CMOS dark current on either the Sony or FLIR websites, despite dark current specs being mandatory for EMVA compliance. https://www.emva.org/standards-technology/emva-1288/ It is certainly not zero, but would not materially change these conclusions for 0.1s exposures.
Re: Understanding Binning
This topic might help:
viewtopic.php?f=29&t=940
It covers the analysis of dark current of a ZWO ASI1600MM that Minos and I conducted last month. Results on the last post.
Brian
viewtopic.php?f=29&t=940
It covers the analysis of dark current of a ZWO ASI1600MM that Minos and I conducted last month. Results on the last post.
Brian
Re: Understanding Binning
Thanks Brian. I will try to measure the dark current in my IMX249 FLIR camera using Robin's method. It doesn't have cooling and runs at 45C with passive heat sink and fins (58C unmounted). I asked FLIR why they didn't publish dark current and was told that they were working on EMVA compliance. But I can guess why. Alan
Re: Understanding Binning
Alan,
I see that the IMX249 has a deep well of 33ke-. I like that but unfortunately it is incompatible with my short refractor due to the large pixel size of 5.86um.
Specs that I found here https://www.ptgrey.com/blackfly-23-mp-m ... ius-imx249 say that the Operating Temperature is 0 to 45C. Why is your camera running so hot?
Brian
I see that the IMX249 has a deep well of 33ke-. I like that but unfortunately it is incompatible with my short refractor due to the large pixel size of 5.86um.
Specs that I found here https://www.ptgrey.com/blackfly-23-mp-m ... ius-imx249 say that the Operating Temperature is 0 to 45C. Why is your camera running so hot?
Brian