Looking for Sensor Analysis/Statistics help & ideas

Official SharpCap and Forum announcements
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Looking for Sensor Analysis/Statistics help & ideas

#1

Post by admin »

Anyone good with science/statistics? I'm looking for some thoughts on an anomaly I'm seeing with the sensor analysis of certain cameras....

I thought I understood how astro cameras worked in 8 bit mode and how I could generate a set of sensor analysis figures for 8 bit mode from those measured in a higher bit depth (ie 12 bit). The idea I had went like this...

The most important features of 8 bit mode (compared to 12 bit mode) are

* The ADU value is reduced to the range 0..255 from 0..4095 (by dividing by 16). This means the e/ADU figure is actually 16 times bigger for the same gain
* The read noise in 8 bit mode is dominated by inaccuracies caused by rounding when you divide the ADU by 16 in the first step above - for instance if the light level would give you a 12 bit adu of 3241, you get 202.5625 when dividing by 16, rounded to 203, an error of 0.4375

The second observation about read noise lets you predict the read noise for a camera in 8 bit mode - basically you assume that all the pixels that give you a readout of 203 really should have values ranging between 202.5 and 203.5 and assume an even distribution in that range. You can calculate the standard deviation of a uniform distribution of width 1, and it turns out to be 1/sqrt(12), or about 0.288.

This value of 0.288 shows up in the measurement of many sensors in 8 bit mode - basically it is the ratio between the read noise and the e/ADU gain, for instance an Altair 183C at minimum gain in 8 bit mode has Read Noise=18.13, e/ADU=62.96, which gives a ratio of 18.13/62.96 = 0.288 :)

When I found that the theory I had matched the data I thought it was all sorted, until I started to look at data for certain cameras that didn't fit the pattern - for instance:

ZWO 290M - ratio about 0.488 +/- 0.006 (but Altair 290M I get readings of ~0.29)
Prototype 294C - ratio about 0.480 +/- 0.007 (I have two sets of data for ZWO 294 cameras, one is ~0.288, one is ~0.48)

Where I get two different ratios for a particular sensor, it's the read noise that differs between the two, not the e/ADU value. There are some other ratios that come out of some sensors, but they are rare, whilst the 0.288 and ~0.48 seem more common.

My problem is that I don't understand the 0.48 value - what causes the read noise at 8 bit for those cameras to behave differently (the readings at 12 bit are very similar). Where does this figure of 0.48 come from?

The read noise is measured by taking two short dark frames, subtracting one from the other - which removes any fixed pattern noise - and then calculating the standard deviation of the subtracted data and dividing by sqrt(2) to account for the fact that two frames contribute noise to the subtracted data. This is a standard approach to measuring the read noise - all it requires is for the sensor to actually be in the dark. If the sensor still has some light on it, then it would give a higher read noise, but not a consistent value of 0.48 across many measurements and not just for 8 bit measurements...

Anyway, I'm stumped on this for the moment - which makes it harder for me to ship pre-recorded sensor data with SharpCap as I would need measurements at both 8 bit and 12 (12/14/16) bit depths, rather than just getting the high bit depth measurement and calculating the 8 bit one. Any thoughts on what is going on would be most welcome!

cheers,

Robin
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#2

Post by oopfan »

Hi Robin,

Let's say that throwing away the high 4 bits when switching from 12-bit to 8-bit mode is common practice among all vendors while keeping the amplifier gain the same but perhaps some vendors also modify the amplifier. We assume that gain of 50 means the same from 12-bit to 8-bit but perhaps some vendors raise the amplifier gain in 8-bit mode. I know that ZWO likes to use the term "lucky imaging". Not quite sure what the definition of that is but if I were to hazard a guess it means that I want to run the camera at a very high frame rate with a very high gain and low storage requirements (i.e. 8-bits) for the off-chance that I might capture some sort of fleeting extraordinary event.

Brian
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#3

Post by oopfan »

Robin,

You said that you see the problem with the ZWO 290M. I went to the ZWO site and copied the sensor analysis figures from their manual (see attachment). Although they don't explicitly state it, it looks like these values are for 12-bit mode after I compared them to my Altair 290M.

It would be interesting to see what Sam says if you were to request the sensor analysis for 8-bit mode. I am curious to know if the e-/ADU values are simply different by a factor of 16.

Brian
Attachments
ASI290 sensor analysis from ZWO manual.jpg
ASI290 sensor analysis from ZWO manual.jpg (77.58 KiB) Viewed 6606 times
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#4

Post by admin »

I'm maybe getting carried away - the main reason for having sensor analysis in 8 bit mode is to be able to give unequivocal advice to those doing deep sky to *not* use 8 bit mode... Actually, the other use is for those who are doing lunar/planetary to make sure that they are exposing in a zone where 8 bits is sufficient to capture all the available detail.

Anyway, I pretty much know that the bulk of camera just divide the ADU by 16 when going from 12 bit to 8 bit (64 if going from 14 bit to 8 bit) - when I do measurements of e/ADU at both bit depths the ratio is clear. They don't tinker with the gain - that would be easy to spot as it would break the ratio of 16 between the e/ADU values. Actually, that's a bit of an over-generalization - I had a sample camera of a large pixel sensor that did do some fancy changes to the gain when it went into 8 bit mode, but that's very much the exception not the rule.

I think I may well have a chat with Sam - he might know what is unusual about these sensors and maybe that will be the clue that solves the puzzle.

thanks,

Robin
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#5

Post by oopfan »

Hmm, I must have one of those odd-ball sensors in 8-bit mode. Attached is my sensor analysis for the Altair 290M. The ratio of Read Noise to Gain is fairly constant at ~0.47 for gains in the range of 100 through 2000. Gains above 2000 and the ratio rapidly increases.
Attachments
Altair GP290M Sensor Analysis MONO8 2018-01-09.png
Altair GP290M Sensor Analysis MONO8 2018-01-09.png (89.38 KiB) Viewed 6584 times
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#6

Post by oopfan »

I distinctly remember running the 8-bit test AFTER the 12-bit test. I wonder if I shutdown the camera and removed it from power if I would get a different result. Perhaps I will give that a try tomorrow.
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#7

Post by admin »

That's interesting, I'm fairly sure my 290m was a 0.288 result I will retest this evening to check.

Thanks

Robin
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#8

Post by oopfan »

Hi Robin,

So I've been taking a course with AAVSO on CCD Photometry. I'm learning how to calibrate my equipment. Last night I conducted a linearity test on the Altair 290M (see attachment).

Part of the process is to measure the range of ADUs where the sensor is linear. According to my tests the upper end of the range is at 3300 ADU. This is MONO12 with Gain 100 (LCG mode). I would need to run more tests to see if the linearity changes significantly for other gains but according to the AAVSO docs it should not matter.

It struck me as a possibility that this might affect/explain the anomalies that you are seeing.

Brian
Attachments
Brians Altair 290M Sensor Linearity.xlsx
(22.2 KiB) Downloaded 238 times
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Looking for Sensor Analysis/Statistics help & ideas

#9

Post by admin »

Hi Brian,

that's interesting - good to see that it is pretty much linear up to within about 10% or so of maximum ADU. I don't think this is going to affect the results of the noise analysis because those are mostly performed at very low ADU, so well within the linear section. It might be a nice thing to include in future versions of Sensor Analysis though to automate the linearity measurement as it would be easy to do.

cheers,

Robin
roelb
Posts: 214
Joined: Wed Mar 08, 2017 11:36 pm

Re: Looking for Sensor Analysis/Statistics help & ideas

#10

Post by roelb »

See my sensor analysis results for the ZWO ASI 290MM and ZWO ASI 224MC in annex.
Attachments
ZWO ASI224MC~RAW8.txt
(5.27 KiB) Downloaded 246 times
ZWO ASI290MM~MONO16.txt
(5.14 KiB) Downloaded 245 times
ZWO ASI290MM~MONO8.txt
(5.15 KiB) Downloaded 219 times
Roel
Celestron Nexstar Evolution 8 - Celestron Nexstar 6 SE - StarSense
ZWO ASI533MC-Pro -- ZWO ASI533MM-Pro
Post Reply