Touptek G3M662M and ATR585M settings in newer Sharpcap versions

[RadHard]

Hello Robin,

thank you very much, the 662 will be mostly used for guiding.
I also had time to run the sensor analysis today. It looks like the lowest read noise is about 0.8. I ran the measurement twice with different ROI, but results were pretty much the same. Is the read noise calculated only from the dark frames during the analysis?

According to the smart histogram, the optimal settings are then 1365gain 396offset, and at least 1-2mins of exposure with the narrowband filter it seems.

ATR585M in 14 bit
Gain Value, Relative Gain, e/ADU, Read Noise, FWD (e)
100, 1, 2,3741271, 6,451541, 38897,69921875
177, 1,7501076, 1,3565606, 5,152723, 22225,888671875
316, 3,1413434, 0,7557681, 4,508312, 12382,5048828125
562, 5,519291, 0,43015075, 3,960042, 7047,58984375
1000, 9,818595, 0,24179906, 3,6687279, 3961,6357421875
1364, 13,426006, 0,17683049, 3,5009851, 2897,190673828125
1365, 13,993967, 0,16965361, 1,0344821, 2779,604736328125
1778, 18,395605, 0,12905946, 0,99288446, 2114,51025390625
3162, 32,208733, 0,073710665, 0,9318156, 1207,675537109375
5623, 57,37615, 0,041378293, 0,8909547, 677,9419555664062
10000, 102,178986, 0,023234984, 0,84644604, 380,6819763183594
17782, 178,31943, 0,0133139, 0,815299, 218,1349334716797
31621, 314,61258, 0,0075461925, 0,8190781, 123,6368179321289

I will experiment with the HDR mode tonight, is the optimal gain setting then also at 1365?

[admin]

Hi,

modern cameras are remarkably consistent across the frame and from sample to sample of the same sensor, which is why doing your own analysis is fun but not really necessary if the analysis data is already shipped with SharpCap for your camera.

The read noise is indeed estimated from dark frame noise (there's another approach involving regression of the noise level vs brightness, but I found that less reliable in testing).

On HDR mode, you can run your own analysis in HDR and see what that recommends - the problem is that how the HDR works isn't really documented, so it's difficult to be sure what to recommend...

cheers,

Robin

[RadHard]

Hello Robin,

i have been testing the HDR mode on the ATR585M yesterday at gain 1600 and 3200 as set in Sharpcap and 2 min exposures. I would say the results are OK, M101 i was capturing is not really target that would require high dynamic range. Quality of the data is i would say on par with what i got in the HCG mode, however comparison at no moon light will be more fair.

One observation was that sensor analysis today took very long time during the offset measurement. I actually thought it was stuck there as it was repeated maybe 5-10times. I have read that in the HDR mode, the offset setting is overridden by value of 512 by the camera. Also i just tested it now manually varying offset, there is no change in HDR more, in normal mode, histogram shifts to the right. I have then re-run the analysis in Normal mode and it actually takes comparable amount of time, so it is maybe just subjective observation.

You mentioned the sensor data for many cameras are already provided in SharpCap, where can I see those?
Are the sensor data used for anything else apart from the Smart Histogram function?

I wonder about the read noise, it looks like its not possible to get to the manufacturer advertised values.

Here are the measurement results in the HDR mode.
ATR585M in 16 bit
Gain Value, Relative Gain, e/ADU, Read Noise, FWD (e)
100, 1, 0,37966737, 24,591654, 24881,880859375
177, 1,7304772, 0,21940038, 1,532483, 14378,623046875
316, 3,1221159, 0,12160579, 1,1723487, 7969,55712890625
562, 5,4232183, 0,07000776, 1,153904, 4588,0283203125
1000, 9,71311, 0,039088134, 1,113719, 2561,679931640625
1364, 13,258798, 0,028635127, 1,102832, 1876,6317138671875
1365, 13,248677, 0,028657, 1,1106399, 1878,065185546875
1778, 17,32162, 0,021918697, 1,113192, 1436,4637451171875
3162, 29,901495, 0,01269727, 1,1172009, 832,1282958984375
5623, 53,366524, 0,007114336, 1,1079706, 466,2451171875
10000, 95,80387, 0,0039629648, 1,052452, 259,71685791015625
17782, 142,84407, 0,0026579148, 1,0379236, 174,18910217285156
31622, 142,72842, 0,0026600685, 1,0199647, 174,3302459716797
56234, 142,64293, 0,0026616626, 1,0298159, 174,43472290039062
100000, 142,49184, 0,002664485, 1,0458921, 174,61968994140625
136500, 142,46042, 0,0026650724, 1,04531, 174,65818786621094

Offset is same across all settings:
"OffsetInfo": [
{
"OffsetValue": 0,
"MeanADU": 502.40836,
"MedianADU": 516.4,
"SigmaADU": 122.33032
},

For the HDR mode, which value of gain setting is actually sent to the camera, is it slider value/9.1 or directly the slider value, or past 1365, slider value divided by 9.1? When look at the results, the Relative gain value is clipped past 17782 (probably past 15000)

Here is the result from Normal mode:
ATR585M in 13 bit <-- interesting, in the measurement from two days ago, 14bits were reported
Gain Value, Relative Gain, e/ADU, Read Noise, FWD (e)
100, 1, 4,7813683, 6,45327, 39168,96875
177, 1,7501228, 2,7320187, 5,1962967, 22380,697265625
316, 3,136384, 1,5244843, 4,532057, 12488,5751953125
562, 5,502623, 0,8689252, 4,0216007, 7118,2353515625
1000, 9,783454, 0,48871985, 3,6990643, 4003,593017578125
1364, 13,353528, 0,35806027, 3,5796862, 2933,229736328125
1365, 13,916293, 0,34358057, 1,0487334, 2814,612060546875
1778, 18,267437, 0,26174268, 1,0113891, 2144,196044921875
3162, 32,00328, 0,14940242, 0,94309515, 1223,9046630859375
5623, 56,84812, 0,08410776, 0,91863275, 689,0107421875
10000, 101,130936, 0,047278985, 0,85980153, 387,3094482421875
17782, 176,20375, 0,02713545, 0,83292246, 222,29360961914062
31621, 309,0935, 0,015469002, 0,83325255, 126,72206115722656

Thanks

Peter

[admin]

Hi Peter,

The black level measurements can take a while, as SharpCap needs to run through each gain of interest and then at each gain needs to measure a range (5 I think) of black level values. As you noticed, for a lot of cameras, the black level results are the same at all gains, but sadly there are some models from some manufacturers where the values depend on the gain, so the extended process is necessary. Having a fast frame rate running on the camera (perhaps using ROI) can help speed things up since these measurements are made at very short exposure.

The sensor data that is shipped with SharpCap is all packed into a single binary file - once upon a time there were individual files, but there ended up being so many that it slowed down the install considerably. The shipped sensor data is available to use (and see the graph) in the smart histogram, and it is used in some other places (for instance the EperADU figure will get saved to FITS headers), but is not 'public' for end users to read or work with. The measurements you do yourself do get saved into an accessible format (for SharpCap Pro license holders anyway) so that you can inspect the data.

In HDR mode, the gain value you read in SharpCap should be the same as the one sent to the ToupTek SDK - it looks like there is an issue with the maximum value of the gain control not being adjusted downward when you put the camera into HDR mode to reflect this behaviour, so as you noted, everything above gain 15000 will be the same.

cheers,

Robin

[RadHard]

Hello Robin,

thank you very much for your kind explanations. Now after my week-long experiment I think i got good grip about how to deal with correct gain setting for different scenarios. I got 20-ish hours of combined data from M101 of various quality using various settings.
It seems that for me the gain in the range of 150-350 (native Touptek) values in HCG worked the best. Somewhere where the brightest parts of the imaged object ended up in the 20k to 30k ADU range in 16-bit mode. Maybe its possible to push it but further or increase the exposure time, i was using 2 mins for the RGB. There was also moonlight adding quite some background offset.

I have seen subs taken in NINA at gain 100 and and i could not tell there was anything wrong with them, they were 90sec at similar f/ratio than mine, peaking at 4-6k for the brightest parts of M101. So i would say, as expected given his shorter exposure time and me using 2-3x higher gain. I have not observed the phenomenon that you described with the image not becoming fully saturated in his gain 100 subs. Most of the star centers was already at the 65520, which oddly seems like the max value, exactly the same as on my subs with higher gain. Maybe there was some problem in the past and Touptek fixed it? The latest cameras we received have 4.50 firmware. In any case i think going a bit higher with the gain in the area of slightly lower read noise is not hurting and can be compensated by bit shorter exposure.

I have found also your old video https://www.youtube.com/watch?v=ub1HjvlCJ5Y All is pretty much still valid

I ended up stacking almost all of it together, which was probably the not the best idea as the quality varied, some settings have overexposed the core, but at least i have some memory from this experiment

I have mostly focused on HCG mode, but took few samples with HDR and it looks like at low gains like 800, 1600, i got strange wavy noise pattern despite i was dithering. Maybe it was caused by light clouds, moonlight, not sure, but was only visible in the stacks from the lower gain at HDR.
Here you can see the pattern in the less bright area (it is about one hour long stack, blue filter, just autostretched):


Hands down the best result in HDR was with using the gain of 6400 (i have not tried higher). Gain 6400 in HDR, 60sec exposure and the M101 core was at about 22k ADU, max white level was in this case at proper 65535. HDR will need another round of experiments at new moon. About 2 hrs of data, green filter, autostretched.

[admin]

Hi Peter,

thanks for sharing your results - considering the challenge of imaging small galaxies in any detail, they all look great

It's interesting to see the sort of mottled pattern in the background of the lower gain HDR image - I don't think it has the same orientation in different parts of the image, so it doesn't look like walking/raining noise from hot/cold pixels. I suspect you might be right about thin clouds possibly being the cause here (perhaps only in a small number of subs, so that a pattern is visible rather than being smeared out if all the subs had different cloud textures).

cheers, and clear skies!

Robin

[RadHard]

Hello Robin, thank you, yes it turned out quite well considering it is taken by the 62mm/360mm fl Evolux refractor. Also focus was primarily on experimenting, rather than getting nice picture.
I will be checking the HDR mode later and lets see, i will then post the findings.

Tonight its again clear but pretty windy, so i am experimenting with 500ms exposures at gain 50000, with luminance filter. I was pretty surprised that i can go down so low and still see something useful in the 60sec stack from the M101 at f/5.8. I am saving the 60s livestacks and will then stack those.

This mode of capturing is quite useful, although I think if the Save and reset every ... will be possible to set to less than 1 minute, it will give even better flexibility.
Also I have not figured out how to to make the autosave to save the 16bit stack only, there is always the processed png and 32bit tif saved as well. Maybe it will be anyways better using the 32bit FIT for further stacking, i will try if Siril can take that.
Filter on star roundness in livestacking will be great addition, it appears that despite tight FWHM filter, some not so good frames make it into the stack. Maybe those are the first two where FWHM is ignored after the stack is saved and reset, not sure. When i look as the individual frames as they are coming, lot of times FWHM is still green, but the stars are visibly elliptical because of the strong wind gusts.
Oh and one more thing would be great, would it be possible to store the calculated FWHM in the fits headers for the saved subs? I made script using astropy to pre-sort the data based on target, exposure, gain, etc. So i could deal with all those different settings in my experiment. I am now also adding the calculation for the roundness. Running through multiple files is however extremely slow with astropy when the picture must be analyzed, rather than data just taken out from the header. If SharpCap already calculated the FWHM, it would be perfect if it can be saved in the fits header.

This is already offtopic here, i guess, i will look at the appropriate section and will post those suggestions. Thank you for your wonderful versatile program that makes all of this possible, calculating the FWHM and live stacking the frames at 2 FPS at 4K resolution on my N97 mini PC is quite impressive performance for me.

[RadHard]

Here is the stacked result, i collected about 140 of one minute stacked subs.
There is significant rainy noise pattern visible. I think the dark subtraction directly in SharpCap might have resulted in negative/zero pixels even though histogram was showing that offset of 2500 should be enough at gain of 50000. Those were quite extreme settings to keep the exposures short, but the wind was so bad that i would have either captured this or nothing. Actually thinking about it now, maybe the direction of this noise pattern somehow relates to how the wind was affecting the subs. I will stack the frames before and after meridian flip separately, maybe that will give some hint if the direction of that pattern changes.

[admin]

Hi,

Nice image - well done. I can see what you are saying about some level of raining noise mostly on the dark rather than the light side - if that is a SharpCap live stack then you could possibly try increasing the hot/cold pixel sensitivity to make SharpCap take more action against outlier pixel values next time.

if you are using live stacking in SharpCap then you should be protected against negative pixel value issues - SharpCap keeps track of -ve values after dark subtraction in this mode and the negative values go into the stack (and are typically later offset by other pixel values for the same area being positive due to random fluctuations). To be honest though, unless you have a very dark sky background, the chances are that skyglow will make you safe from that problem (as long as your dark and light frames match in terms of gain/offset/etc). The raw frames saved during live stacking are *not* dark subtracted, so if you perform secondary processing on those, you need to make sure that your processing handles all the dark subtraction properly.

I think it should be possible to save the calculated FWHM to the raw frames - I will just need to check that things happen in the right order (ie the frame is saved after the FWHM is calculated.

cheers,

Robin

[admin]

Hi,

just to check, what software do you use that is looking for the FWHM in the FITS headers, and what header name does it expect? I just had a quick look and can't see any obvious standard header name for FWHM, but if there is software already out there expecting a particular header name, I'd like to try to match it when writing from SharpCap.

cheers,

Robin