Hot Pixel issues with TouptekIUC26000KPA after upgrading from 4.1.13218 to 4.1.13417

[alphatripleplus]

I've had no issues over the last 2-3months using Sharpcap 4.1.13218 with my Touptek2600KPA colour camera and PlayerOne Apollo-M Mini mono camera during that time. However, I recently updated to SharpCap 4.1.13393 and then 4.1.13417, and in both versions the Touptek camera exhibits very severe uncorrected hot pixels in calibrated livestacks when using darks created with the earlier version (4.1.13218) of SharpCap. When using both lights and darks with the latest versions (4.1.13393 and 4.1.13417), the hot pixels are removed in livestacks, but dark "holes" appear that may be a result of the 4.1.133417 darks with excessive hot pixels overcorrecting the lights. ( Although the Touptek camera does not have a temperature sensor, I usually run an AIS290MM mini simultaneously (with the same sub-exposures) under ASILive and use the temperature reading from the 290 as a proxy for the Touptek camera's temperature).

To help show what I'm looking at, I will include a google drive link below to 3 sub-directories containing a livestack calibrated fits file and the corresponding masterdark for the following combinations I took last night : 1) 4.1.13417 livestack with 4.1.13218 masterdark, which has both hot pixels and dark "holes" in the livestack; 2) 4.1.13417 livestack with 4.1.13417 masterdark, which just has the residual dark "holes" in the livestack; 3) 4.1.13218 livestack with 4.1.13218 masterdark, with no issues in the calibrated livestack. I'm pretty confident that the temperature of the lights and darks were closely matched.

https://drive.google.com/drive/folders/ ... drive_link

I did not save individual fits files used in the livestacks or in the masterdark.

Thanks for any help.

Errol

[admin]

Hi Errol,

thanks for the clear report and included files. There are two real possibilities here - something has changed in the ToupTek SDK between the two versions, or something has changed in SharpCap. I can't see anything obvious in the changes to the ToupTek code in SharpCap between those two versions that would cause this sort of thing, so I'd be guessing an 80% chance or so of it being the SDK, but best to make sure...

If you install 13417, you can copy out the SDK (toupcam.dll) from the install folder and then put a copy of that file somewhere. Then, go back to having 13218 installed, but once you have installed it, copy the toupcam.dll file that you pinched from 13417 over the one in the install folder. You now have a hybrid version that uses the newer ToupTek SDK, but the older SharpCap code - it should work (and more likely to work that way round than the other way round taking the SDK from the older version into the newer version of SharpCap).

Give your hybrid version a test when you have a chance - if it gives good results with the darks from the original 13218 then the issue is a change to the SharpCap code (and maybe we can find and fix that). If it gives problems with those original darks but good results with the darks from 13417 then it is most likely a change to the ToupTek SDK that has caused the problem, and there is less likelihood of getting any sort of fix, but at least we understand what has gone on.

cheers,

Robin

[alphatripleplus]

Hi Robin,

Thanks very much for the clear suggestion on what to try. I'll create and use the hybrid version of SharpCap with both 13218 and 13417 darks, and see what happens next clear night out.

Regards,

Errol

[alphatripleplus]

Hi Robin,

I had a chance last night to use the Hybrid software (13218 build plus 13417 ToupCam.dll) with both the 13218 darks and 13417 darks on the same target. I've included two more folders with the livestack fits and the masterdarks used in the stack in the same google drive location.

https://drive.google.com/drive/folders/ ... drive_link


Unfortunately, neither combination of the Hybrid software with either the 13218 darks or the 134170 darks produces results as good as the older 13218 software with the 13218 darks. I found that the combination of Hybrid and 134170 darks was better (no noticeable hot pixels, but some dark "holes") than the Hybrid and 13218 darks (many hot pixels and some dark "holes'), but not close in quality to using the old 13218 software with 13218 darks. Perhaps that indicates the issue is the new SDK?



Regards,

Errol

[admin]

Hi Errol,

I think it is probably largely related the SDK from those results, which means that the best thing to do is to capture new dark frames with the new SDK to match the new light frames.

If I remember rightly, the model you are using is the industrial version of the IMX571 with a limited maximum exposure and quite a bit of hot pixel noise compared to the astro version (but, at one point, they were very cheap to buy...). I suspect that from ToupTek's point of view, keeping those models behaving in exactly the same way from version to version of the SDK is not a high priority (particularly if the usage is at the extreme maximum exposure for that model).

cheers,

Robin

[alphatripleplus]

Hi Robin,

Yes, you are right I am using the Touptek industrial version of the IMX571 - which you may also have?

Unfortunately the results I posted initially from the first night show that using the latest SharpCap 13417 (which has the Touptek SDK) and new darks captured with 13417 gives lots of residual dark "holes" in the stacked image, when calibrated with the 13417 darks . I've taken two sets of darks with SharpCap 13417, so I guess I can try more experimentation with darks to see if I can find something that works without the residual dark holes. Otherwise, I'll have to stick to the older version of SharpCap built with the older Touptek SDK - the dark frames were working very well with the older software, even with temps close to 30C - and hope I can stay with SharpCap 4.1.13218 for this camera.

Thanks.

Errol

[admin]

Hi Errol,

yes, I thought it was that version of the camera - I have one, but have lent it out at the moment, so cannot test myself with that model

There are a few changes I made to SharpCap for that particular model of camera to make imaging easier - do make sure you have those enabled:

1) the option to 'Detect and Ignore outlier values' in the create dark window. That checks the individual dark frames to try to find pixels that are only intermittently hot. Including those pixels in the dark image creates a warm pixel at that point which then can cut black holes in the dark subtracted images (in the frames where the pixel decides not to be hot).

2) the option to enable 'Dark scaling', which tries to re-scale the dark frame to compensate for changes in temperature.

Both of those might help, but still odd that the newer SDK seems to have less ability with dark subtraction.

In your situation, I would be tempted to do a bit of pixel-level viewing of the individual dark and light frames - find some hot pixels in the dark frames, note the co-ordinates of those pixels and check that they are also showing up at the same co-ordinates in the light frames (and consistently across frames). It should all match up, but if not then maybe that will be interesting evidence that could point towards what is going wrong.

cheers,

Robin

[alphatripleplus]

Hi Robin,

Thanks for the suggestions - I have been using the "Detect and ignore outlier values" option when creating darks for all my cameras. I think you created that feature when I found the dark "hole" problem about a year ago with my Apollo-M mini with H-alpha subs, as discussed in this earlier forum topic [viewtopic.php?p=42294#p42294]. It has been working well with the Touptek industrial camera, until I upgraded from 4.1.13218.

I haven't tried dark scaling yet, mostly because I have had success using darks labelled with an estimated temp from a sensor on my ASI290MM running in parallel. I may try dark scaling if nothing else works.

My suspicion looking at the two master darks (in the google drive location) which were taken at very similar temps (assuming my ASI290MM proxy temperature is reliable) is the 4.1.13417 master dark (estimated temp 29.7C) shows a big bump in both average pixel value and the standard deviation of the pixel distribution versus the 4.1.13218 masterdark (estimated temp 29.3C). (I am surprised that an increase of only 0.4C temp between the darks leads to an increase in average pixel value from 365 to 445 and an increase in the histogram std. devn. from 258 to 312.) I am going to try and take more master darks using both 4.1.13417 and 4.1.13218 to see if this pattern continues , i.e. "hotter" pixel distribution for 4.1.13417 vs 4.1.13218 at very similar temperatures. If I could have both versions installed on the same machine it would make the comparison a little easier, but I can do a run with one version of the software, uninstall, and then reinstall the other version and repeat the run.

I did do the painful pixel peeping exercise on individual darks to find the exact location of the anomalous hot pixels in the earlier spurious hot pixel investigation with my Apollo-M mini camera that I mentioned above. Hopefully, I can avoid doing that this time, if more darks show something systematically different in the pixel distributions between both versions.

I will report back when I have had a chance to do more dark frame comparisons.

Regards,

Errol

[admin]

Hi Errol,

you can have both versions installed at once in two ways...

The easy way is to install one version as 64 bit SharpCap and the other as 32 bit SharpCap. 32 bit has memory limitations, but should be OK if all you are doing is capturing images to look at them, even on a high res camera. It might hit issues with more complex things like live stacking with darks/flats at maximum resolution.

The slightly harder way is to install one version, then copy the whole installed folder somewhere safe, then install the other version. SharpCap will happily work that way with a copy of the installation files in a different folder.

cheers,

Robin

[alphatripleplus]

Hi Robin,

Thanks for the suggestions on dual installations - I will definitely make use of them. I did make use of a 32 bit SharpCap version in earlier testing, and as you noted, memory constraints (particularly with the old astro laptop I am using) made it problematic for creating master darks even at 2x2 binning. However, it worked fine for single snapshots using a ROI.

I did some master dark comparisons yesterday between vs 4.1.13218 and vs 4.1.13417 using 20 subs at 2x2 binning and the "Detect and ignore outlier values" option each time, with the camera having "warmed" to a steady state, which was pretty hot even in an air-conditioned room. Looking at the pixel histogram for 4.1.13417 master darks, I noted that the R,G and B histograms were offset from each other: In particular the G histogram was above the R and B histograms and the G histogram had a number of pixels with zero value (see screenshot below). I was surprised by this as with earlier versions of SharpCap, using a digital offset of 100 moved the histogram just enough to have a non-zero minimum pixel value , and I had always seen the R, G, and B histograms pretty much on top of each other, as was the case with the 4.1.13218 master darks (see 2nd screenshot).

4.1.13417 master dark histogram

Masterdark_v4_1_13417_30.7C_ctemp6503_ctint1000.png (601.24 KiB) Viewed 2116 times

4.1.13218 master dark histogram

Masterdark_v4_1_13218_30.5C.png (674.83 KiB) Viewed 2116 times

(I have added the fits files and screenshots of these comparison darks in a folder labelled "Master Dark Comparisons" in the google drive location mentioned in the earlier posts.)


I then realized that one feature of 4.1.13417 that I had used was the Image Control tab under Camera Controls, as it now has a number of settings that did not exist under 4.1.13218. I had used the first two (colour temp and colour tint) to try and remove the awful colour in the live view when I first started using 4.1.13417, and so I wondered if these settings were impacting not just the live view, but any saved fits images, including darks. I had shifted both the colour temp and colour tint to the left (smaller values) the first time using 4.1.13417 and wonder if that had caused the shift in histograms.

I followed up with some single 15sec captures with scope capped using 3 different values of (colour temp and colour tint) in 4.1.13417 and compared them with a single 15sec capture in 4.1.13218, which does not have those controls. It does appear to me that R,G and B histograms in the fits dark captures are affected by the choice of those image control settings. Again, I was surprised by this as I had assumed changing anything under the Image Control tab would only affect the live view displayed and not saved fits files.

I have included a folder with the single capture comparisons labelled "Single Capture Comparisons" in the google drive location mentioned in the earlier posts.

I also think the offset between the R, G and B histograms in 4.1.13417 leads to higher values of the histogram standard deviation measure, i.e. more histogram dispersion across colours, relative to all 3 histograms on top of each other in 4.1.13218.

Does any of this make sense, or perhaps something else is going on?

Thanks,

Errol