What's new in SharpCap this week... 6th June 2022

All the latest news about new features and improvements to SharpCap
User avatar
admin
Site Admin
Posts: 12984
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

What's new in SharpCap this week... 6th June 2022

#1

Post by admin »

Well, looks like I've missed a few of these, but the last few weeks have been largely bug fix related anyway, so not much of real impact that got missed out...

This week we have a new feature - hot pixel removal - and a change in behaviour for live stacking along with the usual minor changes and bug fixes.

Hot Pixel Removal (without needing dark frames)

This is something that we've had a number of requests for recently. Part of the desire for this comes from the new generation of very low noise CMOS cameras (IMX571, IMX533, etc). These cameras suffer from practically no amp glow or thermal noise, so the need for dark frame subtraction is much reduced. However, they still show some hot/warm pixels, so being able to get rid of those without using dark frames would be an advantage.

You can enable the new hot pixel removal by using the special 'Hot Pixel Removal Only' option in the 'Subtract Dark' dropdown. This also causes a sensitivity slider to show which allows the strength of the hot pixel removal algorithm to be tuned.
Capture.JPG
Capture.JPG (15.98 KiB) Viewed 3634 times
Note that right now it is deliberate that the hot pixel removal is an alternative to dark subtraction, not something you can enable in addition to dark subtraction (dark subtraction already does hot pixel removal based on the most extreme pixels found in the dark frame).

Perhaps the easiest way to see what the hot pixel removal does is to set your camera up to capture dark frames of maybe a few seconds, enable the main histogram functionality and try turning it on and off and adjusting the sensitivity.

Here's the histogram of a dark frame (RAW16) on an ASI071MC-Cool with hot pixel removal turned off:
Capture2.JPG
Capture2.JPG (54.99 KiB) Viewed 3634 times
And here's the histogram (same camera settings) with hot pixel removal turned on (sensitivity = 5, the default)
Capture3.JPG
Capture3.JPG (46.12 KiB) Viewed 3634 times
Almost all of those little spikes to the right of the black level peak, which represent hot pixels, have been eliminated.

To give a bit more detail on what the hot pixel removal does, it looks for pixels that are significantly brighter than any of the neigbouring pixels (how significantly depends on the sensitivity setting). When a pixel is found that exceeds the threshold, the value from that pixel is discarded and a value is taken instead from a nearby pixel (of the same colour for a colour camera).

My testing suggests that at the default sensitivity of 5, the new hot pixel removal detects and removes about 2/3 of the hot pixels that get picked up by a dark frame. However, since it is better at detecting the hottest pixels, those 2/3 account for about 90-92% of the total brightness in hot pixels that a dark frame would remove. At this sensitivity, hot pixel removal tends to have hardly any incorrect hot pixel detections (false positives).

As you turn the sensitivity up to 10, you can get to detecting about 80% of the hot pixels found by a dark frame, and about 98% of the hot pixel brightness being removed, but by this level there will be a significant number of false positive hot pixels being detected - this may or may not be an issue. At minimum sensitivity, you may only remove about 30-40% of the hot pixels that a dark frame will remove, but even then, you will probably get 70-80% of the total hot pixel brightness removed.

If you are interested in how many hot pixels are being detected/removed by this functionality, you can turn on extra logging for 'Image Processing' and you will get entries in the SharpCap log along the lines of '300 hot pixels removed out of 2000000'.

Live Stacking Change

Up until this version, Live Stacking would apply a 'behind the scenes' additional stretch to the stacked data if the images being stacked were significantly below the brightest values that can come from the camera. For instance if the maximum possible pixel value was 65000, but the actual maximum in the frames was only 10000, the live stacking could would pretend that the maximum possible value was only 10000 and 'pre stretch' the data before showing the histogram, etc.

In one sense, this was a good thing - it made it easier to adjust the histogram stretch, since the data was less scrunched over to the left hand side. On the other hand, recent reports have made it clear that this behaviour has some rather negative side effects - basically, if there are hot pixels in the individual frames that are brighter than the stars then as more frames are added and the hot pixels get smeared out, the behind the scenes stretch gets stronger, leading to the histogram drifting to the right, which in turn makes constant histogram tweaks necessary to keep a decent image brightness level.

After some consideration, I decided that any workaround to try to keep the behind the scenes stretch and fix this problem was likely to be a partial fix at best - the right thing to do was to remove the behind the scenes stretch entirely.

For most people, this will not affect anything - if you have any saturated stars in your individual frames then there will be no effect from this change. It's only those who are using underexposed frames for stacking (and that might not be the best use of your camera anyway) that will notice a change.

Since the 'behind the scenes' stretch was added, there have been changes to the live stacking histogram - addition of logarithmic axis mode and zoom mode - that mean that it is possible to adjust the histogram nicely even if it is mostly at the very left of the graph. I have also increased the data point density for the histogram graphs when in logarithmic mode to make these adjustments easier.

Other stuff

Other stuff this week includes a performance tweak to reduce CPU usage when using PlayerOne cameras and some bugfixes (including a couple of new UI issues).

cheers,

Robin
roelb
Posts: 214
Joined: Wed Mar 08, 2017 11:36 pm

Re: What's new in SharpCap this week... 6th June 2022

#2

Post by roelb »

Thanks Robin for this long ago requested addition to SC.
Roel
Celestron Nexstar Evolution 8 - Celestron Nexstar 6 SE - StarSense
ZWO ASI533MC-Pro -- ZWO ASI533MM-Pro
MarMax
Posts: 101
Joined: Sun Sep 19, 2021 11:43 pm

Re: What's new in SharpCap this week... 6th June 2022

#3

Post by MarMax »

Thank you Robin!

I'm really looking forward to not taking darks. :)

Regarding the background stretching of the histogram, I've always wondered about the walking histogram, and always sort of chasing it along. With the removal of this I'm wondering if using FWHM and Brightness filters is accomplishing the same thing. Regardless, it sounds like the result will be a much more stable live stack histogram.
User avatar
Menno555
Posts: 1044
Joined: Mon Apr 20, 2020 2:19 pm
Location: The Netherlands
Contact:

Re: What's new in SharpCap this week... 6th June 2022

#4

Post by Menno555 »

Great addition Robin!
I don't think I need the Hot Pixel Removal (HPR) for my AS071MC Pro but I think it will be useful for my non-cooled AS482MC when doing deep sky.
Plate solving is harder with that because of the small FOV and hot pixels. So for plate solving I have to make darks first, use that and then plate solving works. But maybe with the HPR that in the past now. Will see if it works when I use it.

Menno
User avatar
admin
Site Admin
Posts: 12984
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: What's new in SharpCap this week... 6th June 2022

#5

Post by admin »

Hi folks,

thanks for the comments - glad that this new feature is being welcomed so warmly :)

@Menno, I would be interested to hear how things go with the large pixel camera. One of the risks of the hot pixel removal algorithm is that it could wipe out stars if the FWHM is very small, which is most likely to happen with big pixels and undersampling.

cheers,

Robin
User avatar
Menno555
Posts: 1044
Joined: Mon Apr 20, 2020 2:19 pm
Location: The Netherlands
Contact:

Re: What's new in SharpCap this week... 6th June 2022

#6

Post by Menno555 »

Robin

With the combi of my f/10 scope and cameras I am in the oversampling area. The ASI482 indeed has large pixels with 5.8µm but it's still in the oversampling area. Or at least on that side depending on the seeing. So I assume it'll work.
Maybe tonight in between the clouds I can try it out.

Menno
User avatar
Menno555
Posts: 1044
Joined: Mon Apr 20, 2020 2:19 pm
Location: The Netherlands
Contact:

Re: What's new in SharpCap this week... 6th June 2022

#7

Post by Menno555 »

Hi Robin

I did try but I got only colored horizontal stripes when the HPR was enabled? Using the slider didn't make any difference. See screenshot below.
It was done with 1 second and high gain exposures but that's how I in general also make my plate solve exposures.
This was done with the Zwo ASI482MC camera.

Menno
1_1.jpg
1_1.jpg (141.76 KiB) Viewed 3556 times
User avatar
admin
Site Admin
Posts: 12984
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: What's new in SharpCap this week... 6th June 2022

#8

Post by admin »

Hi Menno,

interesting - almost as though it was finding *lots* of hot pixels. Can you post a raw frame (FITS or PNG format) with no processing that I can play with please?

cheers,

Robin
User avatar
Menno555
Posts: 1044
Joined: Mon Apr 20, 2020 2:19 pm
Location: The Netherlands
Contact:

Re: What's new in SharpCap this week... 6th June 2022

#9

Post by Menno555 »

Hi Robin

I don't have those anymore. But yes, there were a lot of hotpixels. The ASI482MC is very high in it's gain and although I would never go there during capturing because it all would be a mess, for plate solving it's useful.

Menno
User avatar
Tiago Ferreira
Posts: 143
Joined: Sun Mar 21, 2021 10:50 pm

Re: What's new in SharpCap this week... 6th June 2022

#10

Post by Tiago Ferreira »

So you found the source...nice. I will try gradient removal again.

I have my auto stretch settings lower than the default, 1 tick. Always found more pleasant to the eye with my set up, dob and zwo294mc with reducer. Should i readjust it and increase now that the behind the scene is out of scene? Or will not be affected? Thanks
Youtube channel: https://youtube.com/@DobsonianPower

12" f/5 Dobsonian GSO
8" f/6 Dobsonian GSO
6" f/5 Dobsonian Go-To Virtuoso Gti
Dwarf II Smart Telescope
Geoptik Eartha Eq Platform
ZWO ASI224MC + ZWO ASI294MC
Post Reply