Hot pixel sensitivity not quite working properly

A place to report problems and bugs in SharpCap
Forum rules
‍

If you have a problem or question, please check the FAQ to see if it already has an answer : https://www.sharpcap.co.uk/sharpcap-faqs

Please also read about Troubleshooting USB Issues before posting.

*** Please do not post license keys - please report any problems with licensing to 'admin' by private message ***

Please include the following details in any bug report:

* Version of SharpCap
* Camera and other hardware being user
* Operating system version
* Contents of the SharpCap log after the problem has occurred.
[If SharpCap crashes, please send the bug report when prompted instead of including the log]
Post Reply
Borodog
Posts: 487
Joined: Fri Jan 01, 2021 7:25 pm

Hot pixel sensitivity not quite working properly

#1

Post by Borodog »

Hi Robin,

I'm not sure exactly how the hot pixel removal & sensitivity are implemented, but there seems to be a problem somewhere. Using a nonzero hot pixel sensitivity is actually manufacturing hot pixels that are not there otherwise. This becomes particularly clear when you use a dark:
hotpixels.jpg
hotpixels.jpg (369.03 KiB) Viewed 685 times
On the left is a 15s light with the cap on (i.e. a dark). In the middle is the same but calibrated with a 16 dark master dark. Looks pretty good. But on the right is what happens when you raise the hot pixel sensitivity above zero. Strange double hot pixels are manufactured. Stretch level is the same for all three.

To be clear, the same thing happens when you use hot pixel removal without a dark. Most hot pixels are cleaned up, but a significant number of artificial double hot pixels are manufactured.

Thanks,

Mike
User avatar
admin
Site Admin
Posts: 17539
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Hot pixel sensitivity not quite working properly

#2

Post by admin »

Hi Mike,

this is with a colour camera, yes?

I think that what is happening here is that pairs of adjacent hot/warm pixels (adjacent for the same colour of pixel) survive the hot pixel detection/removal process. The process is pretty simple - it looks at the target pixel and the four immediately surrounding pixels of the same colour, and calculates if the target pixel is an 'outlier' in terms of the values of the other 4 pixels. If it's an outlier then it's a hot pixel and is cleaned up.

If you put two hot pixels next to each other then they help each other to survive - when you are checking one of them, the other is one of the neighbours and the first one doesn't look like an outlier, and the same when checking the second (the other way around).

This is just a limitation of the current way of detecting hot pixels - to be able to reject 2 pixels in close proximity would need a more sophisticated algorithm, probably covering something like 25 pixels around the one being examined (I don't think a grid of 9 would be enough).

All of this is a bit of educated guesswork, since I can't make this happen in my testing (made some darks with an IMX571 sensor, applied the dark and hot pixel removal and not getting any double pixels like those shown, although they would be adjacent rather than 1 apart on a mono sensor). Happy to look at the actual image files rather than just screenshots to check though.

cheers,

Robin
Borodog
Posts: 487
Joined: Fri Jan 01, 2021 7:25 pm

Re: Hot pixel sensitivity not quite working properly

#3

Post by Borodog »

Robin,

I don't think that is what is happening. Again, there are no obvious hot pixels that survive application of the dark, at least not at that stretch level. The dark looks like its doing its job pretty well. But after turning hot pixel sensitivity above zero, double hot pixels are manufactured where there were none before. The HPR code can't be interacting with neighboring hot pixels because by that time there are none.

What I CAN see however is that the double hot pixels correspond to single hot pixels in the raw light prior to dark calibration. But after dark calibration, those hot pixels are gone. This makes me think that there is some sort of order of operations problem, almost as if the hot pixel removal code is running before the dark subtraction instead of after it, or is at least using the non-dark subtracted light, even if it is running after the dark subtraction. Then your theory would make sense. If the dark subtraction is done first, and the HPR code were operating on the dark subtracted light, then those hot pixels should be gone, and I don't see how they could be resurrected, much less doubled, by the hot pixel removal code.

There have been a number of reports, including by myself, over at least the last 6 months of hot pixels surviving dark subtraction, and I think whatever is going on here is probably related.
User avatar
admin
Site Admin
Posts: 17539
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Hot pixel sensitivity not quite working properly

#4

Post by admin »

Hi Mike,

that's a possibility - the hot pixel removal happens first (it works best in a frame that has had no other processing done on it), but it does 'borrow' a pixel value from the nearest neighbour of the same channel when it needs to replace a hot/cold pixel. That should be borrowing from a pixel that has already been corrected for hot/cold (earlier in the frame), but maybe there are situations where that goes wrong.

Could you share the sample frames please? It's not happened so far in my testing, so must require some particular condition to happen that isn't obvious.

cheers,

Robin
Borodog
Posts: 487
Joined: Fri Jan 01, 2021 7:25 pm

Re: Hot pixel sensitivity not quite working properly

#5

Post by Borodog »

Robin,

Yes, I'll make some and email them to you.

Thanks,

Mike
Post Reply