Problem with livestacks - dark subtraction - in EEA

[Borodog]

Also not using a 294. I believe I saw it with both the 662 and 533 sensors, although it’s been a while, more than a month.

For what it’s worth, I don’t recall seeing it the last time out, when I had a master dark comprised of 30 darks. I theorize that it may have to do with the number of darks in the master. I was under the impression that even a single dark would adequately correct hot pixels, if not minimize the statistical noise. I even made a post about it on CN. But now I am thinking that a very low number of darks in the master may be causing this problem. So it would be useful if the people experiencing this problem could comment on how many darks are in their master darks.

Robin,

Perhaps a good route is adding another option to the dark dropdown that enables both dark subtraction and cosmetic correction.

[admin]

Hi,

there was a recent thread on here about problems with darker pixels in the stack caused by a rogue hot pixel in the darks - just in one dark was enough to boost the average pixel value at that point and then the brighter pixel in the dark cut dark holes in the image. I have added a new option in the dark creation to filter the darks to remove rogue pixel values (sigma clipped stacking of the darks basically). So, I think part of the issue is that hot pixels just aren't 100% consistent...

I have gone with the approach of showing the 'hot pixel sensitivity' when a dark is selected, so that you can enable both. If you have a dark and turn the hot pixel sensitivity down to zero (currently the minimum is 1, will be zero in next update) then you can get 'pure' dark subtraction. With the sensitivity turned up you get dark subtraction and hot pixel removal. I think that is the minimum impact on existing users.

cheers,

Robin

[Borodog]

Robin,

I like that a lot. All post-stacking software that I am aware of lets you use both, dark subtraction and cosmetic correction.

[timh]

Thanks Robin. I also like that option. Borodog's comment is true of e.g. PI post-processing which seems to get rid of the hot pixels - i.e. following calibration with the flat and dark, the cosmetic correction process is set to remove hot pixel outliers at various levels of significance with a slider. I normally set at about sigma 3. So something that could do all of that on the fly would be great.
Tim

[admin]

Hi,

yes, I think the problem was that with SharpCap having dark correction activated, the statistical removal of hot pixels was based purely on the dark frame (it's easier in the dark frame - no stars to worry about). That's fine on the assumption that the hot pixels remain pretty constant and has worked fairly well for a number of years, but now we are in the situation that we are seeing issues from changing hot pixels and also I have code capable of removing hot pixels in light frames now, so the solution becomes clear

cheers,

Robin

[ChrisR Oz]

Hi Borodog,

To answer your question about the number of dark frames, I have been using between 200-500 darks for a 294MC Pro, stacked, e.g. 200x60s or 500x30s, for about a year now, in order to minimise the statistical scatter contribution upon subtraction. So I don’t collect darks regularly, and I do see the “dither dance of hot coloured pixels” that have been reported here in live stacks. This does not bother me particularly, as I stack again in PI and these get treated there.

Cheers, Chris.

[admin]

Hi,

interesting that you use so many darks - I did some maths a week or two back to try to work out how many dark frames were actually needed to avoid them contributing to the overall final noise level. The preliminary results were interesting, but then I realised that I didn't account for how good the darks are at cancelling pattern noise (how much correlation there is between pattern noise in one set of dark frames and a set of lights, etc). I think it is going to make the maths harder, but I will give it a go. If it all works out, it may come out as a way to calculate the number of darks in the same way that SharPCap can calculate the minimum exposure for deep sky.

cheers,

Robin

[MarMax]

During my last dark site EAA trip I had a problem that was caused by the darks. Here is my post in another topic.

viewtopic.php?p=42743#p42743

I say that I've been taking darks the same but I actually did change the number of dark frames. For the past two years my standard has been to take 20 dark frames. And for the past two years I've never had a problem with black clipping at home when using darks. With the Borodog post on CN about the number of darks I decided to try 10 darks. So the dark site problem with black clipping was using 10 dark frames.

I'll need to do more testing at home and will go back to 20 dark frames unless it's best to try and replicate the black clipping problem. From my experience at home (B8) and at a dark site (B2), a much higher Brightness/Offset is required at a dark site. I had previously thought the black clipping problem was from not using a high enough Brightness/Offset. At this point I'm not quite sure how to troubleshoot since I'll be at home where I've never had a black clipping problem (except for Non-Linear Gradient Removal - which has been tweaked).

[timh]

Just to add in one observation on the livestack problem. I have also observed that - as well as with e ASI294MC camera - you also get the hot pixel problem with the ASI294MM mono camera used in either mode - 11Mb or 46 Mb.

Tim

[timh]

To the point that MarMax makes -- I have also seen something similar in the past - i.e. with needing to increase the offset of the lights when going to dark sites - or else ending up with a clipping problem during EEA.

Tim