Problem with livestacks - dark subtraction - in EEA

[timh]

Hi Robin,

I was running SC 4.1 (latest version) livestacking for EEA observations of some galaxies - with the appropriate darks and flats. At the same time also dithering, guiding and saving the stacked lights for later processing.

It worked well - after 60 min I had a nice image of NGC4449 up on the screen. However it is rather spoiled by a few bright R, G and B pixels that were not subtracted and as you can see do a a sort of dance pattern in accord with the dithering movements (which occurred after each two frames - so 30 dots for 60 frames I think).

I am wondering whether the dark subtraction code is not properly accounting for the dithering ? so the light fram is moved relative to the dark perhaps?

Tim

[Borodog]

I have also seen this, which I believe I communicated in a private email. Here is another report on CN:

https://www.cloudynights.com/topic/9212 ... ry13430665

That thread actually contains *two* reports, in fact, as MarMax (who also posts here) chimes in at the bottom that he has seen the same thing.

[mgCatskills]

I'm the originator of the CN post that Borodog references. If I can be helpful in any way, e.g. by supplying the original image or settings files, probably best to PM me on Cloudy Nights. (Better than email as i get a lot of spam and often miss messages I'm not expecting)

[timh]

Just to confirm that I am using an ASI294MC camera -- re suggestion over on CN that there might be an issue around that particular sensor returning out of range values for some pixels?

[admin]

Hi,

the issue looks like there are a handful of hot pixels in the light frames that are not present in the dark frames, so they are not being corrected.

When SharpCap is using dark frames, it will look for statistically hot pixels in the dark frame and then 'airbrush' those out in the light frame with a value from a nearby pixel - this is in addition to the normal dark subtraction code.

When SharpCap is not using dark frames, but 'hot pixel' or 'hot/cold pixel' removal is selected, the code looks for anomalous pixels in each light frame and airbrushes those out.

The difference here is that when using a dark frame, you rely on the dark frame hot pixels matching those in the light frame - at least in the current code. This means that the darks and lights should match in terms of temperature, gain, offset, exposure, etc. It's also possible that new hot pixels could appear on a camera (or disappear) over time for reasons that we don't understand - if you were using old dark frames and got this result then I would try creating new dark frames to see if that helps.

A selection of raw, unprocessed light frames and also the dark frames in use (master dark definitely, raw darks if possible) would let me dig into this a bit more to see if I am missing anything,

cheers,

Robin

[mgCatskills]

Hi, Robin.

I'm experiencing this issue using a 294mc-pro camera. As far as I can tell everyone on Cloudy Nights that experienced this problem was using a 294 based camera (one using a QHY 294) as was timh.

I'm using darks but no flats. All of my darks are time, gain, and temp matched. When I first noticed this problem I was using darks that were a few weeks old. I replaced them with new ones, and it seemed to reduce the problem but did not eliminate it. I've continued to replace the darks frequently as it does seem to help...

The latest incident (on CN) was using a dark I had captured at the end of the night before, i.e. was one day old.

Because of amp glow on the 294, the pixel removal routine isn't useful. I tried it as an experiment, saw both the amp glow and the fact that the hot pixels were still coming through, and abandoned it.

I don't normally capture lights or individual dark frames but will do so on my next outing and will get you the data.

Thanks,
Michael

[admin]

Hi Michael,

interesting about the 294 - it's an odd sensor as the real resolution is 2x higher in both directions and by default it operates in a binned mode to get the ~11Mp resolution. On the mono cameras you can usually unlock a high resolution 47Mp mode (in theory the colour cameras can too, but it's not much use as the bayer pattern is odd - see below - and nothing knows how to process it).

Code: Select all

RRGG
RRGG
GGBB
GGBB

The 294 also has the weirdness with the gain range that doesn't give saturated images if you put the camera into HCG mode. I don't know if any of this is relevant, but I do have suspicions about the 2x2 binning happening in hardware might mean that a 'hot' pixel can sometimes be 1/4 hot...

With some test files I will rig the code to run both dark subtraction *and* hot pixel removal on the light frame and see if that does the trick. I can't quite work out how to show that option in the UI yet, but no need to solve that part of the problem until I am sure it works.

cheers,

Robin

[timh]

Hi Robin,

Thanks. Maybe it all is indeed to do with some idiosyncrasy of the chip? Just to note that the problems is only apparent while I am viewing live in EEA mode - generally I also concurrently save the stacked raw light files and then calibrate, debayer and integrate them afterwards using exactly the same dark and flat files as used during Sharpcap livestacking. This post hoc processing (in PixInsight) as opposed to the SC live stacking generates integrated images that do not show the dance of the hot pixels.

Maybe it would be helpful to send you the dark frame, light frame and lights for the above image?

Can I send them as an Email attachment?

Tim

[Icharuss]

It is not limited to the 294 sensor as I have the same issue with the 2600MC-Pro. In my case though I was using H/C pixel removal and not master dark frames. Next time out I will test with matched darks...

[admin]

Hi,

thanks to Tim sending me a batch of images to test on, I can confirm that in his case, the issue is hot pixels in the light frames that are not present in the dark frames. For instance, there is a pixel at x=1504, y=964 with values in the region of 7000 in each of the light frames. The surrounding pixels of the same colour have values of about 1100 to 1200 and the dark frame pixel at that position has a value of about 660.

Setting a value of about 7 or higher on the hot pixel removal strength will get rid of pretty much all the hot pixels in Tim's frames, but you cannot currently do that with dark subtraction also on to get rid of the amp glow of the 294. I have hacked the code to allow both in testing and it seems to work nicely.

To deal with this in SharpCap I think I will need to separate the hot/cold pixel removal from dark subtraction, so that both can be enabled. I will have to consider that, since if I just go ahead and do it, people who have previously set up hot/cold pixel removal in the dark subtraction control will find it no longer gets set when loading profiles/restoring camera settings, which is no good...

cheers,

Robin