A different flat option for IMX294 based camera owners

Got an idea for something that SharpCap should do? Share it here.
Forum rules
'+1' posts are welcome in this area of the forums to indicate your support for a particular feature suggestion. Suggestions that get the most +1's will be seriously considered for inclusion in future versions of SharpCap.
Borodog
Posts: 330
Joined: Fri Jan 01, 2021 7:25 pm

A different flat option for IMX294 based camera owners

#1

Post by Borodog »

This will not help me personally, as I do not own an IMX294 based camera. But it is an extremely common and frustrating problem for those that do, to the point that many return the camera in exasperation.

The IMX294 chip can suffer from a problem in which the red channel in particular, and sometimes the green as well, suffer "swirls", particularly around the perimeter of the frame. The exact cause of this is still debated, but it is definitely associated with flat calibration. Here is a thread with a good example:

https://www.cloudynights.com/topic/8290 ... ry11977169

The swirl is not seen without flat calibration, but then of course without flat calibration dust shadows and vignetting are not corrected.

In that thread I came up with a solution to the problem, which was then modified a bit by another poster because sometimes the artifact shows up in the green channel as well as the red, but never the blue. The solution is to create a master flat as normal, but then debayer it, and only use the blue channel for flat calibration of the lights. I figure that this should be pretty straightforward to implement and can be turned on with a setting in the SharpCap settings, such as "Use Blue Channel Only For Flats."

Here is an example from that thread showing the problem after standard flat calibration:

Image

And here is a calibrated light from another user who implemented the blue-channel-only flat fix:

Image

As you can see, the artifact is completely eliminated.

This may be relevant to other cameras than IMX294 based ones, but that is the one where I see this problem come up over and over and over.

Thanks for your consideration.
ChrisR Oz
Posts: 158
Joined: Tue Sep 28, 2021 11:03 am
Location: Melbourne

Re: A different flat option for IMX294 based camera owners

#2

Post by ChrisR Oz »

Hi Borodog,

Very interesting. I have seen posts like that. I find my ZWO 294MC pro calibrates fine. I use a white image on my iPad screen to fully illuminate the OTA entrance and stack the Flats in SC with just Bias subtraction. The exposure is short, so issues with amp-glow do not appear.

I wonder if the problem comes with a bluish flat field, using a sky flat for example, perhaps with some weak scattered red coming in from an external source?

Cheers, Chris.
Celestron EdgeHD 8, reducer 0.7x, Star Sense, CGX-L mount, Focuser, CPWI; Starlight Xpress AO, OAG and Filter Wheel; ZWO 294MC/294MM Pro and 174MM mini; SharpCap Pro, PHD2, Powermate 2x, Baader Neodymium, Astronomik CLS-CCD, ZWO UV/IR, Duoband filters.
User avatar
admin
Site Admin
Posts: 13173
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: A different flat option for IMX294 based camera owners

#3

Post by admin »

Hi,

I don't know if this is relevant, but the IMX294 is a bit of an odd sensor - the switch from LCG to HCG mode gives a gain of 4.5x, but the documentation recommends not using HCG mode unless a standard gain of 2.0x is also used, giving a minimum gain in HCG mode of 9.0x (900 on Altair, about 200 on ZWO).

Now the reason that you shouldn't use the sensor in HCG at gains below 9.0x is that the sensor does not fully saturate in those conditions, even with massive overexposure. This can cause issues with flat frames because you *think* your flat frames are correctly exposed, since the histogram peak is not touching the right hand side, but they are not - the sensor can be 'sort of saturated' in the sense that it is at the maximum value it is going to get to at those gain settings, but it will not be responding correctly to the differences in brightness across the flat.

This is the reason that Altair 294 cameras in SharpCap switch to HCG at 900 gain, not 450. ZWO tell me that they have a workaround for this issue so that the camera is OK to switch to HCG at 4.5x gain (about 120-130 gain on ZWO) rather than needing to wait to 190-200. I am now wondering if perhaps that workaround is only partial. If people only see this problem between gains of about 120 and 200 then that might be the case.

Note - this also applies to the mono version (which actually uses the IMX492 sensor).

cheers,

Robin
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: A different flat option for IMX294 based camera owners

#4

Post by oopfan »

Interesting. Here is another way of thinking about it...

We are familiar with Quantum Efficiency (QE). QE of 80% means that 5 photons are required to generate 4 electrons. QE of 20% means that 20 photons are needed to generate 4 electrons. It is well known that all sensors suffer from some degree of flattening of the QE curve as pixels reach saturation. For example, a sensor may be linear up until 50,000 ADU at which point the QE drops until saturation is reached. Normally this doesn't affect AP but it does affect photometry. Robin said, if I remember correctly, that in certain situations the IMX294 won't even reach saturation (i.e. 65535 ADU).

We should never assume that each pixel is identical to the next. Each has an individual personality that explains why we have hot, warm, and cold pixels! However, for the majority of normal pixels we expect that their irregularities are evenly distributed across the wafer. This appears not to be the case with the IMX294. There seems to be a pattern. Perhaps some anomaly in the fabrication process?

EDIT: Then again maybe all sensors would exhibit this anomaly if it weren't for the control circuitry that ensures that pixels saturate properly. Perhaps this is just an issue with ZWO's circuitry.

Brian
RoscoeD
Posts: 3
Joined: Wed Jun 29, 2022 3:02 am

Re: A different flat option for IMX294 based camera owners

#5

Post by RoscoeD »

That was my photo that Borodog posted showing the red artifacts ("My purpose in life may in fact be to serve as a warning to others").
Some details behind that:
  • I took all pictures and calibration frames at gain 121 (ZWO specs show a drop-off in read noise at 120, I padded that for just-in-case)
  • Flats and dark flats were taken with 3 sec exposures (apparently there is evidence that the 294 does not like short exposures)
  • I used an iPad for flats. Once the artifacts prove to not calibrate out, I tried my wife's laptop. It worked perfectly, but was way too cumbersome (and it's my wife's) so not a long term solution. I then bought an LED tracing tablet from Amazon which was better than the iPad but still produced unacceptable artifacts in the calibrated lights. On the search for an acceptable light source (Sky flats are challenging due to the "long" exposures required meaning significant attenuation is required). Spike-a-Flat may be the answer but my budget at the moment doesn't support that expensive of a solution
  • I haven't the experience with my stacking S/W of choice to figure out the debayer technique Borodog described so the idea of using that as a workaround intimidates me
The "gain should be 200 instead of 120" is new to me (and I've been researching this sensor for weeks). I'd love to know more.

Thoughts and suggestions welcome...
Roscoe
USAF Retired
WO GT 71 | EQ6-R | ASI294MC Pro | ASI Air Plus
User avatar
oopfan
Posts: 1321
Joined: Sat Jul 08, 2017 2:37 pm
Location: New York
Contact:

Re: A different flat option for IMX294 based camera owners

#6

Post by oopfan »

Hi Roscoe,

Thanks for joining the discussion!

I just noticed that you have a William Optics GT 71, similar in size to my William Optics ZenithStar 71. For about $20 you can easily build an EL Panel that works great and that is very portable. Let me know if you are interested.

Brian
RoscoeD
Posts: 3
Joined: Wed Jun 29, 2022 3:02 am

Re: A different flat option for IMX294 based camera owners

#7

Post by RoscoeD »

oopfan wrote: Wed Jun 29, 2022 11:17 am Hi Roscoe,

Thanks for joining the discussion!

I just noticed that you have a William Optics GT 71, similar in size to my William Optics ZenithStar 71. For about $20 you can easily build an EL Panel that works great and that is very portable. Let me know if you are interested.

Brian
I am, you can reach out to me via PM if you wish to avoid spamming this thread any further than I have already :)
Roscoe
USAF Retired
WO GT 71 | EQ6-R | ASI294MC Pro | ASI Air Plus
User avatar
admin
Site Admin
Posts: 13173
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: A different flat option for IMX294 based camera owners

#8

Post by admin »

Hi,

so I just happened to have the Altair 115M camera on my desk when catching up on this thread. The Altair 115M has the same sensor as the ZWO294MM, but with the Altair camera I have the ability to control the LCG/HCG (low conversion gain/high conversion gain) switch manually in SharpCap code, rather than it being automatic based on the gain value as it is with ZWO.

As a test, I changed my code so that the Altair camera switches into HCG at 4.5x gain (450 gian on the Altair scale, about 130 on the ZWO). Then I turned the exposure right up to give an image that is massively over exposed (every pixel should be pure white) and took a screenshot of the image and histogram
Capture.JPG
Capture.JPG (130.94 KiB) Viewed 2227 times
Looks like an almost perfectly exposed flat frame from the histogram, doesn't it? Not so - it's just the way the camera works when the HCG mode is turned on and the total gain is less than ~8.0 to 9.0x

So, my advice would be as follows

If you are using a camera based on the IMX294 or IMX492 sensor, test the gain that you intend to use before committing to image with it. The way to test is to set up the camera at the gain you want and significantly overexpose the image (long exposure, lots of bright light, etc). If you can get the histogram to be a single spike at the far right hand side then you are OK. If you cannot get the histogram to go to that single spike at the right in spite of massive overexposure then *do not use that gain value* - try a higher one if you want to take advantage of the lower read noise in HCG mode or a lower one if you want a bigger full well depth.

cheers,

Robin
Borodog
Posts: 330
Joined: Fri Jan 01, 2021 7:25 pm

Re: A different flat option for IMX294 based camera owners

#9

Post by Borodog »

admin wrote: Sun Jun 26, 2022 1:11 pm Hi,

I don't know if this is relevant, but the IMX294 is a bit of an odd sensor - the switch from LCG to HCG mode gives a gain of 4.5x, but the documentation recommends not using HCG mode unless a standard gain of 2.0x is also used, giving a minimum gain in HCG mode of 9.0x (900 on Altair, about 200 on ZWO).

Now the reason that you shouldn't use the sensor in HCG at gains below 9.0x is that the sensor does not fully saturate in those conditions, even with massive overexposure. This can cause issues with flat frames because you *think* your flat frames are correctly exposed, since the histogram peak is not touching the right hand side, but they are not - the sensor can be 'sort of saturated' in the sense that it is at the maximum value it is going to get to at those gain settings, but it will not be responding correctly to the differences in brightness across the flat.

This is the reason that Altair 294 cameras in SharpCap switch to HCG at 900 gain, not 450. ZWO tell me that they have a workaround for this issue so that the camera is OK to switch to HCG at 4.5x gain (about 120-130 gain on ZWO) rather than needing to wait to 190-200. I am now wondering if perhaps that workaround is only partial. If people only see this problem between gains of about 120 and 200 then that might be the case.

Note - this also applies to the mono version (which actually uses the IMX492 sensor).

cheers,

Robin
Robin,

Which documentation is this? Can you provide a link please?
User avatar
admin
Site Admin
Posts: 13173
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: A different flat option for IMX294 based camera owners

#10

Post by admin »

Hi,

it's in the Sony datasheet for the IMX294/492 - unfortunately I can't share a copy here, but you may be able to find a copy online somewhere if you search a bit. The spec seems to specify that the normal gain should be at least ~8dB before moving to HCG, but most Astro cameras can get away with about 6dB (factor of 2) with no obvious ill effects. If you happen to find the document, search for 'Conversion Gain' inside it.

cheers,

Robin
Post Reply