Problems with flats

[admin]

Hi Michael,

interesting results - glad to know that you are getting good flat correction at least part of the time, but I can't help you (yet) explain why it is failing on other images...

Things that you do in the live stacking panel should have no effect on flat correction - the flat correction is done before frames are added to the stack, wherease things like the live stacking white balance happens as the data is copied out of the stack and processed for display. The only thing that is relevant there is that as you stretch the image more, things like donuts that were previously too faint to see may become visible. Other than that, it's the camera controls on the right that are most likely to affect flat correction if they are changed in the sort of ways we have previously discussed (offset, exposure, white balance, etc).

I can't see the 0.6 degree difference making any significant difference either - far too small for any sort of big effect...

It is worth checking the SharpCap log when you see the problem - Sometimes SharpCap cannot apply the flat if it is 'incompatible' with the light frame - normally that is reported when you select the flat frame, but if you select the flat first then change some camera setting that makes the flat incompatible, it would end up being reported in the log.

cheers,

Robin

[MikeB838]

Hi Robin,

thanks for your time and explanations!

I had not changed the camera controls (offset, exposure, balance).

In the log I do not see an error loading the flats.

Here are 2 logs:

logs.zip

(100.79 KiB) Downloaded 124 times

And the setting files of all the flats:

settings.zip

(10.43 KiB) Downloaded 118 times

For example: Target M57 looks good, 2378 (and many more) afterwards do not look good,
ngc6662 does look good again.

Closer inspections now show, that not so many stacke were as good as I thought at the telescope
(looking at the *.png).

After I took the 12 flats I made 5 stacks of M81 with the following flats and the results:

1. Monochrome, Manual, Dark subtraction: bad
2. Monochrome, Manual, None : bad
3. Monochrome, Autogain, Bias subtraction: good
4. Monochrome, Autogain, None : good
6. Colour, Manual, Bias subtraction: bad

(decoded in:

flatlist.zip

(446 Bytes) Downloaded 111 times

)

And: the camera setting files show for both dark and bias subtractions: UsedBias=True

Cheers
Michael

[admin]

Hi Michael,

there is nothing obviously wrong in those logs or settings files that I can see. Tricky to know what to suggest at this point... A couple of steps I would probably try myself are

* Once flat capture is complete, leave the flat illumination in place and auto-stretch the image - this is applying the flat correction to the flat frame illumination, so it should give a uniform image with no donuts or vignetting visible. If you turn the flat off at this point, the donuts will re-appear

* With the flat back enabled and the flat illumination still in place, decrease the exposure by a factor of 2 over the one used to generate the flat - do the auto stretch again - still should be uniform - this checks that the flat is correcting well at different brightness levels. Then try halving the exposure again. If the previous step was fine and this fails then the issue is with the way that the dark/bias levels are being accounted for in the flat correction

* When actually imaging with the flat, without live stacking, again stretch the image strongly, and check that the donuts at least reduce when the flat is enabled vs when not enabled - this checks that the flat is doing something and not being ignored for some odd reason.

cheers,

Robin

[MikeB838]

Hi Robin,

thanks for looking into it and your good suggestions, which I tried inside (now with SC 4.1.13447).
And good I made no obvious stupid mistakes.

1. as expected: no donuts or vignetting with flat applied / both are there without flat applied

2. I tried halving (or doubling) the exposure time for various start points (exposure and gain):

- For all variants with no substraction, I see donuts arise more and more for shorter times:
eg.: Start: 1200ms, gain 0: ok / 600ms, gain 0: faint donuts / 300ms, gain 0: more obvious donuts

- For all variants with bias or dark substraction I do not see donuts come into sight (even if I increase
the gain for slower times, to have the values in the middle of the histogram (and not at the left side)).

3. Not tested yet, but I expect to have the activated flat to have an effect (as I see it here)

I also tried to perform other tasks inside SC (some I do sometimes at night: eg. Stellar photometry Estimation),
but that did not provoke more donuts.

What makes me wonders: today it seems, that doing a flat with bias or dark substraction solves the puzzle, but as I said
in my last post: all the stacks (eg. for Asterois 2684) which have bad donuts, are made with a flat with bias substraction applied.

Thanks for your patience and help!

Cheers
Michael

[MikeB838]

Hi Robin,

I did some testing last night and for the third point: yes switching flats on and off has an effect (as expected).

I saw at my indoor tests that decreasing the exposure time too much (histogram peak below 10%) it will begin to show up donuts.
Makes sense for me: may be not enough headroom for corrections to be applied correctly.

Last night I had donuts again, especially with targets near the horizon.
I found out that in the histogram I use in Live stacking (where I thought I'm okay with the peak to be in the middle of the histogram)
I have "Log" selected and the peak is also below 10% (I was not aware of that). So I thought I had found the problem.

But today I checked with the folder monitor the older stacks (I sent you the logs from).
Asteroid 2684 has the histogram peak at 3% (with very visible donuts).
But M57 has the peak at lower values (0,7%) and only with extreme stretching a hint of donuts.

With my theory (not enough headroom for the calculations) M57 should be worse than 2684!?

So I'm still scratching my head

Cheers
Michael

[admin]

Hi Micheal,

another thing that I should mention to check is that the 'black level' (brightness/offset on some brands of camer) value is high enough that the histogram doesn't quite go all the way to the left when capturing dark or bias frames. If the peak is all the way at the left then that could affect the ability to correct properly. Another thing to note is that ideally you shouldn't move *anything* in the optical path between taking the flat and using it - if you rotate the camera on the telescope to change the image orientation, or even significantly change focus, those can affect flat correction.

I just tested with a camera that has a couple of smudges on the sensor - when I capture a flat using bias frames (flat captured at 500ms exposure), then I can turn the exposure down to about 5ms with the flat applied and still see that the flat is fixing the dark spots caused by the smudges on the sensor (I am turning up the display stretch as I reduce exposure to keep the displayed image on screen about the same brightness even as the camera image itself becomes very dim). Once I go below 5ms, I can't be sure if the flat is doing a good job or not, since the smudges are no longer easy to spot even with flat correction turned off - the image is just too noisy to see them properly. This is again a 'what can I do to check my flats before I use them in anger' step, but it might help.

cheers,

Robin

[MikeB838]

Hi Robin,

thanks for your thoughts!

I had adjusted the offset, so that the black level is away from the left side:

dark-Neptune-C II offset.jpg (35.11 KiB) Viewed 2444 times

After taking the flat I do not rotate the camera, but focus can be the point.
I will take care as I sometimes have to refocus, but I checked the logs (I posted here) and it looks for me like I did not do a refocus before the
donuts became visible (eg. in the time I take the flat and I see the donuts in the stack "2684"). "Only" I stopped and started SC again inbetween.

I will try to do this routine (checking the flat with the flat itself and different exposures) each time I take a flat to be sure it is okay and I will
take a new flat after I had to change focus.

Cheers and have a nice weekend
Michael

[admin]

Hi Michael,

normally small focus tweaks should be fine... but still worth keeping an eye on.

Seeing your offset graph above, that's a very big offset level. Now, in theory, all the dark subtraction/bias/etc should account for this, but it's the failure to properly deal with the offset that causes the flat correction not to work, so I would be tempted to set the offset so that you *just* clear the black level, rather than leaving a big margin like this.

cheers,

Robin

[MikeB838]

Hi Robin,

So I'm the exception here

I tested inside again, did a flat, did the correction with the flat without moving the light source.

Stretched a lot: everything was fine. Commanded my ZWO EAF (from inside SC) to change the focus
(The focuser is a Omegon 2" Newton Hybrid Crayford focuser, dual speed).

And donuts jump into view (with only small changes (eg. 5 steps, where a focus change can normally hardly be seen in the live view).
I was not able to remove the donuts by commanding old focus (or any value around).
I repeated it and even a slight touch to the camera brought the donuts back.

The whole construct is sturdy and I can see no play as I touch it.
The EAF is connected to the reduction gears of the focuser and a step should be in the range of 1/10 mm focus change (according to Omegon).

Okay that means to me: focus change = new flat

I had put the offset nearer to the left side and saw more donuts, so I had moved it to higher values.
I will check it again at night, thanks for the hint.

Cheers and thanks,

Michael

[admin]

Hi Michael,

that's odd - I would have expected the effect from focus change to be relatively gradual (ie small change in focus gives a small effect if any, then more effect from bigger changes). Exceptions to that would be if you have backlash compensation in the focuser settings (either SharpCap or in the ASCOM driver) and the movement triggered the compensation to make a bigger move.

In either case though, if the dust donuts are coming from something on the camera side of focuser, then the effect of moving focus should only be that the light cone (shadow cone) of the dust changes by the focus movement divided by the focal ratio. If on the other hand the donuts are coming from things on the other side of the focuser then any focuser sag change could have an effect.

You can work out how far out from the sensor the dust shadow comes from - measure the number of pixels across the donut, divide by 2, multiply by the pixel size and then multiply by the focal ratio - that should give you the distance from the sensor to the thing creating the shadow. Might help understand what is going on.

cheers,

Robin