Problems with flats

[admin]

Hi Michael,

OK, I just tested with your master flat frame against the raw frames that you previously sent, and it worked fine in the folder monitor camera. Then I went and re-checked the code around the warning that you are seeing and realised that in actual fact the warning itself is spurious - by a mistake of copy+paste when editing the code, the warning ended up in the part of the code where the flat *IS* compatible with the camera frame, and will be used. So, the fact that you were seeing the warning indicates that the flat is being used.

That unfortunately leaves us with the issue that you weren't seeing the flat working when testing, but at least we are no longer chasing the wrong cause.

My tests with the supplied files actually gave a slight over-correction of the dust donuts, so not perfect, but definitely the flat was being applied. I have also just tested with a Player One camera (different model) in RAW16 mode and created and used a new flat and it is working, so I can't see anything currently wrong with the flat code except the spurous warning.

I would say that the next time you have a chance, turn on a strong display stretch of individual frames (not live stacking), and try enable/disable of the flat correction to see if you can see the effect in the individual camera frames.

cheers,

Robin

[MikeB838]

Hi Robin,

good to hear and thank you very much for your work!

Last night I still got the warning (as you explained it is okay ), but the donuts were not so obvious as in the night before, strange.
I mostly only adjust the live stacking histogram (not the display stretch histogram) and in the night before the donuts were visible in the live stack always (more or less) no matter how I adjust the histogram slider. Last night there were only some extreme slider positions, which brought out
the donuts. Strange

I will try to experiment as you said.

Can the light source be the reason for my problems? I use a pdf with a white page on an tablet as my light source for my flats.
I checked "use monochrome" and the 4 histogram curves are side by side but in the middle around 40% to 60% after the "Auto adjust".

On another topic I thought I had found a bug (SharpSolver giving a different orientation (6° E of N) as PlateSolver3 (354° E of N)), but Astap
reports as SharpSolver and Astronomy.net gives a third variant (174° E of N). So I trashed my post and I'm happy with the PlatesSolver3 values, which
are the values Astrometrica needs.
Everything not as easy as first thought

Thanks and cheers,
Michael

[MikeB838]

Hi Robin,

I made some tests:

1) no live stacking: I stretched a lot, took a snapshot, no donuts
2) switched to live stacking (without changing sth, but activating display stretch again): very visible donuts in each single frame and in the result stack.

I use the same dark and flat for 1) and 2).

The same result if the flat is monochrome or colour.

For me it looks like a problem in the code for live stacking, handling the corrections differently to when taking a snapshot.

Do you want some files?

Thanks and cheers

Michael

BTW: I log into the Forum, answer the last post and when I click "Preview" for the first time, I'm logged off and have to log in again.
I have to write the text again (now I always save my text before doing so). Is this a known problem?

[admin]

Hi Michael,

first, on the forums, as long as you ticked the 'remember me' button when you logged in, then you should stay logged in. In theory there is no auto logout. However, I have noticed that you do sometimes get logged out (maybe once every few months or so), so maybe that happened. There is also a timeout on writing a message, but that shouldn't take you to the login screen, just refuse to post.

If you do end up with a message that fails to post, you should be able to use the 'back' button in the browser to go back to the message - the browser will remember and restore the text you had written before trying to submit/preview the message (at least it always seems to for me). That way you can copy/paste the text to somewhere else if necessary.

Right, on to flat frames...

There *is* a difference between the processing for live stacking and the normal processing, so it's plausible that there could be an issue that only shows up in live stacking. Using the files that you previously sent me (5 raw frames, the master dark and the master flat), I just tested with the folder monitor camera and it all seemed to work as expected. In particular if I run the same frames through the live stack with the flat frame , then again without the flat frame, I can clearly see the difference in the results.

Were you testing and able to repro this with the folder monitor camera? If so, can you check that you have the very latest version of SharpCap from our downloads page please? There was an issue similar to this with flat frames that only affected the folder monitor camera. It's fixed in 4.1.13343.

cheers,

Robin

[MikeB838]

Hi Robin,

thanks for your answers!

I always use the lastest version and now work with 4.1.13393.

I compare the saved stack during live stack with the saved stack with folder monitor and the same frames. They look the same. Two donuts are
visible (arrows).

Live_stack_with_flat.jpg (328.13 KiB) Viewed 3868 times

Foldermonitor_with_flat_Stack.jpg (411.59 KiB) Viewed 3868 times

Using no flat with the folder monitor shows many donuts (same settings as in folder monitor with flat)

Foldermonitor_without_flat_Stack.jpg (372.77 KiB) Viewed 3868 times

For a check I tried to add the files in Deepskystacker (DSS) and the results are similar. The 2 donuts are visible.
But DSS gives me the warning, that the flat has a different gain (22 and not 200 as the other frames).
Shouldn't the flat have the same gain as the frames/dark?

So it seems that I should avoid "Auto Adjust Gain/Exposure" and instead only adjust my exposure time and perhaps the brightness of my light source!?

On the forum:
it happens to me nearly each time (not today) I try to answer a post.
I do not stay logged in but log in, when I want to answer a post.
I discovered that the back button brings back my text only at the second time it happened to me
I can live with it.

Thanks and cheers
Michael

[admin]

Hi Michael,

OK, sounds like we are starting to understand this one...

Using a flat with different gain should be fine, providing you stick with the rules for flats for deep sky imaging - that is

* The flat must be paired with either a dark flat (or, if short exposure flats in use, a bias frame). The dark flat/bias should be taken with the same gain/offset/colour balance settings as the flat frame. This allows the subtraction of the bias/dark flat from the flat to leave behind only the component of the flat frame image that comes from light falling on the sensor - removing any contribution from the camera pedestal black level or thermal noise. Obviously in real life this is multiple flat frames and dark flat/bias to make a master flat. SharpCap flat capture will do this for you if you choose the options for subtract bias or subtract dark flat

* The light frames must be paired with dark frames that are taken at the same exposure/gain/offset/colour balance settings as the light frames. Again this allows the subtraction to leave behind only the portion of the image data caused by light falling on the sensor, and the flat correction is applied to this (not to the whole image data).

These are covered in more detail in the SharpCap documentation - https://docs.sharpcap.co.uk/4.1/#Gettin ... Correction

Failing to do one or both of these can give over or under correction.

SharpCap will let you get away with ignoring these and will do its best (after all, sometimes you don't have time to do things properly, or just can't be bothered). In particular, if you create a master flat in SharpCap and use dark/bias subtraction, the filename gets a % value stuck on the end. This is the image background level that SharpCap calculated during the flat creation procedure. If you do not use a dark frame then this level is used as a guess of how much pedestal black level should be subtracted when applying flat correction, but that only works right if the gain/offset/colour balance are the same between the flat and light frames.

Hope this helps,

Robin

[MikeB838]

Hi Robin,

thanks a lot for taking your time to explain!

I will give it a try the next time the weather permits.

On my tests I tried it once (dark flats and bias flats), but with no good results. I will check it again.

Why is there no option to do the correction for both (dark and bias)? Would that not make sense?

Cheers,

Michael

[admin]

Hi Michael,

there's no need to use both dark and bias frames unless you are using dark frames that don't have the same exposure length as the light frames, which won't happen during flat capture in SharpCap when using dark flats.

cheers,

Robin

[MikeB838]

Hi Robin,

okay, good to know, thanks!

cheers,
Michael

[MikeB838]

Hi Robin,

now I had time and weather to do some tests.
I used version 4.1.13417.0. on two nights.

I took a dark (30s, gain 200) and I took 12 different flats:
monochrome and colour flat frame
with no offset correction and with bias and with dark subtraction
I used auto adjust/gain (gives me 1000ms and gain 25)
and I did not use auto adjust/gain (I changed the exposure time to 100ms and gain stayed at 200)

All flats work perfectly. I used different flats (not all) with live stacking the same area
and used all flats in the folder monitor (the next day) with perfect results. No dust donuts to see

But: during the night I had several stacks with severe donuts again.
I compared the settings:
- temperature 0.6°C lower,
- LiveStack.BrightnessLimit higher
- LiveStack.BlackLevel higher
- LiveStack.WBx.db different
for the bad stacks.

The next night I only took one flat (manual set to 100ms, gain 200 (as my dark and frames) with dark substraction.
I took a lot of stacks and none has a donut.
I compared again and the settings were as different as the night before. But this time no donuts.

So I have the feeling, as if something doing with SC (using tools or FX or else) does spawn SC to "generate" the donuts:
changes something, which lets the flat correction fail slightly.
I tried the second night to provoke that again, but had no success (what sounds good). All frames were without donuts.

I will see if I can find something the text time I use SC.

If you need some of the data, I will again be happy to provide it.

Sorry for the long text and thanks a lot!

Cheers,

Michael

PS: I see the Forum entry: viewtopic.php?t=8795
May be there is a connection, although I see a difference between colour and monochrome
flats. Switching between this flats I have to adjust the colour sliders a great deal, so it
looks for me that they are different.