Live Stack with Flat Frame Bug?

A place to report problems and bugs in SharpCap
Forum rules


If you have a problem or question, please check the FAQ to see if it already has an answer : https://www.sharpcap.co.uk/sharpcap-faqs

Please also read about Troubleshooting USB Issues before posting.

*** Please do not post license keys - please report any problems with licensing to 'admin' by private message ***

Please include the following details in any bug report:

* Version of SharpCap
* Camera and other hardware being user
* Operating system version
* Contents of the SharpCap log after the problem has occurred.
[If SharpCap crashes, please send the bug report when prompted instead of including the log]
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Live Stack with Flat Frame Bug?

#11

Post by admin »

Hi there,

sorry to hear that it still isn't quite working. I would be very interested in seeing the 2 flats to compare with each other i.e. the one taken in SharpCap that didn't work so well and the one taken in the other software that did work.

Cheers, Robin
BlackWikkett
Posts: 387
Joined: Sat Jun 23, 2018 8:48 pm
Contact:

Re: Live Stack with Flat Frame Bug?

#12

Post by BlackWikkett »

Hi Robin,

Here's link the the raw files and the master flat: https://drive.google.com/drive/folders/ ... sp=sharing

ImageMasterFlat_ISO0 by Black Wikkett, on Flickr
This is the master flat created with ASICAP and Astrotoaster, this flat gave positive results.

I'm at the scope now waiting for clouds to pass. I've created several different test master flat via ASICAP and SC. I'll be testing these tonight if skies clear and post results.

-Wikkett
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Live Stack with Flat Frame Bug?

#13

Post by admin »

Hi,

one thing I can see is that the fits file that you are using is your new flat frame is actually a raw image, so if you look at it close up you can see the Bayer pattern in the image. That's not necessarily a bad thing but it is little unusual and using a raw image as a flat frame will tend to automatically reset your colour balance to neutral since it corrects the different colour pixels differently.

The original SharpCap flat frame that you shared has got slightly stronger decrease in brightness towards the corners than the new frame your shared. By my measurements the SharpCap frame goes from a maximum of about 37,000 down to a minimum of about 29,000 with a bias level of 2% which is about 1300. That means that the brightness increase from the edge to the corner is about 28%. In the low resolution copy of your latest flat frame that brightness difference looks to be about 23% which explains why the SharpCap flat would over correct slightly in the corners which is probably then enhanced by strong stretch. What we don't understand yet of course is why the SharpCap flat frame has that increase difference in levels.

Cheers, Robin
BlackWikkett
Posts: 387
Joined: Sat Jun 23, 2018 8:48 pm
Contact:

Re: Live Stack with Flat Frame Bug?

#14

Post by BlackWikkett »

Hi Robin,

Here are the latest images of M45 testing SC and ASICP/AT and SC flats. I've also included CCDInspector Flat Analysis images. Also a correction; I captured RAW16 color images in ASICAP to create the master flat .tif but seems that AstroToaster generates a mono flat master. Created new flats in SC via tool and selected no mono and no bias these were also mono for the final master flat that SharpCap created.`

ImageASICAP-AT-MasterFlat-TIF by Black Wikkett, on Flickr
ASICAP / AT Flat (debayer grid much exaggerated from compression)

ImageSharpCap-Color-MasterFlat-FITS by Black Wikkett, on Flickr
SharpCap Flat (debayer grid much exaggerated from compression)

ImageM45 Stack_28frames_840s_WithDisplayStretch by Black Wikkett, on Flickr
Using SC Flat vignetting almost gone, some gradient but getting closer

ImageM45 Stack_24frames_720s_WithDisplayStretch by Black Wikkett, on Flickr
Using ASICAP / AT Flat vignetting and gradient almost not detectable, getting really close with this (after meridian flip)

ImageStack_44frames_1320s_WithDisplayStretch by Black Wikkett, on Flickr
Using SC Flat vignetting and gradient much better, took it a little easier on the stretching tried not to clip so much black point as in previous image

Some conclusions after the latest testing:
1. ASI294 is very color sensitive
2. using smart histogram prior to live stack apply dark and flat master and change WB to get the histogram peeks to line up close as possible
3. any changes to dark, flat, offset, gain will require WB tweaking for the ASI294, easily fixable if post processing but it's must if you want a flat image in Live Stack
4. Live Stack and display histogram black and mid point and to lesser degree white point will require a lot of tweaking to get the result you're looking for. There's a fine line to keep the gradients and remaining vignetting suppressed while teasing out nebulosity and maintaining good star color. Nice addition to SharpCap may be more granular color adjustment control in the Live Stack Sliders. Could implement holding "shift key" while adjusting color half of normal step. Maybe an intensity slider in settings much like the stretch intensity currently implemented.

Moving forward:
Next round of testing I'll create new flats in SharpCap using BIAS, and MONO Flat. Then apply my lessons learned. I'll report back soon as I have a result.
Now a request to anyone with one shot color camera using Live Stack with darks and flats what are your results. If we could get some other users trying different camera's and posting results. Is this an ASI294 isolated issue or more common in all OSC cameras.

Thanks Wikkett
donboy
Posts: 256
Joined: Sat Mar 25, 2017 1:57 am

Re: Live Stack with Flat Frame Bug?

#15

Post by donboy »

I believe this is more of a 294 issue. I have not experienced these issues with a ASI071MC Pro or a ASI1600MC Cool. I have had similar issues with my first 294 and it was replaced under warranty. The current 294 seems to be much better and I see very little gradient when using an Idas D1. However, when using an Astronomik UHC + IR combination I have a red gradient on the right side of my images depending on how much I stretch it. I also am working on finding the best Dark & Flat processes that reduce this gradient.

ZWO found, while testing my original 294, that inserting a red filter in the optical chain does yield color gradients. I suspect that if I use an Idas P2 instead of a Idas D1 that there will be a more obvious red gradient, since the P2 lets in more red. What is similar with my original 294 and this current one is that the red gradient appears on the same right side. With Chris's gradient it appears red on both sides.

Also I've found that the more vignetting the greater the red gradient. With my original 294 I had more vignetting when using my 127mm refractor with .7x FR/FF than I did with my 8" RC with a .67x FR and thus there was a larger area of the red gradient on the right. In my current setup there is still a greater amount of vignetting on the right side of the 127mm and thus I see the red gradient when I use the UHC filter.

I suspect also that the driver amplification for the three color channels is not linear. What I mean is that a decrease or increase in exposure and or gain should cause a uniform decrease or increase in each color channel. What I've seen on all ASI294's is that a change in exposure or gain will cause a non equal response for the three color channels, especially the red channel. So every time one makes even a slight exposure or gain change it then requires an adjustment in the white balance. This was not the case with my 1600 or 071 cameras, they required very little to no adjustment when changing the exposure or gain.

Don
Last edited by donboy on Fri Nov 30, 2018 8:54 pm, edited 1 time in total.
donboy
Posts: 256
Joined: Sat Mar 25, 2017 1:57 am

Re: Live Stack with Flat Frame Bug?

#16

Post by donboy »

Filter UHC+IR using the 127mm @f/5.3 as seen on SC 75% of full resolution


NGC281_00_06_16_83frames_1245s_WithDisplayStretch.jpg
NGC281_00_06_16_83frames_1245s_WithDisplayStretch.jpg (245.54 KiB) Viewed 3626 times
Attachments
NGC281_23_02_35_161frames_2415s_WithDisplayStretch.CameraSettings.txt
(852 Bytes) Downloaded 141 times
Last edited by donboy on Fri Nov 30, 2018 8:51 pm, edited 5 times in total.
donboy
Posts: 256
Joined: Sat Mar 25, 2017 1:57 am

Re: Live Stack with Flat Frame Bug?

#17

Post by donboy »

Idas LPS D1 using the 127mm @ f/5.3 as seen on SC - full resolution


attachment=1]NGC281_23_02_35_161frames_2415s_WithDisplayStretch.CameraSettings.txt[/attachment]
NGC281_23_02_35_161frames_2415s_WithDisplayStretch.jpg
NGC281_23_02_35_161frames_2415s_WithDisplayStretch.jpg (745.75 KiB) Viewed 3626 times
Attachments
NGC281_23_02_35_161frames_2415s_WithDisplayStretch.CameraSettings.txt
(852 Bytes) Downloaded 143 times
BlackWikkett
Posts: 387
Joined: Sat Jun 23, 2018 8:48 pm
Contact:

Re: Live Stack with Flat Frame Bug?

#18

Post by BlackWikkett »

After completing the above M45 testing I continued to try some other objects applying the lessons learned. I've posted in the gallery here:

viewtopic.php?f=16&t=1296

Each object is saved exactly as seen from live stack. The second image is processed in Photoshop, the originals live stack the vignetting is still evident. I'll be be doing more testing on the next clear night.

Thanks for your posts Don!

-Wikkett
BlackWikkett
Posts: 387
Joined: Sat Jun 23, 2018 8:48 pm
Contact:

Re: Live Stack with Flat Frame Bug?

#19

Post by BlackWikkett »

Hi,

Here's the latest round of testing from 12-2-18. Below are the images along with an image of the flat used with each test image image. All flats were captured with the SC flats tool and saved as .fits. The flat images below were converted to .tif to allow posting to flickr.com so I could link to them here. I've also included some images with no calibration frames as reference. Also one without the reducer.

Current take away is color flats seem to work better than the mono flats created in SC. Next test is capture raw frames and single flats and dark and stack in DSS. I'll post these new images once the test is complete.

Key to image names:
AWB = Automatic White Balance SC Red / Blue sliders (usually results in red 99 and blue 59 or 60)
5050WB = SC Red / Blue WB sliders at 50
COL = color flat
BIAS = Bias used in SC flat tool
MONO = Mono used in SC flat tool
Camera settings file attached to each image on flickr just follow the link of the image to see it.

As you can see from the images below the darker color flats seem to work better than using the mono brighter flats. This makes no sense to me as it's my understanding that a brighter flat should have more effect on the image thus removing more vignetting, dust and applying overall flattening to the image. Hope these help narrow the problem if there is one in SC.

Thanks for taking the time to have a look!
-Wikkett

ImageM45 No Reducer No Flat Stack_30frames_1350s_WithDisplayStretch by Black Wikkett, on Flickr

ImageM45 No Dark AWB COL Flat Stack_4frames_120s_WithDisplayStretch by Black Wikkett, on Flickr

ImageM45 No Dark No Flat Stack_3frames_90s_WithDisplayStretch by Black Wikkett, on Flickr

ImageM45 AWB COLOR FLAT Stack_13frames_390s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-WB-COL 20_20_03 by Black Wikkett, on Flickr

ImageM45 AWB MONO FLAT Stack_4frames_120s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-WB-MONO 20_17_28 by Black Wikkett, on Flickr

ImageM45 AWB BIAS MONO Flat Stack_5frames_150s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-WB-BIAS-MONO 20_14_35_offset=2.997% by Black Wikkett, on Flickr

ImageM45 AWB BIAS COL Flat Stack_8frames_240s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-WB-BIAS-COL 20_39_22_offset=3.003% by Black Wikkett, on Flickr

ImageM45 5050WB MONO Flat Stack_8frames_240s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-05050WB-MONO 20_28_45 by Black Wikkett, on Flickr

ImageM45 5050WB COL Flat Stack_15frames_450s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-05050WB-COL 20_32_07 by Black Wikkett, on Flickr

ImageM45 5050WB BIAS MONO Flat Stack_7frames_210s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-05050WB-BIAS-MONO 20_26_09_offset=2.289% by Black Wikkett, on Flickr

ImageM45 5050WB BIAS COL Flat Stack_19frames_570s_WithDisplayStretch by Black Wikkett, on Flickr

ImageFLAT-167ms-300g-20offs-05050WB-BIAS-COL 20_35_49_offset=2.288% by Black Wikkett, on Flickr

Imagedark_40_frames_-15.4C_2018-11-13T04_21_13 by Black Wikkett, on Flickr
Dark used on all above images that used dark
BlackWikkett
Posts: 387
Joined: Sat Jun 23, 2018 8:48 pm
Contact:

Re: Live Stack with Flat Frame Bug?

#20

Post by BlackWikkett »

Hi,

Finally been able to get back out there and complete some more Live Stack Flats testing.

The goal of this test is determine what effect using 0 offset / brightness and color or mono flats has in SC Live Stack. Additionally capture raw frames for darks, flats and lights then stack in DSS and process in Photoshop.

Equipment (no Changes, except new CEM60 mount no effect on testing) RC6, ZWO ASI294 PRO, Astro-Physics CCDT67 Reducer, IDAS D1 LPS filter.

You can follow the link of any image to get camera / capture settings but in general 30 second capture, 300 gain, 10 offset for lights, darks, specified flat, cooler set to -15c

Conclusion as of now for using the ASI294 with flats in SharpCap Live Stacking. Use color 5050 White Balance 0 offset flats. I've been using 300 gain same setting as lights and exposure time of 205 - 209 ms. This pushes the histogram to the right edge just before clipping. Applying these flats to Live Stacking removes almost all dust bunnies and does not enhance vignetting but removes it and flattens the image.

I still think there may be something going on how SC applies flats since the mono flats are still generating enhanced vignetting. If anyone has some Live Stacking color images using flats that were generated with a OSC other than the ASI294. I have other OSC camera but doesn't generate vignetting so it's of no help for comparison.

Below are the test images of the old favorite M45. I left the sensor dirty to determine if flats were removing dust.

ImageM45 No Flat No Dark Stack_21frames_630s_WithDisplayStretch by Black Wikkett, on Flickr
Live stack no calibration frames

ImageM45_2018_12_17_5050ColFlatDark by Black Wikkett, on Flickr
Raw light, dark and flat frames stacked in DSS processed in Photoshop

ImageM45 5050WB 0 Offset Color Flat Stack_46frames_1380s_WithDisplayStretch by Black Wikkett, on Flickr
Longer Live Stack using flats that give best result as described above

ImageM45 AWB 0 Offset MONO Flat Stack_13frames_390s_WithDisplayStretch by Black Wikkett, on Flickr
AWB 0 offset mono flat, enhances vignetting but does remove most dust

ImageM45 5050WB 0 Offset Color Flat Stack_16frames_480s_WithDisplayStretch by Black Wikkett, on Flickr
Shorter run with working flats technique to provide a more fair comparison to non working flat

I'm willing to complete more testing if there's something I've missed but the process of creating flats and applying them in SC is very simple. I'll ask again as in the initial post am I expecting too much? Is there something wrong in my process? I hope we get to the bottom of this issue but in the mean time there is a solution that is generating acceptable, for me, results

Thanks
-Wikkett
Post Reply