Many thanks everyone!
It's amazing how quickly you can get such helpful answers on this forum and such a lot to think about. I would never have thought to find or look at the sample darks supplied by ZWO that Menno points out. That clearly settles one of my questions then - no I do not need to send my camera back because it behaves pretty much as expected - and maybe more or less at the limits of where CMOS technology is?
I also had the same question as you Don about using PNG darks - I moved over to FIT simply because it seemed sensible to use the same format as for the lights that the dark is subtracted from and didn't really investigate further. I think that to examine the darks critically they do need to be stretched to the point that you can see the hot pixels and amp glow - because - at the limits - lights will be similarly stretched?
Thanks Menno. It makes good sense to stack up more frames into the master dark to try and average things out more. My 10X and 20X masterdarks were inadequate. The same must in fact also apply to the lights - perhaps the more the better? So is that another argument to stack more shorter lights - or do longer exposures average themselves out more anyway?
If, as Robin suggests, the pulse width modulation control of the Peltier cooling is the underlying reason that banding increases with cooling then I wonder whether this interference occurs during exposure or just during the time frame that the CMOS frame is read out? If the latter then that is an argument for longer exposures and fewer read outs? I also wonder whether using a really big power supply with low internal resistance and strapping a big millifarad capacitor across it would smooth out the effect - if indeed it is mediated via slight voltage drops down the powerline?
Although this thread has rather mutated from a question about how to use the Brain to set camera parameters in an optimal way into a detailed discussion about the AS1294 Pro I hope that there is still some relevance to the original topic? i.e. if the banding is indeed inherent to these and other popular CMOS cameras and is exacerbated at low gain and with cooling then maybe future versions of the Brain could suggest or parameterize mitigations (e.g maybe the banding suppression tool which -for me at least - seemed effective at a threshold of only 10%)?
best wishes
Tim
Questions about Brain and when and when not to use it?
Forum rules
If you have a problem or question, please check the FAQ to see if it already has an answer : https://www.sharpcap.co.uk/sharpcap-faqs
If you have a problem or question, please check the FAQ to see if it already has an answer : https://www.sharpcap.co.uk/sharpcap-faqs
Re: Questions about Brain and when and when not to use it?
Tim
Also from investigation & experience, exposure is king (will vary depending on object). Poor data requires excruciating post processing. This results in excessive stretching which washes out star colour and increases noise. With https://www.astrobin.com/full/1graw1/0/ I think some forum users would be quite surprised at how little processing was required after the calibration frames had been applied. That capture was a year ago - I could probably improve on that.
Consider this - 60x30s v 30x60s - same integration time - which is 'best'. Answer - try it with a few objects and see what the results are. M27 and M31 are good candidates for this type of evaluation. With a target like M33, 60s won't get you much particularly at B6 or higher.
Some thoughts:
Logical I would say. Not using the same format introduces unneeded complexity, especially when troubleshooting.it seemed sensible to use the same format as for the lights that the dark is subtracted from
A systematic demo of stacking here viewtopic.php?f=18&t=1808, draw your own conclusions.My 10X and 20X masterdarks were inadequate. The same must in fact also apply to the lights - perhaps the more the better
No simple answer to that but some things to consider:So is that another argument to stack more shorter lights - or do longer exposures average themselves out more anyway?
- brightness of sky background
- brightness of object
- Bortle number
- camera capability
Also from investigation & experience, exposure is king (will vary depending on object). Poor data requires excruciating post processing. This results in excessive stretching which washes out star colour and increases noise. With https://www.astrobin.com/full/1graw1/0/ I think some forum users would be quite surprised at how little processing was required after the calibration frames had been applied. That capture was a year ago - I could probably improve on that.
Consider this - 60x30s v 30x60s - same integration time - which is 'best'. Answer - try it with a few objects and see what the results are. M27 and M31 are good candidates for this type of evaluation. With a target like M33, 60s won't get you much particularly at B6 or higher.
Some thoughts:
- Understand & trust in your histogram.
- Don't be afraid to experiment (systematically).
- Think of any information in books/forums as 'guidelines' rather than 'rules'.
- For any object, have a look at https://www.astrobin.com/ under Explore and Image of The Day or Top Picks. Study the acquisition settings that skilled imagers use.
Re: Questions about Brain and when and when not to use it?
How about if you are only livestacking within SharpCap for EAA viewing and then saving any images using "save with adjustments," which saves the image as a png file? With 40 frame darks, the fits masters are over twice the size of the png master darks. Does this provide any increased capability within the SharpCap livestacking process?
I am in the process of re-doing my dark library with fits files, but I don't know if I will gain/lose anything by doing that compared to the png files I have now.
Don
Re: Questions about Brain and when and when not to use it?
Don
Dave
I would do a controlled test with FITS & PNG and scrutinise the results before committing time & effort to building the dark library.I don't know if I will gain/lose anything by doing that compared to the png files I have now
Dave
Re: Questions about Brain and when and when not to use it?
That is a nice set of data Dave. thanks for the links
The 60s stack data are interesting to look at. What conclusions do I draw?
The colours get deeper and the signal to noise improves with exposure (I guess as you would expect from 1 to 100 frames by a factor of 10). With all of the stacks stretched in exactly the same way there is also the appearance of diminishing contrast between the bright stars and the rest after about 25 to 50 frames. So on the face of it and aesthetically I might pick the 25 frame picture if a vista of M27 set in a less crowded overall star field were my object but the 100 frame picture to get everything big and bright- but, in reality, would anyway take the 100 frame picture and then post process according to taste.
On the question of banding though - there is relatively little anyway in your pictures (and it is a different camera?) but it is perhaps just about visible at 1-5 frames and submerged in the signal beyond that.
You must be quite right that dark skies are king --- but for me the motivation to move from visual to electronic astronomy was the realisation that, at home at least, I am stuck with Bortle 6-7 skies. I never cease to be amazed at what you can see electronically even under such difficult conditions. However I probably now am beginning to better understand where poor skies begin to limit things - with, as you say, excessive stretching, poor colour and increased noise. On that note here is a chamber of horrors example of the heart nebula which started me down the track of trying to work out where the banding was coming from. Not the kind of thing that one posts with much pride
. 74 x 45s exposures, F 5.6, UHC filter, - 5C at near unity gain.
https://onedrive.live.com/?cid=330584F8 ... 40&o=OneUp
I think that the truth is the conditions were much worse than I had realized at the time. When you use the Sharpcap brightness filter to collect a stack I think that it is relative rather than absolute - so if the first frame is dim it sets a low standard. I have much better photographs using similar camera parameters of different objects taken at different times and no doubt under better sky conditions -- but having seen the banding I got interested in where it was coming from and how to minimise it and hence this thread.
There is so much to learn!
thanks
Tim
The 60s stack data are interesting to look at. What conclusions do I draw?
The colours get deeper and the signal to noise improves with exposure (I guess as you would expect from 1 to 100 frames by a factor of 10). With all of the stacks stretched in exactly the same way there is also the appearance of diminishing contrast between the bright stars and the rest after about 25 to 50 frames. So on the face of it and aesthetically I might pick the 25 frame picture if a vista of M27 set in a less crowded overall star field were my object but the 100 frame picture to get everything big and bright- but, in reality, would anyway take the 100 frame picture and then post process according to taste.
On the question of banding though - there is relatively little anyway in your pictures (and it is a different camera?) but it is perhaps just about visible at 1-5 frames and submerged in the signal beyond that.
You must be quite right that dark skies are king --- but for me the motivation to move from visual to electronic astronomy was the realisation that, at home at least, I am stuck with Bortle 6-7 skies. I never cease to be amazed at what you can see electronically even under such difficult conditions. However I probably now am beginning to better understand where poor skies begin to limit things - with, as you say, excessive stretching, poor colour and increased noise. On that note here is a chamber of horrors example of the heart nebula which started me down the track of trying to work out where the banding was coming from. Not the kind of thing that one posts with much pride
. 74 x 45s exposures, F 5.6, UHC filter, - 5C at near unity gain.https://onedrive.live.com/?cid=330584F8 ... 40&o=OneUp
I think that the truth is the conditions were much worse than I had realized at the time. When you use the Sharpcap brightness filter to collect a stack I think that it is relative rather than absolute - so if the first frame is dim it sets a low standard. I have much better photographs using similar camera parameters of different objects taken at different times and no doubt under better sky conditions -- but having seen the banding I got interested in where it was coming from and how to minimise it and hence this thread.
There is so much to learn!
thanks
Tim
Re: Questions about Brain and when and when not to use it?
Thanks Tim.
These are the capture details:
The data is a set of 100 lights (60s exposure, gain 200, black_level (offset) 30), 60 darks, 100 bias, 100 flats taken with an Altair 183C + Vixen 81s refractor.
With that particular camera, the Altair 183C, it has taken me a while, probably best part of 2 years to get satisfactory images (imaging learning curve rather than camera issues). I have settled on ~100 bias, ~100 flats and ~50 darks. Those numbers seem to give consistent results. My process (for a CMOS camera) is:
I follow the above rigorously. I can remember the offset settings for the gains so the above process is quick. My emphasis is on getting a good histogram for a light.
The black_level (offset) setting being wrong impacts the bias frames, which in turn impacts all the other types of frames. I place emphasis on keeping gain and black_level the same throughout (bias/dark/flat/lights frames), adjusting only exposure.
I now calibrate/process using Astro Pixel Processor and always generate a Bad Pixel Map which seems to help with junk removal.
I am not using filters so the gain/exposure might need adjustment (upwards) to compensate for filters.
I did have some banding issues with an Altair 183M + Ha filter, using a gain of 1000 and black_level 150. This was back in Sept. 2018. I have since realised that user error was the problem - gain is no substitute for exposure. With narrowband imaging exposures of 300s+ are likely to give better results. Of course, exposure gets limited by brightness of the sky background. That drives me down the guiding route, which I am wrestling with at the moment.
The journey continues ......
Dave
These are the capture details:
The data is a set of 100 lights (60s exposure, gain 200, black_level (offset) 30), 60 darks, 100 bias, 100 flats taken with an Altair 183C + Vixen 81s refractor.
With that particular camera, the Altair 183C, it has taken me a while, probably best part of 2 years to get satisfactory images (imaging learning curve rather than camera issues). I have settled on ~100 bias, ~100 flats and ~50 darks. Those numbers seem to give consistent results. My process (for a CMOS camera) is:
- Decide on the gain, gain for that camera is in the range 100 - 4999. For brighter objects such as M42, M27 I use a gain of 200 (4%), for dimmer objects such as M45 I use 400 (8%). Unity gain for that camera is 400.
- For that gain, establish the lowest black_level/offset/brightness (Altair/QHY/ZWO) that results in a bias frame such that the image histogram is clear of the left and right axes. Invariably this results in a centralised histogram.
- Take single exposures until a satisfactory histogram is obtained.
- Capture the lights.
- Capture the flats. I leave gain and black_level as for the lights and adjust exposure down until I have a a fairly centralised histogram (around 30,000 mean ADU in FITS Liberator).
- Capture bias frames using lowest exposure available to the camera. Gain and black_level as for lights.
- Capture the darks using same exposure as lights. Gain and black_level as for lights.
I follow the above rigorously. I can remember the offset settings for the gains so the above process is quick. My emphasis is on getting a good histogram for a light.
The black_level (offset) setting being wrong impacts the bias frames, which in turn impacts all the other types of frames. I place emphasis on keeping gain and black_level the same throughout (bias/dark/flat/lights frames), adjusting only exposure.
I now calibrate/process using Astro Pixel Processor and always generate a Bad Pixel Map which seems to help with junk removal.
- Altair-183C_Bad-Pixel-Map-from-APP.PNG (41.55 KiB) Viewed 1704 times
I am not using filters so the gain/exposure might need adjustment (upwards) to compensate for filters.
I did have some banding issues with an Altair 183M + Ha filter, using a gain of 1000 and black_level 150. This was back in Sept. 2018. I have since realised that user error was the problem - gain is no substitute for exposure. With narrowband imaging exposures of 300s+ are likely to give better results. Of course, exposure gets limited by brightness of the sky background. That drives me down the guiding route, which I am wrestling with at the moment.
- NGC7000-60x90s-g1000-bl150-Ha-filter-showing-banding.PNG (278.37 KiB) Viewed 1704 times
The journey continues ......
Dave
Re: Questions about Brain and when and when not to use it?
Thanks Dave. I see Robin has already answered this question at least a couple of times- here: viewtopic.php?t=2302#p11877 and here: viewtopic.php?f=19&t=2383#p12328.
My confusion was caused by what looked like a big difference in viewing a fits flat image in Fits Liberator vs viewing a png file in an photo app. Now I see that using Gimp to open the fits flat image shows the same image as does a png file. Guess I'll have to look at Fits Liberator further to see why it presents the same image differently.
Don
Re: Questions about Brain and when and when not to use it?
Don
In the 2nd link, the FITS Liberator shows 'flip image' checked. I think that is the default. Maybe the other software is not flipping or you have flip unchecked?
Maybe post a side-by-side comparison to help work out what is a happening.
Dave
One of the unintended side effects of forums - a good place to hide information. If the right question isn't asked the answer isn't found.Robin has already answered this question at least a couple of times
In the 2nd link, the FITS Liberator shows 'flip image' checked. I think that is the default. Maybe the other software is not flipping or you have flip unchecked?
Maybe post a side-by-side comparison to help work out what is a happening.
Dave
Re: Questions about Brain and when and when not to use it?
Yes, at least in my case with my setup and Bortle 8. Shorter exposures, so less light pollution creep. And then more exposures. So no 10 x 240 seconds but 40 x 120 seconds.timh wrote: ↑Sun Aug 16, 2020 11:12 am Many thanks everyone!
Thanks Menno. It makes good sense to stack up more frames into the master dark to try and average things out more. My 10X and 20X masterdarks were inadequate. The same must in fact also apply to the lights - perhaps the more the better? So is that another argument to stack more shorter lights - or do longer exposures average themselves out more anyway?
best wishes
Tim
More darks did take care of the horizontal banding. I experimented with 100 darks (bit much
) and works great. For me I did find that 40 to 70 darks (depending on how much you have to stretch) really makes the horizontal banding disappear.Menno
Re: Questions about Brain and when and when not to use it?
No, it was something even more basic. The FITS Liberator preview automatically applies a certain stretch, which GIMP does not. So, the GIMP preview shows a totally black screen. Stretching it just appeared to brighten the background to gray without showing any white pixels. However, by continued stretching and adjusting the black level, I was finally able to view the white pixels. It didn't exactly match the FITS Liberator image regarding the relative size/brightness of certain pixels, but I'm not so concerned about that. That's probably just a matter of more tweaking. I was concerned about the totally black (GIMP) vs pixelated (FITS Liberator) radically different images, and that has been solved.
Thanks again,
Don