Stacking the stacks

[Nyx]

If the clouds ever disappear, I'd like to try something and wanted to know what others think.

I like to keep my subs short (10-15s). I also like to stack my subs using AstroPixelProcessor (APP) in order to take advantage of things like Bayer-Drizzle. What I don't like is having to stack thousands of subs. I'm also a fan of working with "good subs" only and not stacking everything.

My idea is to use live stacking to create 5-minute flat+dark calibrated stacks, only using the best subs (FWHM/Brightness filtered) as well as dithering between the stacks, which I can later feed to APP for further stacking and processing.

Has anyone tried anything similar before and if yes, how good did it work? Would the "default" stacking algorithm be better than sigma clipping, considering the fact that APP will take care of rejecting the outliers later on?

Thanks in advance and clear skies!

[celkins]

Programs such as APP & StarTools only really work as intended when you feed them raw, linear subs to work on: if they have been pre-processed in some way, then the assumptions made in the stacking software don’t hold true, and so the results will be sub-optimal - how much so is very hard to say…

If you want to use very short subs, then you have to accept that, when imaging deep sky objects, you have no choice but collect a large number of them, where large is a result of complex calculations, such as “the brain” in SC’s histogram panel provides.

If you want good images, then you have to collect enough photons from your target to differentiate it from the noise to a level that you find acceptable : your stacking software will assume that these subs, together with their accompanying calibration frames, have come straight from your camera - how can it know what’s been done to them, otherwise?

At the end of the day, however, it’s up to you to decide what compromises you are prepared to accept, and what constitutes an acceptable final image… you can use the live stacking FWHM & brightness filters to weed out sub-optimal subs, possibly combining this with saving only stacked raw frames for later re-stacking…

Regards, clear skies,
Carl

[Nyx]

Programs such as APP & StarTools only really work as intended when you feed them raw, linear subs to work on: if they have been pre-processed in some way, then the assumptions made in the stacking software don’t hold true, and so the results will be sub-optimal - how much so is very hard to say…

If you want to use very short subs, then you have to accept that, when imaging deep sky objects, you have no choice but collect a large number of them, where large is a result of complex calculations, such as “the brain” in SC’s histogram panel provides.

If you want good images, then you have to collect enough photons from your target to differentiate it from the noise to a level that you find acceptable : your stacking software will assume that these subs, together with their accompanying calibration frames, have come straight from your camera - how can it know what’s been done to them, otherwise?

At the end of the day, however, it’s up to you to decide what compromises you are prepared to accept, and what constitutes an acceptable final image… you can use the live stacking FWHM & brightness filters to weed out sub-optimal subs, possibly combining this with saving only stacked raw frames for later re-stacking…

Regards, clear skies,
Carl

What kind of pre-processing (in SharpCap) are you referring to?

[celkins]

Stacking dark & flat corrected subs…

[Nyx]

Stacking dark & flat corrected subs…

Example:

I have 1000 subs. I calibrate them using APP and save the calibrated subs. I take the first 20 subs (in chronological order) and stack them in APP using no outlier rejection. I repeat the same process 49 times more. I now load all 50 stacks as Lights and stack them again, this time using outlier rejection.

I am aware that stacking all 1000 subs in one go will produce a better image, but it will also take an astronomical amount of time to finish.

Taking that into account, how is creating 20-sub stacks in SharpCap using the same rules (dark/flat calibration and no outlier rejection) different than doing it in APP?

Now imagine shooting the same target over multiple nights using 15s subs. Stacking e.g. 20k subs is unrealistic, which is why the the idea of creating small 5-minute stacks in SharpCap crossed my mind in the first place.

[oopfan]

...or you could purchase a faster computer.

I'm really old-school. I don't LiveStack; I process one sub at a time in APP. I don't have a very fast scope (f/5.9) so I typically capture a minimum of 2 hours of 6-second subs: 1 hour with a red filter and 1 hour with a green filter. (Why 2 colors and not 3? That's another story.)

On my old Core-i3, APP takes 3 hours to calibrate, register, and stack. On my new Mac M1 Max, APP takes a little less than 1 hour. Those figures are for a 1920x1080 mono sensor.

Here is a recent example:
viewtopic.php?p=32723#p32723

If you go with Apple, I recommend a minimum of 16GB of memory.

EDIT: My M1 Max has a kick-a** GPU but I'd have to ask Mabula if APP takes advantage of it. I just want to be clear that the processing times that I am reporting are for a new M1 Max.

Brian

[Nyx]

...or you could purchase a faster computer.

I'm really old-school. I don't LiveStack; I process one sub at a time in APP. I don't have a very fast scope (f/5.9) so I typically capture a minimum of 2 hours of 6-second subs: 1 hour with a red filter and 1 hour with a green filter. (Why 2 colors and not 3? That's another story.)

On my old Core-i3, APP takes 3 hours to calibrate, register, and stack. On my new Mac M1 Max, APP takes a little less than 1 hour. Those figures are for a 1920x1080 mono sensor.

Here is a recent example:
viewtopic.php?p=32723#p32723

If you go with Apple, I recommend a minimum of 16GB of memory.

EDIT: My M1 Max has a kick-a** GPU but I'd have to ask Mabula if APP takes advantage of it. I just want to be clear that the processing times that I am reporting are for a new M1 Max.

Brian

A Bayer Drizzle Integration takes A LOT longer than 3 hours to finish in APP. And that's not counting the "mini integrations" you have to make in order to find the best combination of droplet size and scale.

My PC has the same specs as yours, SSDs and all. A new PC isn't in the cards. I could use a server at work with 40 cores, SSDs and 1TB RAM, but that's something I'd rather avoid.

Look, as I mentioned before, I know that stacking the individual subs all together gives the best results possible. I get it.

My question is whether the strategy I mentioned in my OP is a viable one

[Chuckwagon]

Are you currently using APP to stack individual short (6s or so) frames taken with Sharpcap, that had a dark and flat subtracted by Sharpcap when captured? If so, and that is working, then I see no reason why live-stacking those same frames and saving multiple raw un-stretched results would not also stack fine with APP. So long as you only calibrate (dark/flat) with Sharpcap, and don't try to do it again in APP, there shouldn't be any problem. If there is, then I'd suspect APP is doing something odd. I have combined stacks using other programs, like Nebulosity, and didn't have problems. But I don't use APP, so YMMV. I'd say, give it a try and see how it works for you.

Chuck

[Nyx]

Are you currently using APP to stack individual short (6s or so) frames taken with Sharpcap, that had a dark and flat subtracted by Sharpcap when captured? If so, and that is working, then I see no reason why live-stacking those same frames and saving multiple raw un-stretched results would not also stack fine with APP. So long as you only calibrate (dark/flat) with Sharpcap, and don't try to do it again in APP, there shouldn't be any problem. If there is, then I'd suspect APP is doing something odd. I have combined stacks using other programs, like Nebulosity, and didn't have problems. But I don't use APP, so YMMV. I'd say, give it a try and see how it works for you.

Chuck

This is what I normally do: I use NINA to shoot short (usually 10-15s) subs. I create flats using NINA's Flat Wizard, works as expected. I have created a master dark using 100 15s subs, which I then "scale" in APP since my camera isn't cooled.

The first thing I do in APP after every imaging session is to calibrate the lights and save the calibrated frames. Then I load the calibrated lights as Lights again and perform a star analysis. I keep only the good subs and delete the rest.

I then throw the sorted, calibrated lights in a folder with other calibrated lights from the same target shot over multiple nights (months...years...). Over time, the number of frames goes in the thousands. Every stacking session feels like an attempt to murder my PC

Since I can tell SharpCap to save a live (calibrated) stack every 5 minutes and then reset, that could dramatically speed up my APP integrations. That's the idea at least

I am going to try it of course. I was just curious if anyone has done it before

[turfpit]

This is a stack of 5 x 30 minute Live Stacks which gave a good result

https://www.astrobin.com/330099/

Dave