New Feature : Planetary Live Stacking

[admin]

Hi Mike,

thanks for the update - can you just confirm whether the images above are individual frames or saved live stack images (I suspect the latter). If that's the case then it looks like the alignment routine isn't coping with the shift of the image very well. Part of that could be that the area currently used for surface alignment is the middle of the frame, and that's a bit of a low contrast, low detail area in this case. If you re-align so that some sort of feature like a sunspot group is in the middle of the frame then that might well help. I'm currently working on a multi-point alignment and stacking option, which will help address this sort of problem.

Another thing that's worth checking is that the stacking rate (shown on the left hand side of the live stack area) matches the 60fps being delivered from your camera. It's possible that the CPU may not be able to keep up with stacking at 60fps and may be only processing some of the incoming frames.

One more thing to think about is the option of how long to use for calculating quality statistics (on the filtering tab). If this is set to a high value then you can end up with long periods with no frames added to the stack if the quality drops. In general this is probably best set to a value similar to the number calculated from stack length * 100 / percent to keep.

cheers,

Robin

[lumenmikie]

Hello Robin,
The frames are saved live stacked images, saved as 16bit .tif. I've managed to get 2 more sessions of 1hr 15' and 1hr 43' with at best moderate seeing. It is fascinating to watch the uncorrected image display and see it rendered as perfectly stabilized much of the time. My mount exhibits abrupt jumps of about 10 pixels, as often as every few seconds. This was completely corrected by the surface stabilization. The second, longer, session I left unattended more frequently and it had 2 periods of getting lost, like the images I posted previously. Once I manually reset the stack, it went back to humming along, churning out good images. There were high clouds coming in towards the end and that definitely reduced the contrast of the surface detail and making tracking more challenging. While having multiple alignment points might help in this situation, when the stack has turned into a large mushy mess of an image, a fallback action of resetting the stack might be the only way back.
I think the capture parameters I'm using are a good match to my hardware and the needed stacking rate. Thanks for tip on setting the integration length for the quality stats, that's helpful.
I did get useful time lapses but they were boring. But time lapses are now possible with my equipment which is very exciting to me. Thanks again for all of your efforts.
Mike

[RonAM]

Thank you both for envisioning and creating this planetary livestack capability.

I ran my SER file from the December 2020 Jupiter Saturn conjunction and it looks slightly better than my extensively post-processed version.

If only I had this software feature then, but I’m looking forward to planetary, solar and lunar imaging more than ever now, once the skies clear and warmth returns.

Thank you,

Ron

[admin]

Hi Mike, Ron,

good to hear that you both are having good results I'm surprised at the Saturn image turning out better than a fully-post processed one, as in a number of places I have taken the 'quick to run, but simple' approach to processing to ensure that image updates rapidly and stacks at a high frame rate. On the flip side, perhaps having all the adjustments in one place and seeing the results update live helps compared to processing through separate applications.

I do have some significant improvements coming in the next update - should be with you all tomorrow.

cheers,

Robin

[RonAM]

Looking forward to seeing what’s improved.

Yes, having all the controls in one place and seeing the effects live helps a lot, as well as the fineness of each control.

The focal length for that December 2020 session was set to see both Jupiter and Saturn simultaneously in a 1-deg FOV.

So the available details on each planet were limited compared to longer focal length imaging.

The biggest challenge is to get the processing set to brighten Saturn enough to see some details while not overexposing Jupiter which would hide its details. The livestack algorithm made this easy to accomplish and would have been of great assistance for the original imaging session.

Some folks imaged each planet separately but I wanted to capture both simultaneously in one image frame. It took some planning to set the gain and exposure just right for the original session so that the SER would contain usable information for both planets.

[admin]

Hi folks,

I have just uploaded a new SharpCap update - 4.1.11648. This contains several new enhancements to Planetary/Solar/Lunar live stacking

Multi Point Stacking

Multi-point stacking is here at last - when enabled, SharpCap will create a grid of align points across the interesting parts of the image and calculated image movement at each align point for each frame, taking account of that in the stacking process.

Screenshot 2024-01-08 151755.png (14.14 KiB) Viewed 168825 times

This can be enabled for both planet/full disc mode and surface align mode. I have found that the best results from this seem to come from solar imaging, where it helps give crisper features across the whole image. Lunar stacking also gets some benefit, but I have not yet noticed much improvement for planetary stacking by using the multi-point mode.

Multi-point stacking does increase the amount of calculation needed, so it will slow down the maximum stacking rate by maybe 30 to 50% - the rate at which the image updates is also reduced by a similar amount. For high resolution cameras (over maybe 4 to 6 megapixels), it may be worth binning 2x2 on the camera to reduce the amount of computation needed for stacking (unless you have a very fast PC). I have already done some work on making the multi-point stacking as fast as possible, including using multiple cores on your CPU if available.

You can adjust the spacing between align points and the noise reduction level used when evaluating image offset at each align point (this is a binning factor, so noise reduction 2 means the image will be binned 2x2 before the alignment calculations happen, 4 means binned 4x4 - I have found that already binned images often work nicely with level 2, other images usually do better with 4). Note that changing these will reset the stack.

You can also show the grid of align points on screen in SharpCap (this will not affect saved files).

Screenshot 2024-01-08 152648.jpg (153.48 KiB) Viewed 168825 times

The small green squares show the align points themselves and are at the center of each stacking tile. The large pink boxes centered on the green squares (yes, really, but the neighbouring ones also have edges that go through the center of the green squares) show the area over which alignment is calculated for each align point. The smaller blue boxes are the stacking tiles themselves. Actually, the bulk of the alignment data comes from the inner part of the area covered by the pink square, roughly matching up with the stacking tile.

If an area has a poor alignment result either for particular frames or continuously then it will use the alignment offset calculated for the frame as a whole - this also applies to dark areas outside any of the alignment boxes... These dark areas *are* stacked but use the global frame offset rather than a local alignment calculation.

Improved Surface Alignment

For higher resolution images (more than 1000x1000), surface alignment now takes measurements at 9 regions in a grid across the image to calculate the movement of each frame, rather than using a single region in the center of the image. This will help avoid problems when the center of the image is low contrast or has few features. This gives improvements even with single point stacking in use.

Improved responsiveness to adjustment changes

For larger images, SharpCap already only updates part of the image while you are changing the sharpening/adjustment sliders to give quick feedback. In this version, SharpCap also pauses regenerating the image from the stack data during slider adjustment - that means that the image onscreen will react even faster to slider changes, but won't show the improvements due to new data being added to the stack until you stop moving the sliders.

Fix timelapse brightness stabilization

Previous versions of timelapse brightness stabilization only worked if the bright part of the image covered more than about 75% of the image area - OK for solar/lunar but didn't work well for planetary. This is now fixed.

Better handling of loading corrupt SER files into 'Test Camera 2 (High Speed)'

I've seen a few crash reports that seem to be related to loading bad SER files (or files that aren't SER at all) into the high speed test camera. I have improved the checks used when opening SER files to try to avoid this sort of problem.

Still on the TODO list

I have few things that I still want to improve, including

* File naming during a stack
* Sharpening causing over-bright (saturated) areas, particularly in high contrast regions
* Sharpening causing ring halo effects around planets or at the solar/lunar edge

I probably won't be attacking these straight away, as I have a couple of other improvements elsewhere in the software that I want to complete.

cheers, and clear skies!

Robin

[lumenmikie]

Hi Robin,
I installed the new update and got 2 sessions of data to report on. The first day I used the new multi-point stacking mode.The seeing was poor to moderate, but the live stacked images seemed to be staying registered, which was encouraging. I ran the images through a image derotation python script I use and then used PixInsight Blink to examine the results. The results were strange: the sunspots were rotating rather than fixed. I then blinked the original raw files. The multi-point had done such a good job it had warped the stack to de-rotate the sunspots. As the warping got greater on successive frames, the image contrast declined, presumably because it was overwarped.
I ran the original frames through PIPP's frame quality processor to try to understand what was happening. Two of the quality measures seemed to track what was observable from visual assessment of the images, I include both, normalized to their respective starting values.

2024-01-08 17_53_23Z_pipp_quality_normalized.png (483.85 KiB) Viewed 159613 times

Every jump upward on the graph corresponds to a stack reset. The PIPP quality measures are based on gradients in the image, the falloff in the quality measure corresponds to a loss of contrast on the solar disk.
Every time the stack was reset, the image would rotate to the current solar orientation and stabilize to that - until the next stack reset.
Today, I used single-point stacking in the latest version. Seeing was moderate to good. I ran the results live frames thru the PIPP quality routines and got this result:

2024-01-09 18_20_53Z_pipp_quality_normalized.png (674.66 KiB) Viewed 159613 times

While there is some falloff of quality between resets, it is nowhere near the degradation seen with the multi-point method.
Image rotation is an inherent problem for users of alt-az mounts, whether doing timelapses or doing EAA. The multi-point stacking is clearly very powerful and I think it would make a difference even if the stack was reset for each frame.
Finally, I did get enough good data for a short prominence video! Very exciting!
If I can clarify any of these comments or supply more data, please let me know.
Thank you again for a great tool.
Mike

[admin]

Hi Mike,

great feedback and analysis - really useful. I don't use an AltAz mount myself, so don't see this sort of thing when testing.

For better or worse, at the moment the multipoint stack picks the first frame to arrive as the alignment frame and tries to align everything else to that frame. When I was testing during development I tried various ideas like replacing the alignment frame with the current state of the stack at regular intervals, but those choices led to a poorer final image on the test data I was using, so I didn't implement them in the end.

The multi-point align can't actually rotate the image, since it only performs horizontal and vertical offsetting, but it can (and will) try to pull the alignment points back to their original positions on the first frame. Imagine cutting the image up into squares then moving the centers of the squares in a circular movement while keeping the squares themselves still oriented the same way around - that's pretty much what is happening.

It will obviously take me some time to work out the best approach to fixing this - I will probably take some existing videos that I have and add an additional slow rotation to simulate the problem so that I can experiment with different options.

cheers,

Robin

[lumenmikie]

Hi Robin,
I got about 2 hours of data yesterday, the first 45 minutes or so being rather boring, and then whoosh!, this happened:
https://www.instagram.com/reel/C19HM5AL ... c0MzIxNw==
These frames were acquired with the single point stacking algorithm. I want to see what impact stack size had, given I wanted to take time lapse frames 20 seconds apart. Here is the chart of PIPP quality ratings.

20240110_pipp_quality.png (566.33 KiB) Viewed 153165 times

I took roughly 1/2 hour of data (90 frames) for each stack size in this order 32 (15%), 64 (20%), 128 (25%), 96 (20%). The percentages are the percent frames retained. Roughly every 30 frames I did a stack reset. The 32 frame stack did poorly. The best range is probably 64-96 frames; 128 seemed to struggle to get enough frames of sufficient quality in 20 seconds.

From frame 381 to finish, I used multi-point stacking, but reset the stack every 3 frames, to mitigate the effect of image rotation. Some of the frames that captured the prominence looked a little strange, like there was vibration or something. The prominences are low brightness, so maybe that makes them a harder target for the multi-point algorithm to sample sufficiently. I can supply data; I'm not sure spamming this list is the best way.

Anyway, totally excited about the capture today; thanks again for all the support.

Mike

[admin]

Hi Mike,

lovely capture of the erupting prominence - just the sort of thing the timelaps is for (well, that and Jupiter rotation, obviously!)

I still haven't had a chance to look at the tweaks needed to multi-point align yet to make it rotation proof. Having thought it through, I am concerned that replacing the 'reference' frame at any point will require quite a lot of book-keeping calculation to avoid the new reference frame distorting the stack on its own Trial and error will be involved I think!

cheers,

Robin