M51. The use of magic to improve sharpness

[timh]

More on seeing deeper into the whirlpool

This earlier post viewtopic.php?t=5252 described using images integrated from selected 'good' short (3s) frames - e.g. FWHM-filtered autostacks - as luminance as a way to improve the sharpness of the brighter high SNR areas within images of subjects such as galaxies - M 51 in that case.

Here I combine the short frame luminance approach with Deconvolution to improve sharpness even further. It worked really quite well so I though that folk might be interested. It is particularly useful applied to long focus and oversampled images with high SNR (Menno you might find it useful?)

For a long time I had avoided Deconvolution as a topic because it looked 'too difficult'. Actually it turned out to be tricky (have to be careful not to create artifacts) but actually not bad and worth the effort -- not sure how many other folk use it routinely already? - and I was really pleasantly surprised by the the size of the improvement it brings. Basically (in the 'dynamic' PixInsight exemplification) you generate a PSF function from measurements of stars within the image that you wish to 'restore' - this star shape function in comparison with the theoretical ideal contains information about the particular way in which optics, movement and conditions etc added distortion. This calculated distortion function is then deconvolved from the image to restore the true underlying image. As far as I am concerned though it really just works by magic .

The pictures are probably self-explanatory ?

The first set show the luminance from my 'normal' 40s M51 image and the 3s frame image of the same and then the sharpness improvement that you see from deconvolving each of them. You can see that the 3s image becomes ultra sharp -- but rather noisy reflecting its lower SNR.

The second set shows colour pictures with A ) being where I was this time last year (PDS200 images), B ) a somewhat sharper image taken with a VX12 telescope c) the effect of adding the luminance of short frame images to the RGB image of B and D) the effect of adding the now deconvolved short frame luminance to B.

The third picture combines everything -- a high dynamic range composition of three sets of luminances (the inner two-deconvolved) back into an M51 composition combining in some Bortle 4 with Bortle 6 frames.


Tim

[oopfan]

Hi Tim,

Just curious, measure the average pixel value of a small square of dark space in a single 3s light frame, and then do the same for a bias frame in approximately the same square. What is the difference in pixels and in ADU? Interested to know how much LP there is in 3s. (Dark current has got to be virtually negligent in a cooled camera.)

Thanks,
Brian

[timh]

Hi Brian

I don't have individual bias or dark frames - always simply use a library of SC generated master darks so I always have one to correspond to the lights. But herewith some data that I hope is helpful in some way.

The AS1294 MM camera was used in the 'bin 1' mode has 2.15 uM pixels and a 12 bit ADU (4095 electron but only at low gain). My 3 second frames were at gain 200 (~0.4 electron/ ADU) and cooled to -10C ---as you say - negligible dark current is expected (about 0.02 electron)

Background signal in individual 3s light frames . Mean bckgrd signal was ca 50 with stddev ~ 8.5 (electrons)
(..taking an area that was mainly galaxy, in the better frames the mean was ~ 75 with stdev ~ 22)

The master dark integration of 125 x 3s frames. Mean bckgrd in this was 32 with stddev 2.5

The integrated lights (896x3s) . Mean backgrd was 18 with a stddev of about 0.30 (which makes sense given the above and that the dark is subtracted).

As you will appreciate what I am really up against is squashed dynamic range due to the read noise - about 1.6 e per read - and so at least 1200 electrons there with 800 reads.

So I will be experimenting with slightly longer 'short' frames of about 5-6s and at gain 100 since it may be a slightly better compromise?

Tim

[oopfan]

Thanks Tim for looking into that for me. I'm following your work with great interest. I'm still a little skeptical that "lucky imaging" (i.e. ultra-short exposures) is the reason for the increased sharpness. You mentioned "squashed dynamic range". That got me thinking that something else might be happening. Perhaps lucky imaging is having a similar effect as employing a special type of filter in post-processing?

I've seen photos of M51 taken with a similar aperture as yours. They used a CCD so it is not surprising that the exposure was 300s. They captured great signal per frame at Bortle 4. After processing, M51 lived up to its name: Whirlpool. It definitely had depth, truly like water being flushed down a drain. Sadly your photo lacks that same 3D quality. I'm wondering if it is due to the "squashed dynamic range" like you mentioned? Don't get me wrong, your photo is great, certainly better than anything I've done. My hat is off to you!

Brian

[turfpit]

Tim

A very good M51. Thanks for the detailed write-up. It looks like deconvolution was worth the effort.

Dave

[timh]

Hi Brian,

On the short frame sharpening effect I completely share your skepticism - which in a way is why I keep experimenting with it. It seems unreasonable.

My thoughts/ insights after spending far too long trying it out and thinking about it is that there is an effect but it is not huge and the probable explanations quite simple -- I probably wouldn't even be trying this stuff at all if skies here were habitually steadier ---

1) It’s only a small effect. On a night where the FWHM of my - weakly selected - 40s frames might average out at 2.4 typically I can get my - more strongly selected - 3s frame set to average out at about FWHM 1.8-1.9.

2) A key word here is 'selection'. When I describe an image as being comprised of say 800 x 3s frames - these are actually only the 'good' frames that have been selected from an initial number of probably ~ 6000. First the FWHM and brightness filters in Sharpcap are set to pass perhaps only a third or less - so I don't even collect frames at FWHM > 2.2. Secondly they are then further filtered by Subframe selector in PixInsight to pass - on the basis of # stars, ellipticity and FWHM - again only about a third. So the overall selection is quite harsh (about 12%).

So one key point about 3s frames as opposed to longer frames is that you can collect so many that it is feasible to select them harshly - collecting just 1 in 10 longer frames would simply not be feasible for longer exposures (which anyway - as far as I have observed - do not exhibit such wide varaiation in FWHM as shorter frames).

Of course - you can go further. Out of 800 so-selected 3s frames I identified 20 that really were superb -- lots of stars, little ellipticity and an FWHM of only ~ 1.6. Interestingly these few came together in what I assume were short periods of good seeing. But --not practically useful because selection at the 0.5% level is beyond even my powers of patience!

I think that another key point about 3s frames versus 40s simply comes down to probability. It is more likely that seeing (plus telescope movement etc) will remain reasonably good for any given timespan of 3s than it is for 40s or longer. There is more variation in 3s frames and the SD about the mean of the FWHM greater I think. This can be seen in star shapes -- i.e. the measured roundness of stars (r factor in the PSF) looks higher in longer frames than it is in shorter frames which I think is due to the longer period encompassing more apparent movements in more directions and so a better averaging to a rounder albeit larger PSF .

But ‘reasonable seeing’ here is just a relative term --it is not ‘lucky imaging’ territory


3) There is also a sleight of hand here in the FWHM values themselves. The FWHM of the 40s frames refers to OSC frames that will have been slightly blurred by being debayered whereas the 3s frames are mono. So the debayered FWHM of say 2.4 may translate down to about 2.2 for an equivalent mono frame (the debayer blur is only slight because of oversampling but it does bring the 40s and 3s FWHM values closer).


4) Finally and perhaps most importantly there is a major trade off involved in all of this - which is why I am interested in exploring lower gain and times up to 5s as maybe nearer the sweet spot? With short frames I am clearly sacrificing SNR for spatial resolution. Because of this the integrated image from the 3s frames can really only be productively applied to the very brightest high SNR regions of deepsky objects.

That is the reason that I am trying to use 64k HDR compositions of luminance with the 3s luminance contributing and being applied only to the brighter regions. But - as you will know - getting an HDR composition to work depends critically on getting the masking transitions to work at the right level and smoothly.

In the image of the core that I show above I have really not got the masking right - the 3s ‘domain’ extends too far and adds noise where it shouldn’t -- but I was more anxious really to demonstrate the deconvolution effect on spatial sharpness That I think is the reason for the rather poor appearance of the M51 core - including more of the deconvolved 40s image and also getting the stretch right will make for a better core I think -- plus of course more data --this is only a couple of hours albeit at F 4 and 300 mm.

Part of the inherent problem with HDR compositions is also that in the attempt to capture everything much of the contrast gets flattened out. It really might be better to just go for 'total galaxy' pictures to best show the overall galaxy-hood and all and a separate picture for the core just concentrating on the depth of the whirlpool

Thanks for being interested - it is appreciated !

Tim

[oopfan]

Hi Tim,

I know I'm talking about real money here, but perhaps a 14 or 16-bit camera will yield more depth.

EDIT: Maybe try running at Gain 120 with your ASI294. Looking at ZWO's website, that should give you one more stop of DR. RN will be slightly higher. Still it might be worth a try.

Brian

[timh]

That would be great Brian ...but £4k

But should work pretty well with the current set up. Agree with your analysis. Drop to about unity gain and RN goes up a bit (1.55-1.8 or so) but a big gain in FW capacity (1.5-4k) and another stop...so worthwhile.

In addition --- if the analysis of the way the STDEV of the mean of FWHM varies with exposure time is correct (i.e a Poisson shape distribution - with SD proportional to 1/ SQRT exposure (s)) ... then it would make sense to go to 5 rather than 3s to improve SNR. The cost would be that it will require a tighter culling of frames (~ 93% versus 88%) in order to keep FWHM down at about 1.8 or so.

I also revisited the masking of the HDRC of luminance and stretching to try and get a better view of the galaxy core -- it is comforting to see not too much in the way of artifacts as far as I can judge by comparison with the HST picture.

Tim