Save as RAW (32bit) & Sigma Clipping Stacking algorithm

Discussions of Electronically Assisted Astronomy using the Live Stacking feature.
fercapa
Posts: 15
Joined: Sun Mar 11, 2018 5:39 pm

Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#1

Post by fercapa »

Hi,

I have a problem when I save Live Stacking as RAW (32bit) and I have selected sigma Clipping as stacking algorithm. The saved fits files are only noise or some kind of corrupted data, like if data where on 16bit but the file definition where on 32bit.

But when I select 'Default' as stacking algorithm the saved files are ok.

Could it be a bug or am I doing something wrong?

Thanks in advance.

--
Fernando
fercapa
Posts: 15
Joined: Sun Mar 11, 2018 5:39 pm

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#2

Post by fercapa »

I have just seen that in "Status" tab of Live stack when I use Sigma Clipping algorithm the Bit Depth value remains at 16.0 and with Default algorithm it increases the more photos are stacked.

Maybe Sigma Clipping only allow 16bit.

--
Fernando
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#3

Post by admin »

Hi,

the sigma clipping mode stores the stack data as floating point numbers, so the concept of bit depth is somewhat fuzzy. Sounds like the corruption should be fixable though - I will look into it.

cheers,

Robin
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#4

Post by admin »

Hi,

what application are you trying to load the FITs files into? I have just tested saving 32 bit from Sigma clipped stack and had success loading into in DSS, FitsLiberator, PI and AstroArt. PIPP didn't like the files at all and refused to load them. No sign of any corrupt output.

cheers,

Robin
fercapa
Posts: 15
Joined: Sun Mar 11, 2018 5:39 pm

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#5

Post by fercapa »

I load the images at PixInsight. I'm using SharpCap 3.1.5059 and my camera is a QHY5III-174C

This is a screen capture with two images.
LiveStack.jpg
LiveStack.jpg (87.98 KiB) Viewed 4095 times
Left is the one with "Default" algorithm, right one is the same image with Sigma Clipping.
As you can see it is not an astronomical target. Is for a telecine machine I'm building to digitize old family movies. ;-)

Thanks

--
Fernando
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#6

Post by admin »

Ok, that's not right somewhere then. Please can you share the capture settings you are using (just save an image or a video and SharpCap will save the capture settings file alongside the saved file).

cheers,

Robin
fercapa
Posts: 15
Joined: Sun Mar 11, 2018 5:39 pm

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#7

Post by fercapa »

I can show you the capture settings of above images. Althoug they only differ in "TimeStamp", "Total Exposure" and "StackedFrames".

This is the settings for Default algorithm image:

Code: Select all

[QHY5III174C]
Debayer Preview=Off
Output Format=FITS files (*.fits)
Binning=1x1
Capture Area=1920x1200
Colour Space=RAW16
Pan=0
Tilt=0
Enable Live Broadcast=Off
USB Traffic=60
Offset=0
Amp Noise Reduction=On
Frame Rate Limit=15 fps
Gain=10
Exposure=0,497765168756617
Timestamp Frames=Off
White Bal (B)=249
White Bal (G)=83
White Bal (R)=106
Contrast=0
Brightness=0
Gamma=0,4
Apply Flat=None
Subtract Dark=None
#Black Point
Display Black Point=0
#MidTone Point
Display MidTone Point=0,5
#White Point
Display White Point=1
TimeStamp=2018-03-12T22:24:50.2249723Z
SharpCapVersion=3.1.5059.0
TotalExposure(s)=0,025883788775344
StackedFrames=52
And this is the settings of Sigma Clipping image:

Code: Select all

[QHY5III174C]
Debayer Preview=Off
Output Format=FITS files (*.fits)
Binning=1x1
Capture Area=1920x1200
Colour Space=RAW16
Pan=0
Tilt=0
Enable Live Broadcast=Off
USB Traffic=60
Offset=0
Amp Noise Reduction=On
Frame Rate Limit=15 fps
Gain=10
Exposure=0,497765168756617
Timestamp Frames=Off
White Bal (B)=249
White Bal (G)=83
White Bal (R)=106
Contrast=0
Brightness=0
Gamma=0,4
Apply Flat=None
Subtract Dark=None
#Black Point
Display Black Point=0
#MidTone Point
Display MidTone Point=0,5
#White Point
Display White Point=1
TimeStamp=2018-03-12T22:25:27.0811421Z
SharpCapVersion=3.1.5059.0
TotalExposure(s)=0,0378301528255028
StackedFrames=76
Settings of Live Stack where also the same in both images. I only changed Stacking algorithm.
fercapa
Posts: 15
Joined: Sun Mar 11, 2018 5:39 pm

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#8

Post by fercapa »

Hi again.

I'm also comparing two images with Default stacking algorithm, one saved as 16bit and the other as 32bit, and I'm not sure I know how to work with 32 bit images. It seems that the 32bit image is the same than 16bit with an ofsset of 0.5.

This is the same image saved as 16 and 32 bits:
test16b.jpg
test16b.jpg (143.26 KiB) Viewed 4074 times
test32b.jpg
test32b.jpg (111.59 KiB) Viewed 4074 times
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#9

Post by admin »

Hi,

the 32 bit files are saved as signed integer values (-2 billion odd to + 2billion odd). Unless you are using a dark frame then the -ve values don't get used at all (and even with a dark, they don't get used much), but that explains the starting at 50% brightness because SharpCap essentially only uses the top half of the available range.

I'm still having no luck reproducing the issue you are seeing - tested with the QHY178C in RAW16, so very close to your setup and get working results - please see the contents of this zip : https://drive.google.com/file/d/1163da8 ... sp=sharing

I've included a 32 bit fits that displays fine for me in PI and FITSLiberator and some screen captures of the various pages of the live stacking setup that you can compare with your config.

cheers,

Robin
fercapa
Posts: 15
Joined: Sun Mar 11, 2018 5:39 pm

Re: Save as RAW (32bit) & Sigma Clipping Stacking algorithm

#10

Post by fercapa »

Hi Robin.

Thank you very much for your time. I'm not going to tell you all the tests I'm doing, it will be really boring, so lets go with the last one.

Actually the sigma clipping algorithm, saved at 32 bit, sometimes works and sometimes not. But I don't know why.

I have wroten a script that pause Live Stack when reach certain number of frames stacked and then saves two files, one at 16 bit, and the other one at 32 bit. Then it moves the film (I'm digitizing an old movie film) to the next frame, resets Live Stack and resumes it, and so on.

Sometimes both images (16 & 32 bit) are identically like this one:
ok.jpg
ok.jpg (44.56 KiB) Viewed 4019 times

But in the next frame 16 bits image is ok but 32 is only noise, just like this:
ko.jpg
ko.jpg (287.59 KiB) Viewed 4019 times
The images are saved without debayer, for faster stack. I debayer them in PixInsight.

This is the part of the script that saves the files:

Code: Select all

	filename32="%s\\Fot%04d.fits"%(pathDestiny, frame)
	filename16="%s\\Fot%04d_16b.fits"%(pathDestiny, frame)
	SharpCap.LiveStacking.Reset()
	SharpCap.LiveStacking.Parameters.Paused=False
	while (SharpCap.LiveStacking.TotalFrames < numStack):
		time.sleep(1)
	SharpCap.LiveStacking.Parameters.Paused=True
	SharpCap.LiveStacking.SaveFrame(32, filename32)
	SharpCap.LiveStacking.SaveFrame(16, filename16)

I have to do more tests to find the cause, but I have the feeling that it fails more in low bright and fuzzy images.

Probably the 16 bit images are enough for my project. But I'm uncomfortable with this problem, so I will keep searching a bit. ;-)

--
Fernando
Post Reply