HOWTO Debayer Captured Images

A place to post guides, instructions and HOWTO documents on the use of SharpCap and Astrophotography in general
Forum rules
One tutorial per topic please.
Please only post content that you have created in this form.
Tutorials should be either in PDF format as an attachment or written in the first post of the topic. Links to video tutorials you created are also acceptable.
User avatar
1CM69
Posts: 61
Joined: Tue Nov 14, 2017 9:49 pm

Re: HOWTO Debayer Captured Images

Post by 1CM69 » Thu Sep 13, 2018 1:20 pm

Thought I'd share my recent experience.

Last night/this morning I set my ASI224MC up as an All-Sky_Cam using the supplied 150deg 2.1mm f2 lens & after some fiddling around getting the correct exposure/gain combo I set it going capturing frames for a Star Trail at 45s & saving as RAW16 .tif files.

All good, ended up with 630 frames that needed debayering, no problem, I had already thought that I would be using PIPP for this even before I found this thread.

Problem was that as soon as I added any of the files to PIPP I got an error message:
PIPP ERROR.png
PIPP ERROR.png (30.88 KiB) Viewed 64 times
I then spent a good couple of hours trawling the net to find ways to debayer/demosaic the files.

I already had RAWTherapee installed and that does demosaic but only true RAW files, .tif is excluded.

I downloaded GIMP and the plugin G'MIC, that did sort of work but only 1 file at a time.

Searching the SharpCap forum I found this thread and tried again to follow the steps to use PIPP (current installed version 2.5.9) but still the same error.

The files were captured in SharpCap v3.1.5219.0 if that makes a difference.

Suddenly I had a brainwave, I had installed the SharpCap v3.2 beta & that has the Folder Monitor feature.

I started SC v3.2, pointed the Folder Monitor to the directory where my Star Trail frames were located, set Output Format to .ser & Debayer to FORCE RGGB, set the Frame Rate Limit to 60fps and then started an Unlimited capture before hitting the Play Button under Camera Controls.

This quickly produced a .ser file containing my 630 frames.

I was then able to process this .ser file using PIPP to debayered .tif files.

Long winded but I was nearly about to give up until finding this workaround.

Regards..,

Kirk
Last edited by 1CM69 on Thu Sep 13, 2018 2:52 pm, edited 1 time in total.

User avatar
turfpit
Posts: 312
Joined: Mon Feb 13, 2017 8:13 pm
Location: UK
Contact:

Re: HOWTO Debayer Captured Images

Post by turfpit » Thu Sep 13, 2018 1:40 pm

Thanks for posting this Kirk.
The files were captured in SharpCap v3.1.5219.0
Kirk - do you have the Sharpcap capture settings file to post? Did the files you put into PIPP come straight from Sharpcap or via some other software?

Robin - any thoughts on the "TIFF file has unsupported compression".

Dave

User avatar
1CM69
Posts: 61
Joined: Tue Nov 14, 2017 9:49 pm

Re: HOWTO Debayer Captured Images

Post by 1CM69 » Thu Sep 13, 2018 1:59 pm

turfpit wrote:
Thu Sep 13, 2018 1:40 pm
Thanks for posting this Kirk.
The files were captured in SharpCap v3.1.5219.0
Kirk - do you have the Sharpcap capture settings file to post? Did the files you put into PIPP come straight from Sharpcap or via some other software?

Robin - any thoughts on the "TIFF file has unsupported compression".

Dave
Hi Dave,

No I’m afraid I ditched the settings file but I know exactly what I had the settings as & I still have the RAW16 files that were direct from SC, not any other processing between capture and attempting to put them through PIPP

Kirk

User avatar
admin
Site Admin
Posts: 1879
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: HOWTO Debayer Captured Images

Post by admin » Thu Sep 13, 2018 3:19 pm

Hi Kirk,

a neat workaround! Well thought out :)

SharpCap writes TIFF format using the OpenCV library and it looks like it is compressing the TIFF file (good) but in a format that PIPP doesn't understand (bad...).

It looks like there may be a semi documented way of avoiding the compression - https://stackoverflow.com/questions/120 ... ing-opencv - the question is what the right choice is here? Always uncompressed? An option?

cheers,

Robin

User avatar
1CM69
Posts: 61
Joined: Tue Nov 14, 2017 9:49 pm

Re: HOWTO Debayer Captured Images

Post by 1CM69 » Thu Sep 13, 2018 3:45 pm

admin wrote:
Thu Sep 13, 2018 3:19 pm
Hi Kirk,

a neat workaround! Well thought out :)

SharpCap writes TIFF format using the OpenCV library and it looks like it is compressing the TIFF file (good) but in a format that PIPP doesn't understand (bad...).

It looks like there may be a semi documented way of avoiding the compression - https://stackoverflow.com/questions/120 ... ing-opencv - the question is what the right choice is here? Always uncompressed? An option?

cheers,

Robin
I have actually found my settings file now, still had it on my Astro Laptop & I noticed that the SC version I am running there is 3.1.5193.0, so to mislead. I have obviously updated the install on my processing PC but not on my Laptop, don't know whether this help ;)

Maybe Chris, the owner of PIPP can implement something so SC & PIPP can talk to each other :D

Here's my settings file:
2018-09-12-2053_6.CameraSettings.txt
(697 Bytes) Downloaded 3 times
Kirk

User avatar
1CM69
Posts: 61
Joined: Tue Nov 14, 2017 9:49 pm

Re: HOWTO Debayer Captured Images

Post by 1CM69 » Thu Sep 13, 2018 4:04 pm

admin wrote:
Thu Sep 13, 2018 3:19 pm
- the question is what the right choice is here? Always uncompressed? An option?

cheers,

Robin
Not really sure I would want masses of uncompressed .tif files on my system or possibly causing a bottleneck somewhere.

Perhaps an accompanying app to decompress the files, similar to the workaround. This does not need to be located on the imaging PC/Laptop, just a thought.

Kirk

User avatar
turfpit
Posts: 312
Joined: Mon Feb 13, 2017 8:13 pm
Location: UK
Contact:

Re: HOWTO Debayer Captured Images

Post by turfpit » Thu Sep 13, 2018 6:38 pm

Robin

Out of interest, what is the compression ratio?

Would it not be expedient for you to have a brief discussion with Chris Garry. PIPP is very robust at handling all sorts of file formats. The phrase unsupported compression is key. Is it just that PIPP cannot handle a particular type of compression or is it something to do with the OpenCV library? Is PIPP the only software that cannot handle these files? This might be building a problem for the future with software that has yet to be written.
there may be a semi documented way of avoiding the compression
That wouldn't go down well in the nuclear industry.

At the end of the day, maybe the imager should have the option to choose, particularly as this is potentially an issue about standards based formats?

Dave

User avatar
admin
Site Admin
Posts: 1879
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: HOWTO Debayer Captured Images

Post by admin » Thu Sep 13, 2018 8:12 pm

Hi Dave,

I have added an option as both use cases had value.

The compression was about 3x smaller on a mono camera, but only about 15% on a colour camera in RAW mode. However I found an alternate compression mode (mode 8, DEFLATE rather than mode 5, LZW) that gives better compression on RAW images (35-40% reduction).

Anyway, I have added a new option to the 'General' tab to turn compression on/off (default on). As far as I can see, PIPP does not support any of the compress modes (I tried several). I suspect it may be using hand written code to read TIFF rather than a library, which would make adding support for compression a bit of an effort for the author.

While I was tinkering on the general settings page I upped the intensity of the warning about not using the 'don't stretct FITS' option unless you know what you are doing.

cheers,

Robin

User avatar
turfpit
Posts: 312
Joined: Mon Feb 13, 2017 8:13 pm
Location: UK
Contact:

Re: HOWTO Debayer Captured Images

Post by turfpit » Thu Sep 13, 2018 8:34 pm

Thanks Robin.

30-40% reduction is worth having, especially with today's big megapixel cameras.

I will make sure I pick up the update SharpCap beta.

Dave

User avatar
1CM69
Posts: 61
Joined: Tue Nov 14, 2017 9:49 pm

Re: HOWTO Debayer Captured Images

Post by 1CM69 » Thu Sep 13, 2018 8:39 pm

Hi Robin,
while allowing people the choice pre capture, do you think most people will choose compression for space saving & then be in a position of trying to decompress later?

Just wondering if a separate decompressing app or option can be of benefit post capture to allow files to be processed in far more image processing applications, plus act as a safety net for those that forgot/missed the OFF option at the start.

Also, how would SharpCap be affected with handling uncompressed image files during an imaging run, does the fact that SC does not have to compress outweigh the handling of larger files?

Just interested for your take on this.

Regards..,

Kirk

Post Reply

Who is online

Users browsing this forum: No registered users and 0 guests