Recording in ADV format

Discussions of using SharpCap for Solar or Lunar Imaging
Tonisee
Posts: 7
Joined: Wed Jun 03, 2020 2:12 pm

Re: Recording in ADV format

#11

Post by Tonisee »

Robin,

first thank you for impressive program! I'm trying to tame it's Live Stack functionality and I'm very impressed about it's capabilities.

FITS format has indeed a nice default/standard possibility to store many frames - as a data cube (when number of axis, NAXIS = 3). Each frame (or extension) can have it's own unique header OR there can be one global header and a dedicated small(er) header for each individual slice in FITS data cube. A'la global for everything that goes for e.g. object, observatory, equipment, ... etc and very simple specific ones that cover filter and exposure/timing information. FITS as a data cube is commonly used in professional astronomy, both in optical and radio. FITS offers also (lossless) compression as a kind-of-standard feature (fpack/funpack), however I'm not sure if any of amateur astronomy programs support that out of the box.
And it is also trivial to get individual frames out from FITS datacube (astropy/pyfits/cfitsio) or even slice it along any axis (that's why it is popular e.g. in radio astronomy).

However, should you implement FITS datacube functionality, please ensure that NAXIS = 3 is used only for datacubes. Many programs tend to do the sanity check of the file by checking IF NAXIS = 2 - and if it is not, file is considered incorrect or broken. Which is not necessarily true, especially because a FITS datacube (NAXIS=3), containing just one frame is basically an ordinary FITS, expressed just very slightly differently (but correctly after the standard). Just the length of the third axis, NAXIS3 = 1 in that case. :-) In principle, it would be easy to fit a RGB imaging session as a data cube, too: e.g. when there is a mono camera with filter wheel. A'la Jupiter.fits could contain R, G, and B (e.g. live stack) frames in one file instead of having many different ones. Probably it is not that useful for imaging planets but in system when there are many filters.

I'm keeping my thumbs for FITS datacube functionality, I could use it easily for photometry of very bright objects that require taking many frames per object.

Best wishes,
Tõnis
User avatar
admin
Site Admin
Posts: 13173
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Recording in ADV format

#12

Post by admin »

Hi,

I must admit I wasn't aware of the datacube possibility in fits format. Do you happen to know of somewhere I can download a sample video in that format so that I can have a play and see if the fits libraries are using SharpCap are capable of writing such a thing.

Thanks, Robin
Tonisee
Posts: 7
Joined: Wed Jun 03, 2020 2:12 pm

Re: Recording in ADV format

#13

Post by Tonisee »

Hello Robin,

sorry for delayed answer.

From that link: https://drive.google.com/drive/folders/ ... sp=sharing you get two spectroscopic datacubes, one for flat and another for Vega (there some changes can be seen). This is not really a video but rather a sequence of frames. And that datacube has just one header (the main one).
IMHO standard (c)fitsio and it's not too much stripped down derivates should be able to read and write datacubes without any problems. Everything should be just of a matter how that additional axis (e.g. NAXIS3 is interpreted). According to FITS homepage, HDU-s (extensions) are often used with dedicated headers (see https://fits.gsfc.nasa.gov/fits_primer.html). But a 3-D datacube is a standard feature and has usually just one (primary/main) header.

You can visualize those cubes I shared using SaoImage DS9, too.

Best wishes,
Tõnis
User avatar
admin
Site Admin
Posts: 13173
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Recording in ADV format

#14

Post by admin »

Hi,


Thanks for the files – I can see how they are put together, although I'm still unsure as to whether the fits library that SharpCap uses is capable of building such a file in a sustainable way. SharpCap uses a C# port of the Java nom.tam.fits libraries. I could definitely create a file with as big a dimension as I wanted on the third axis to allow for multiple frames, but I can't yet see a way to do that without having to hold all the data for the file in memory at once. I obviously have to have a much deeper look at the fits library than I have previously :-)

Thanks, Robin
Tonisee
Posts: 7
Joined: Wed Jun 03, 2020 2:12 pm

Re: Recording in ADV format

#15

Post by Tonisee »

Hello Robin,

I think you are correct. In principle, the design of FITS file should allow writing data into a file till the header is updated for actual values (either in when opening a file for writing or just before closing the file).
A'la something like that:
NAXIS = 3
NAXIS1 = 2 // x
NAXIS2 = 2 // y
NAXIS3 = 2 // z
END
(x1y1z1) (x2y1z1) (x3y1z1) (x1y2z1) (x2y2z1) (x3y2z1) (x1y1z2) (x2y1z2) ... (x3y3z3)

When pixels of first frame get written, you have that (x#y#z1) set, IMHO each new frame can be added. As the very final change, NAXIS3 value can be set to correct value.

Best wishes,
Tõnis
Arnaud Leroy
Posts: 1
Joined: Sun Sep 06, 2020 5:32 pm

Re: Recording in ADV format

#16

Post by Arnaud Leroy »

Hi all

a little question about ADV format . For example with ZWO camera , when I record in ADV , and when I open the video in Tangra , there's no information about the timestamp . In SER format it's ok . How to have embedded timestamps with ADV and such camera (ZWO, altair , ...) ?

Thanks for your reply
Arnaud
User avatar
admin
Site Admin
Posts: 13173
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Recording in ADV format

#17

Post by admin »

Hi Arnaud,

just checked out the code.

If you have a GPS camera then SharpCap will set the start and end timestamps of the frame in the ADV file. However if you do not have a GPS camera then SharpCap will only set the SystemTime tag on each frame (setting that timestamp to the best guess that SharpCap has of when the frame started being captured).

I'm not sure if this is correct behaviour or not - the documentation on ADV suggests that the 'SystemTime' is lower quality timing than the frame timestamps, so it seems to be the right place to put that data. The concern is that if SharpCap introduces its 'best guess' timestamps for start/end frames into the main frame timestamp data then that could be mistaken later for being accurate and precise timings.

Open to feedback on this - if ADV had a 'timestamp source' or 'timestamp quality' metadata field then SharpCap could set that to 'low quality' and write the full timestamp info, but I don't see anything like that in the spec.

cheers,

Robin
Post Reply