ASI 6200 download time

[Cobarcore]

I'm pushing to limits my ASI 6200 MM camera to get the highest framerate. I save to FITS format full resolution (62 Mpix!) 16 bit to my PC SSD internal drive. I look at the "DATE-OBS" field and I compute dt=time(n)-time(n-1)-exp(n-1), so dt is the effective dead time between two exposures.
I found that dt=786 ms for 0.1 ms series of exposures, but when increasing exposure e.g. to 512 ms, I measure a dt=274 ms that is exactly 786-512. Real measured dt of many exposures have small deviations +/-0.020 ms, but the average is the above value.

Up to now it is all ok.

But problems arise when I change exposure time between series of exposures. The dt of the first shot with changed exposure value is sometimes the 786 ms (same as above) and sometimes 1572 ms (the double).
Example with the double:
T [ms] dt [ms]
0.1 -
0.1 786
0.1 786
11 1562*
11 776
11 776
512 1060*
512 274
512 274
0.1 1572*
0.1 786
0.1 786
The * values sometimes are lower, and I cannot understand why. Most of all I'd like to get a method to get always the shortest dt as possible. And also a constant behaviour would be important for me because in a long sequence with many exposure changes this results in a random total time duration.

Any help?

Regards,
Lorenzo

PS: example of SCS sequence I used:
LIVE MODE
SET EXPOSURE TO 0.0001
CAPTURE 3 LIVE FRAMES
SET EXPOSURE TO 0.011
CAPTURE 3 LIVE FRAMES
SET EXPOSURE TO 0.512
CAPTURE 3 LIVE FRAMES
SET EXPOSURE TO 0.0001
CAPTURE 3 LIVE FRAMES
STILL MODE

[admin]

Hi,

I think I can help a bit...

Your limiting factor is the time taken to transfer the frame data from the camera to the computer - a 16 bit frame at 62 megapixels is about 1 gigabit of data (124 megabytes). In theory that could transfer over USB3 in about 0.2s, but there are overheads in the transfer which sound like they are leading to a transfer time of about 786ms in your case. You might be able to improve this by increasing the 'Turbo USB' setting if not already at 100, you could certainly improve this by using a smaller capture area (run the camera in ROI mode).

When you have a very short frame, the camera captures the image then spends the next 786ms transferring the data to the computer before being able to capture the next frame.

When you have a longer exposure like 512ms, the camera will capture and start transferring the data, but after about 274ms it will start capturing again - that's because by the time the *next* 512ms frame is finished the previous transfer will be done, so the camera can re-use the memory to read the next frame.

If you increase the exposure further to about 800ms, you will find that the camera can capture basically without pausing since it can transfer the whole of the data from the previous frame during the exposure of the next frame.

The double delays do not have such an easy explanation, but I suspect that sometimes the change of exposure may cause the camera SDK to decide to abandon a frame that is already being (or has been) transferred, meaning an extra 786ms needs to elapse while replacement data is transferred to the computer. I don't know if anything can be done about this, since I think this is happening inside the ZWO software - it's not something that SharpCap does for the sort of exposure times that you are looking at. Note that for exposures of above 2s, SharpCap does restart the camera video stream when the exposure is changed and the camera is in video mode - that means that the new exposure starts straight away rather than waiting for the old long(ish) exposure to end first.

I'd also say that you need to be careful about the use of FITS format for high speed capturing - it's the slowest output format for writing, and while SharpCap will buffer image frames in memory if the writing code can't keep up, the buffer will eventually fill up. SER is the best option for high speed capture, and does include a timestamp field for each frame.

cheers,

Robin

[Cobarcore]

Dear Robin,
thank you very much for your inspiring reply. This helped a lot to modify my script and obtain much better results.

First using someting like below avoid the loss of saving intermediate frames (the ones during an exposure time change), this independently of using FITS or SER saving. And also very important for me, the timing is perfect regardless of the download speed increases due to e.g. buffer full or pc slowed down. Before I used the command CAPTURE n LIVE FRAMES after each SET EXPOSURE.

Code: Select all

SET EXPOSURE TO 0.0001
CAPTURE START
DELAY 5
SET EXPOSURE TO 0.032
DELAY 5
SET EXPOSURE TO 0.512
DELAY 5
CAPTURE STOP

Then using SER format instead of FITS improved a bit the write speed on my SSD, even if not dramatically (I estimate less than 50%).
There is no better solution than a recent SSD! A personal comparison of two SSDs: a 2022 i7 pc write at 2600 MB/s while a 2017 i7 pc write at 530 MB/s (CrystalDiskMark 8, SEQ1M q8t1)

A disadvantage of SER format is that it does not contain the set exposure value of each frame! Or at least I was not able to find in the FITS exported using Siril (any better way?).

Most of all your advice to use "Turbo USB" (from standard 40 to 100) and High Speed Mode (from Off to On) improved a lot the download speed, now the time between to frames decreased from 786 ms to 317 ms, less than half!

About this I've a question, because I noticed that I get some corrupted image. In example during a series of 50 frames, I got a group of about 10 corrupted, with horizontal noise lines. Btw I connected the camera with a very short USB 3 cable 0.5 m long, so to reduce the possibility of corruption.
Any advice on how to improve?

Another question is about the buffer size, I've increased to 9 GB. I've 16 GB RAM. Is it possible to increase more with this RAM size? And if I have 32 GB RAM?

Great to know of the 2.0 s limit above wich SharpCap restarts the camera stream, so for very fast results I'll avoid exposures above that limit. Is that limit exactly at 2000 ms? So up to 1999 ms the camera does not restarts?

I wait impatiently for you precious reply

Regards,
Lorenzo

[admin]

Hi,

glad to hear that the info has helped you.

On the subject of SSDs, one thing to watch out for is that most SSDs these days store multiple bits per flash memory cell (MLC, TLC, QLC) - these drives write more slowly because of this, but usually have an area set aside for 'fast writing' to single-bit-per-cell storage. This fast area might be 10% of total space, and for normal use that is sufficient as you don't write enormous amounts of data in one go, and the SSD shuffles the data to slower storage in the background. However if you plan to capture *lots* of high speed video, you can end up filling up this area and the write speed drops massively when it is full.

I'm not sure if I would expect 'High Speed Mode' to have an effect, as I thought that only applied in 8 bit modes. You may find that 100 Turbo USB is too high and is causing your frames with glitches - try turning it down to 80 or 90 and see if that helps get rid of the glitches but keeps most of the performance boost.

Unfortunately, SER does not record per-frame exposure times, just per-frame timestamps. SharpCap does support ADV output (AstroDigitalVideo), which does have more info on a per-frame basis, but you are limited in the applications that will read that format - there is 'Tangra' (http://www.hristopavlov.net/Tangra3/) - not sure about anything else.

Maximum ram allocation in SharpCap is 1/2 of PC memory or PC memory less 6GB, whichever is more. Since you have to leave 1Gb minimum for 'other' use, that means PC memory less 7GB for storing frame data.

For the video/still switchover point, the camera will be switched to still mode when the exposure time is 2000ms or more, so 1999ms should stay in video mode.

cheers,

Robin