80% of frames dropped when recording

[1stSolo]

I have ASI662MC, 64GB RAM (16GB sharpcap buffer), licensed version, fast SSD that tests 1600-2400fps write speed.
Sharpcap v4 April 2023 build, latest ZWO drivers and ASCOM platform 6.6 SP1.
The title bar in windows shows ASI662MC USB3.
"Turbo USB" is set to "Auto", but I tried "80" also.
Video format is SER.
In preview, the camera is streaming 186fps for AOI 800x600 with 2ms exposure, gain=300.
But when I start recording the fps drops to 36fps and when 5000 frames are saved it shows 32K dropped. Also it's not using the high speed cache.
What could be the reason for 80% dropped frames?

[admin]

Hi,

the best thing to do is to check the log while capturing, since SharpCap will record some info in the log about what is causing the dropped frames. There will be lines in the log like

Code: Select all

Dropped Frames : 500 of type XXXXX

It's the type that is important - it tells me where in the code they are being recorded and that will help understand what is going on.

Based on your description that the memory buffer is not being used, then I am thinking of two possibilities

1) The frames are being dropped before they are getting to SharpCap - somewhere in the ZWO code

2) The CPU level is hitting 100% when saving and that is meaning that SharpCap cannot keep up with the frames coming from the camera.

One thing worth checking - if your SSD is connected over USB then that could be the problem - if it shares a USB hub with the camera (even an internal hub), then as soon as you start saving it will start using bandwidth on the USB and affect the frame rate from the camera.

cheers,

Robin

[1stSolo]

Hi, Robin. I'll check the logs.
The SSD is a nvme board. The CPU is barely used - it's a 12-core 24-thread AMD Ryzen 9 chipset with 64GB RAM at 3.1GHz.
As I mentioned, the speed test on that drive estimates 1600+ fps write capability.
Frames aren't dropped in preview, but only when recording.
I had a slower computer before which was always filling up the buffer first before beginning to drop frames due to slow SSD. This one used to work perfectly - I could get 138fps at 1960x1080 before, so something changed...
I'll update ZWO drivers for Windows and ASCOM, and I already upgraded ASCOM platform to 6.6 SP1.

[1stSolo]

Screenshot 2023-06-08 213605.jpg (500.11 KiB) Viewed 9065 times

I'm attaching the screenshot.

At first I thought I found a problem: after unplugging from power source my laptop drops CPU speed to 500Mhz. Still it was using only 2-5% CPU when recording video. Then I rebooted the laptop and now it's at 2.91GHz (95% power per current Windows Power Plan). And still CPU is 2% utilization.
Hard drive (SSD) is not too busy and neither is nVidia card with 2300+ CUDA cores.
The log is showing a small fraction of frames dropped due to "ZWOFromSDK", but majority - 80% - are dropped due to "PreviousFrameNotProcessed"

Please advise. When not recording the camera is showing 180+ fps with ~2ms exposure, 800x600, or 76fps at 1980x1080.

I attached the log.

[1stSolo]

I switched to the latest ASIStudio/ ASICap and able to capture 185-186fps (2ms exposure) with 0 (zero) dropped frames with that software, directly to disc without memory buffering.
Seems like a bug in the most recent SharpCap.

[admin]

Hi,

thanks for testing and sending the results. Not what I expected! The frames are coming through from the camera, but are being dropped internally by SharpCap quite early in the processing flow.

There are a couple of things that could cause this

* One or more of the processing options being turned on in the SharpCap 'preprocessing' control blocks - these can take time and will cause frames to be dropped if it hasn't finished by the time the next frame arrives

* Any actiive tools while you are capturing (but from the look of the log you have turned them off).

I would suggest that you try opening the camera with the CTRL key pressed down which will reset all the camera controls and turn off any pre-processing ones that may have been turned on previously and remembered by SharpCap re-loading previous settings when you open the camera.

It's also worth having a look at SharpCap 4.1 beta to see if the same problem happens there - I am no longer updating SharpCap 4.0, and in fact the only changes in it in the last 6 months or so have been SDK updates from camera manufacturers, so they are unlikely to cause the issue you are seeing.

cheers,

Robin

[Andreas83]

Hi!
I think I have the same problem, but my log-file is about 7 MB so I cannot attach it...

I have an ASI174MM USB3 and USB3-cable+port on laptop and a fast SSD.

After some records it happen that to much frames are in the puffer for writing.
I have the latest sharpcap-version.

Could this help from the logfile?

Thanks
Andreas





Info 12:26:25.087796 #18 Capture Event : DrainingWriteQueue in void SharpCap.MultiCaptureProvider.SendCaptureEvent(object sender, CaptureEventArgs e)
Debug 12:26:25.088383 #1 Notification (Status=OK): Writing buffered frames, 315 to go. in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Info 12:26:25.296108 #18 Capture Event : DrainingWriteQueue in void SharpCap.MultiCaptureProvider.SendCaptureEvent(object sender, CaptureEventArgs e)
Debug 12:26:25.296854 #1 Notification (Status=OK): Writing buffered frames, 310 to go. in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Info 12:26:25.462920 #18 Capture Event : DrainingWriteQueue in void SharpCap.MultiCaptureProvider.SendCaptureEvent(object sender, CaptureEventArgs e)
Debug 12:26:25.463747 #1 Notification (Status=OK): Writing buffered frames, 305 to go. in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Info 12:26:25.715194 #18 Capture Event : DrainingWriteQueue in void SharpCap.MultiCaptureProvider.SendCaptureEvent(object sender, CaptureEventArgs e)
Debug 12:26:25.716059 #1 Notification (Status=OK): Writing buffered frames, 300 to go. in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Info 12:26:25.974463 #18 Capture Event : DrainingWriteQueue in void SharpCap.MultiCaptureProvider.SendCaptureEvent(object sender, CaptureEventArgs e)
Debug 12:26:25.975436 #1 Notification (Status=OK): Writing buffered frames, 295 to go. in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Info 12:26:26.235811 #18 Capture Event : DrainingWriteQueue

[admin]

Hi,

thanks for posting the log fragment - what seems to be happening in your case is that the hard disk/SSD isn't able to keep up with the amount of video data coming from the camera. Sometimes a hard disk/SSD can initially keep up and then slows down after some data is written.

When the hard disk/SSD cannot keep up, SharpCap will hold video frames in memory, but the memory will then fill up too and after that frames will be lost. When you stop capturing, you have to wait for the frames in memory to be written to disk.

Things you can do:

* Upgrade to an SSD if you are using a hard disk
* Remove any unneeded large files from the disk/SSD to clear more free space
* Ensure that your hard disk/SSD has recently been defragmented/optimised
* Adjust the memory settings in SharpCap to assign more memory to 'High Speed Frame Cache' if you have plenty of memory - see https://docs.sharpcap.co.uk/4.1/#Memory%20Settings . Note this requires a SharpCap Pro license

In particular, it's worth mentioning that there are issues with some budget SSDs that are included in mini PCs - They are really fast when writing relatively small amounts of data (up to about 10% of capacity) and then slow down a *lot*. If you leave them running for a few hours then they shuffle the newly written data from the fast part of the SSD into slower parts and then they will be fast again for a bit.

cheers,

Robin

[Andreas83]

Hi Robin,

thanks a lot for the reply: I will test another SSD. Maybe thats the problem!!!

A few minutes ago I opened a new topic at ASI here in the forum...

Greetings

Andreas