Performance issues

[astra]

Thanks for the detailed explanation. I have only a few questions.
1. Is it a display frame rate? Right now frame limiter option limits both display and writing framerates
2. With the automatic estimation of TargetFrameRate, does it limit display or both framerates?

I see what you plan to do with the threshold, so basically I had to estimate the value for me so that fps display(1.?) limit is not more than say 10-15. If any user wants the same, he has to estimate the same threshold for the desired fps, right? Then why not give an option to set DISPLAY TargetFrameRate explicitly? As an alternative to automatic balancer

[admin]

Hi,

the 'Frame rate limit' that you see in the controls on the right controls the rate that SharpCap will accept frames from the camera, so will affect both display and capture.

Internally, SharpCap tracks the rate that frames are displayed and the CPU load, and will try to adjust the display frame rate to keep the CPU load reasonable and the application responsive. If you are into control theory, this uses a P-I algorithm to continuously update an internal target rate at which frames should be displayed. This *does not* affect the rate that the frames are taken from the camera or the rate that they are saved.

I had to update this week's version because of an issue in the new live stacking 'Unstack' feature, so the update (4.1.12342) ended up with the code in to allow the CPU load target to be varied.

Please use the following registry file to alter the performance settigns - this file will turn on the monitoring UI and reduce the target CPU load from the default of 0.75 (75%) to 0.5 (50%). Below is another reg file that will reset defaults.

perf_settings.reg

(332 Bytes) Downloaded 39 times

Once this reg file has been used, restart SharpCap and open a camera - you should see a small extra window that gives details of SharpCap's measure of CPU load and the target frame rate. This may well try and hide behind your main SharpCap window, but can be brought back to the foreground via the taskbar.

Screenshot 2024-07-04 141038.png (4.65 KiB) Viewed 3165 times

You can tinker with the 0.5 target load value in the above reg file - If you set it too low then the display frame rate will drop to 1fps - this happens when the load cannot be brought down below the target load value regardless of display frame rate. Setting a high value will leave the display target FPS at 30, but there should be a range where the display FPS is pulled down and the UI becomes responsive.

I *think* the problem is that the WPF thread (wpfxgfx_cor3.dll) that you spotted in the ProcExp info is actually (and unusually) a bigger bottleneck than the main UI thread that SharpCap monitors and tries to manage. Unfortunately there is no clean way of finding the WPF thread from within SharpCap to monitor it - you would have to stackwalk all of the threads to try to find the target, which is a bit extreme to say the least

Hopefully a lower CPU load target will be enough to bring the wpfgfx_cor3 thread down too and make things responsive again. Do let me know!

cheers,

Robin

[astra]

Robin, finally, that's a solution!
No sun today, but in synthetic tests, that looks very promising! With additional flats postprocessing and a stretched histogram, I could find a suitable value to make the UI responsive during display and writing on disk. PID algorithm would work probably even better than a fixed FPS limit.
So I would play with that in real conditions, but for now, I have good hope that this is a fix!
Thank you so much!

Update. After a short session, everything looks great. Yeah, I have to restart to adjust param, but that's fine. Mosaic alignment is more CPU intensive by obvious reasons, and I also want to test the mosaic capture that I had issues with before.
Thank you for your patience!

[admin]

Hi,

thanks for letting me know that it's helping - I would be interested to know the CPU load limit value that you end up with as a 'working' value - If I put anything like this into the UI, I'm not sure if I should offer a slider or a 'low/medium/high' option to keep things simple...

cheers,

Robin

[astra]

So today I've tried a real timelapse and taking my words back, sorry

I mean, when the limit was applied in a single capture, it was ok, short tests worked fine.
In a real-life scenario, it worked a little bit differently. So my fps should be below 82-80, in the screenshots you can see it much lower. Overall thread CPU local doesn't correspond to actual process(or pc load). And that is probably the issue that algorithm is targeting indirect parameter of cpu load to control real display fps. So app sees that thread load is not high and increases display fps, overall process cpu usage goes high, writing speed decreases signtificaly.
Btw, does "Thread CPU Load" take into account processing flat step?

Another point is that when you minimize the app, that wpfgfx shouldn't draw much, right? And while minimized I get about 40% cpu load and 30 display fps, but overall cpu usage is high

One more point is that I would tweak pid alghoritm to change faster, when writing starts, it is better to limit display fps faster.



That is my observation for now

PS. In small tests the value of 0.45-50 is fine, but that depends in pc performance and won't suite everyone

Edit. After about 15-30 mins, target fps becomes 30 for values 0.4-0.6. In the algorithm, do have a sliding window or operate on the whole history of values?

[admin]

Hi,

the load measurement does take into account both the utilization of resources in the UI thread of the application *and* the total CPU load - basically the measurement is the higher of the two, so a high overall CPU load should push the frame rate down. If you set a particularly low threshold (maybe 0.2 or .25 ) then it should just push the FPS for display down to 1fps regardless of anything else unless the whole PC is very lightly loaded, which gives a way to check with minimum display load.

I must say though that the log and screenshots are pointing towards the image processing thread (probably flat frame) not being able to keep up with the rate the frames are coming from the camera. Additionally the SER file writing is not keeping up either, with over 200 frames held in memory by the end of the capture waiting to be written out.

I will need to test flat framing on a Celeron level CPU here - my initial thought is that the CPU may well not be up to the task - looking at the documentation I wrote for the flat frame feature, I specifically mention that a 'fast CPU' is required for real time flat frame correction of images at USB3 speeds (see https://docs.sharpcap.co.uk/4.1/#Using% ... Correction)

cheers,

Robin

[astra]

UI thread of the application *and* the total CPU load

I think, that's the problem. From my observations, "Thread CPU load" doesn't correlate at all with actual process and overall cpu load at all. Please see attached screenshots. Why UI thread only and not CurrentProcess cpu load?

Yeah, I've. I've enabled flat frame processing. It works fine while observing and handles 5-7fps fine. While capturing and writing I get 70% load. So I assume you process flats that are displayed only or written to the disk?

[admin]

Hi,

it is supposed to pick the higher of the thread load and the total CPU load. I do wonder if on your system the CPU part of it isn't working, so it is always just the thread load.

In the next update I have introduced new ways to measure both metrics that should be more reliable and give better results - we will have to see if they work in practice.

cheers,

Robin

[astra]

Thanks! I'll be happy to be a beta tester for you!

Can I please ask you for another request? Would you mind to adjust PID alg coefficients to react more quickly? The reason for is is then capture is started, I'd like to lower display fps as fast as possible. But it gets low closer to the end of capture of 600-800 frames.

[admin]

Hi,

just to let you know that the latest update (4.1.12395) contains updates to the code used to read both thread and whole system load level that may help keep things running more smoothly. There is also more detailed logging of the two load levels in the main log when 'performance' extra logging is enabled.

I haven't changed the PI parameters at this stage - I want to see how things work with good data going into the algorithm (presuming we can get to that stage) before tinkering with those.

thanks,

Robin