Got a bit frustrated that theoretical full size image frame rates seemed - in practice - to be only a quarter or less of that advertised ( e.g. 11-14 fps rather than 45 fps) for e,g, the ASI 715MC. So my lunar images have comprised less than ideal total frame numbers.
Having investigated all the usual things --swapping out USB 3.0 cables, updating the drivers, checking all the SC settings (fast mode on, USB set to 100) and shutting down every conceivable background process on my imaging laptop (an old Celeron i5 with 8 Gb RAM and 500Gb SSD) still no change.
Then I installed SC on my fast 64 Gb Ram desktop, 12th Gen Intel(R) Core(TM) i7-12700F (2.10 GHz) with 4 Gb SSD. Immediately got 45 fps as advertised with no dropped frames.
I don't know what the critical computer difference is? According to the Hardware device manager the laptop ports are USB 3.1. Maybe they don't work so well as they should (drivers?) or the problem lies in writing to the older SSD?
Anyway for lunar and planetary it seems that a better computer is the way to go
Tim
Frame dropping in lunar imaging. It's the computer!
Re: Frame dropping in lunar imaging. It's the computer!
I also found this on Cloudy nights which was useful. The problem seems to be the SSD storage speed and capacity
https://www.cloudynights.com/forums/top ... otography/
https://www.cloudynights.com/forums/top ... otography/
-
admin
- Site Admin
- Posts: 17363
- Joined: Sat Feb 11, 2017 3:52 pm
- Location: Vale of the White Horse, UK
- Contact:
Re: Frame dropping in lunar imaging. It's the computer!
Hi Tim,
The first question I guess is if this happens purely during capture, or all the time (when capturing *and* when just viewing the image). If it's the former then the focus has to be how quickly the data can be saved to SSD/Disk. If it's the latter, it will be down to CPU power...
Celeron processors were always Intel's 'cheap, low speed, low power consumption' models, at least in recent years, so if your normal imaging laptop is a Celeron then it may well struggle with high frame rates. Usually I would have expected to see dropped frames get recorded in the situation where the PC cannot keep up with the camera, but that would depend on the ZWO SDK reporting the dropped frames to SharpCap, since they would be being lost at the USB/Driver/SDK level.
Back in the days of USB2 planetary cameras with roughly 1 megapixel resoultion (ASI120, etc), a low spec PC could keep up, but newer cameras like the 715 can send through data at a much faster rate.
cheers,
Robin
The first question I guess is if this happens purely during capture, or all the time (when capturing *and* when just viewing the image). If it's the former then the focus has to be how quickly the data can be saved to SSD/Disk. If it's the latter, it will be down to CPU power...
Celeron processors were always Intel's 'cheap, low speed, low power consumption' models, at least in recent years, so if your normal imaging laptop is a Celeron then it may well struggle with high frame rates. Usually I would have expected to see dropped frames get recorded in the situation where the PC cannot keep up with the camera, but that would depend on the ZWO SDK reporting the dropped frames to SharpCap, since they would be being lost at the USB/Driver/SDK level.
Back in the days of USB2 planetary cameras with roughly 1 megapixel resoultion (ASI120, etc), a low spec PC could keep up, but newer cameras like the 715 can send through data at a much faster rate.
cheers,
Robin
Re: Frame dropping in lunar imaging. It's the computer!
Tim
Summary:
Summary:
- For the tests, use the shortest USB3 lead you have, connect the camera directly to the laptop.
- Use USBTrreeView to check the speed the device is actually connecting at.
- Run the hard drive speed test from within Sharpcap. File > Sharpcap Settings > Filenames tab > Speed Test button
- Use Disk Defrag to analyse the drive and note if Trim is being used or when it was last run.
- Use Disk Defrag to Optimise (defrag & trim) the drive.
- Run the hard drive speed test from within Sharpcap.
- Test with various capture areas to see the effect on fps.
- Plug an external USB3 drive into the port and perform a large file transfer from the laptop to the external and vice versa. Note the green graph in the file transfer window.
I did a test with my Skyris 236M USB3 camera plugged into a USB2 port, then a USB3 port - with exposure set to 10ms (typical lunar/planetary capture) and capture area = 1920x1200 (full sensor).
In the USB2 port the returned speed was 15fps, in the USB3 port the rate was 54fps. It 'feels' as though the port on your laptop is running at USB2 speeds.
Using USBTreeView https://www.uwe-sieber.de/usbtreeview_e.html
By clicking on the device, an H or S is displayed which shows the speed the device is connected at. H = High Speed (USB2), S = SuperSpeed (USB3)
- Skyris236M_in_USB2_port.JPG (23.14 KiB) Viewed 398 times
- Skyris236M_in_USB3_port.JPG (17.04 KiB) Viewed 398 times
Clicking on the device will show 'far more than you ever wanted to know' in the right hand pane. A complete USB info report can be saved via the menu.
My capture laptop is a 10 year old i3 machine with 16Gb RAM and a SATA2 5400rpm drive. Because of the use of the laptop - constant capturing and deletion of big files - I elected not to upgrade the drive to an SSD. That way no chance of getting caught out with Trim. Not trimming an SSD after big file deletion can lead to slow performance. I just bought a 1Tb drive as an upgrade but have stayed with a mechanical drive.
My report from an Analyse run, note the SSD drives are being 'trimmed'.
- Analyze_drive.JPG (54.85 KiB) Viewed 398 times
For a large file (9Gb) copy from USB3 external to SSD internal. Note the dip, indicative that some drive maintenance is needed.
- large_file_copy.JPG (21.48 KiB) Viewed 398 times
On my capture laptop, the SharpCap Speed Test returned 109fps with a Write Speed of 129Mb/s.
Hopefully something may come out of this. Maybe working at a reduced capture area will give you the desired speed increase. On my 183C (over USB3)- 5440x3648 = 9fps, 3648x3648 = 15fps, 2720x1824 and below = 35fps.
Dave
Re: Frame dropping in lunar imaging. It's the computer!
Thanks very much Robin and Dave! It's going to take me a little time to absorb but on first reading makes sense.
I should clarify that the laptop storage is a 500Gb SSD and not a hard drive - the desktop for which capture is at the expected rate has a 4Tb SSD - misprint above.
Firstly, in reply to Robin. Using the laptop the speeds of full frame captures are always about a quarter to a third of what they should be. But it seems frames are only dropped for larger/ longer captures and larger ROIs/ full frame. Cut down the ROI to less than a quarter of the full frame area and all is fune. The full frame FPS speed also varies. Consistent (I think) with the theory expounded on the Cloudy nights and with your suggestion that it is to do with saving onto the SSD the very first capture can be as fast as 15-17 fps -- but subsequent captures are always slower - going down to about 10 fps depending I think upon how long I leave it following the first capture - so that would be consistent with needing to wait several minutes before the RAM and SSD buffer capacity has cleared?
Dave thanks very much for those ideas. I didn't know about trimming and - in fact- I've never done it. It seems a very sensible thing to do anyway. It also seems quite feasible that my USB 3.1 hub and ports may not in fact be performing as such and so thanks for the info on how to monitor their speed --useful to know.
As you mention - when I reduce the down to a smaller ROI and generate smaller files the fps goes up proportionately and for planets (640 x 480) for example the observed speeds increase up to speeds about as expected. For lunar though I always find the bigger fields more attractive - more context somehow - so I'd like to get that faster.
So I'll play around and report back -- either some of the above will be enough or yet more expense on something.
thanks again
Tim
I should clarify that the laptop storage is a 500Gb SSD and not a hard drive - the desktop for which capture is at the expected rate has a 4Tb SSD - misprint above.
Firstly, in reply to Robin. Using the laptop the speeds of full frame captures are always about a quarter to a third of what they should be. But it seems frames are only dropped for larger/ longer captures and larger ROIs/ full frame. Cut down the ROI to less than a quarter of the full frame area and all is fune. The full frame FPS speed also varies. Consistent (I think) with the theory expounded on the Cloudy nights and with your suggestion that it is to do with saving onto the SSD the very first capture can be as fast as 15-17 fps -- but subsequent captures are always slower - going down to about 10 fps depending I think upon how long I leave it following the first capture - so that would be consistent with needing to wait several minutes before the RAM and SSD buffer capacity has cleared?
Dave thanks very much for those ideas. I didn't know about trimming and - in fact- I've never done it. It seems a very sensible thing to do anyway. It also seems quite feasible that my USB 3.1 hub and ports may not in fact be performing as such and so thanks for the info on how to monitor their speed --useful to know.
As you mention - when I reduce the down to a smaller ROI and generate smaller files the fps goes up proportionately and for planets (640 x 480) for example the observed speeds increase up to speeds about as expected. For lunar though I always find the bigger fields more attractive - more context somehow - so I'd like to get that faster.
So I'll play around and report back -- either some of the above will be enough or yet more expense on something.
thanks again
Tim
Shifting memory allocation in SC seems to help
1) In viewing mode SC reports the full frame rate as 45 fps (as per ASI 715MC specifications) when the camera is connected to one of the USB 3.0 (or 3.2 gen 1 which seems to mean to same thing) ports on the laptop.
2) Connecting via USB 2.0 the rate drops to about 4 fps. This confirms that the laptop USB 3 ports are working correctly at 10X faster than USB 2.
3) The sharpcap disk speed test reported that writing to the internal 0.5Tb SDD via the bus was fast (0.7Gb/ s) equivalent to 611 fps for a 1280x960 .ser file over 23 s. So the inherent write speed would not be limiting FPS capture speed even for full frames that are 7x larger (i.e. you might expect 87 fps for these?)
4) Capturing full frames (3684 x 2192) appeared just as fast as viewing (44.9 fps) over a 20s capture period. i.e. 901 frames (only 2 dropped) captured over a 20s capture with no buffering.
5) However over 40s of capture the apparent FPS rate dropped to 38 with 1524 frames captured (eventually after waiting for 250 frames to write from the buffer) and 280 frames dropped.
6) Over 80s the apparent FPS rate dropped dramatically to 12.4 fps with only 995 frames captured (again after waiting for 250 buffered frames to write) and 2610 frames dropped.
7) However altering the sharpcap memory allocation setting to maximise the write cache to 3 Gb things improved. Over 80s the apparent FPS rate dropped to only 38 fps with 3095 frames captured (after waiting for now 350 buffered frames to write) and only 548 frames dropped.
250 full frames - apparently the buffer capacity - corresponds to about 2 Gb - coincidentally also the expected fast write cache size for a 0.5 Tb SSD disk?.
So one limitation does appear to be the computer RAM buffer filling up which might be solved by adding more memory (16 Gb rather than 8) ?
There is also variability which I think is associated with the write speed slowing downs further as the SSD gets hotter and/or is still clearing the cache after too many close together successive preceding captures.
I also wonder whether there would be more headroom adopting a larger external SSD drive with a a larger onboard SSD fast write cache. After all - even a 2min full frame save worked perfectly well at 45fps without making any adjustments to the memory allocation of the computer RAM when the camera was connected to my desktop 4 Tb SSD. So that is another way. But for now I am pleased that simply shifting the memory allocation in Sharpcap looks to have made a significant improvement in the effective speed of writing over a longer period.
Tim
2) Connecting via USB 2.0 the rate drops to about 4 fps. This confirms that the laptop USB 3 ports are working correctly at 10X faster than USB 2.
3) The sharpcap disk speed test reported that writing to the internal 0.5Tb SDD via the bus was fast (0.7Gb/ s) equivalent to 611 fps for a 1280x960 .ser file over 23 s. So the inherent write speed would not be limiting FPS capture speed even for full frames that are 7x larger (i.e. you might expect 87 fps for these?)
4) Capturing full frames (3684 x 2192) appeared just as fast as viewing (44.9 fps) over a 20s capture period. i.e. 901 frames (only 2 dropped) captured over a 20s capture with no buffering.
5) However over 40s of capture the apparent FPS rate dropped to 38 with 1524 frames captured (eventually after waiting for 250 frames to write from the buffer) and 280 frames dropped.
6) Over 80s the apparent FPS rate dropped dramatically to 12.4 fps with only 995 frames captured (again after waiting for 250 buffered frames to write) and 2610 frames dropped.
7) However altering the sharpcap memory allocation setting to maximise the write cache to 3 Gb things improved. Over 80s the apparent FPS rate dropped to only 38 fps with 3095 frames captured (after waiting for now 350 buffered frames to write) and only 548 frames dropped.
250 full frames - apparently the buffer capacity - corresponds to about 2 Gb - coincidentally also the expected fast write cache size for a 0.5 Tb SSD disk?.
So one limitation does appear to be the computer RAM buffer filling up which might be solved by adding more memory (16 Gb rather than 8) ?
There is also variability which I think is associated with the write speed slowing downs further as the SSD gets hotter and/or is still clearing the cache after too many close together successive preceding captures.
I also wonder whether there would be more headroom adopting a larger external SSD drive with a a larger onboard SSD fast write cache. After all - even a 2min full frame save worked perfectly well at 45fps without making any adjustments to the memory allocation of the computer RAM when the camera was connected to my desktop 4 Tb SSD. So that is another way. But for now I am pleased that simply shifting the memory allocation in Sharpcap looks to have made a significant improvement in the effective speed of writing over a longer period.
Tim
Re: Frame dropping in lunar imaging. It's the computer!
Tim
Those tests validate your USB architecture and your settings. 2 things remain:
With the Skyris 236M (1920x1200) I capture 2000 frames @ 54fps which takes ~35s to complete. The results from those captures seem decent.
Dave
Those tests validate your USB architecture and your settings. 2 things remain:
- Run the disk defrag/optimise to ensure Trim has been done and retest capture speeds.
- Memory upgrade
With the Skyris 236M (1920x1200) I capture 2000 frames @ 54fps which takes ~35s to complete. The results from those captures seem decent.
Dave
Re: Frame dropping in lunar imaging. It's the computer!
Thanks Dave,
Forgot to say that, following your suggestion, I did indeed run optimise and trim on the SSD and that no doubt sped things up - and also a disk clean up. Defrag didn't really seem to be recommended for the SSD? something about reducing it's life?
Hmm - the memory upgrade to 16 Gb - yes it's the logical thing to do and I did it on the desktop. Dare I do the same on the laptop -- a bit more fiddly.
Yes indeed your images are certainly more than decent! And maybe enough speed improvement here already for me ? After all three thousand 3684 x 2192 frames over about a minute is a three fold improvement over where things were. A bit embarrassing in the end though that it was just a question of knowing Sharpcap better - after all these years - just clean forgot about the memory settings.
Tim
Forgot to say that, following your suggestion, I did indeed run optimise and trim on the SSD and that no doubt sped things up - and also a disk clean up. Defrag didn't really seem to be recommended for the SSD? something about reducing it's life?
Hmm - the memory upgrade to 16 Gb - yes it's the logical thing to do and I did it on the desktop. Dare I do the same on the laptop -- a bit more fiddly.
Yes indeed your images are certainly more than decent! And maybe enough speed improvement here already for me ? After all three thousand 3684 x 2192 frames over about a minute is a three fold improvement over where things were. A bit embarrassing in the end though that it was just a question of knowing Sharpcap better - after all these years - just clean forgot about the memory settings.
Tim
Re: Frame dropping in lunar imaging. It's the computer!
Tim
Laptop memory upgrade - most laptops have a panel on the underside, allowing access to the memory slots. I have come across models where the keyboard had to be removed. The increase memory will help providing the SharpCap memory parameters are adjusted.
Looks like there will be lots of lunar images coming our way.
Dave
Google 'ssd trim reduced life'.Defrag didn't really seem to be recommended for the SSD? something about reducing it's life?
Laptop memory upgrade - most laptops have a panel on the underside, allowing access to the memory slots. I have come across models where the keyboard had to be removed. The increase memory will help providing the SharpCap memory parameters are adjusted.
Looks like there will be lots of lunar images coming our way.
Dave
Re: Frame dropping in lunar imaging. It's the computer!
Yes all we need now is the moon Dave 
Just to sort of complete this thread there is another solution that works too irrespective of the sharpcap memory settings or adding more onboard RAM.
I connected a 2 Tb USB external SSD to one of the USB 3 ports and made that the default directory for sharpcap to write to. The write speed was slower (0.35 Gb/s) than to the internal SSD which is connected to the bus but still adequate to keep up with the camera (which is of course also connected via USB 3.1) and the bigger 2 Tb size seemed to mean that it's write cache didn't get choked up and so it stayed fast even after writing a 90s 44 Gb 4400 full frame file at 45 fps with negligable dropped frames and no buffering. That is way more than I am ever likely to need. External SSD is probably going to be a better faster storage solution overall compared to my current practice of writing to an ever decreasing amount of space on the computer internal SSD, syncing to one drive and then continually removing data back off of onedrive for long term storage on a growing collection of hard disks.
Tim
Just to sort of complete this thread there is another solution that works too irrespective of the sharpcap memory settings or adding more onboard RAM.
I connected a 2 Tb USB external SSD to one of the USB 3 ports and made that the default directory for sharpcap to write to. The write speed was slower (0.35 Gb/s) than to the internal SSD which is connected to the bus but still adequate to keep up with the camera (which is of course also connected via USB 3.1) and the bigger 2 Tb size seemed to mean that it's write cache didn't get choked up and so it stayed fast even after writing a 90s 44 Gb 4400 full frame file at 45 fps with negligable dropped frames and no buffering. That is way more than I am ever likely to need. External SSD is probably going to be a better faster storage solution overall compared to my current practice of writing to an ever decreasing amount of space on the computer internal SSD, syncing to one drive and then continually removing data back off of onedrive for long term storage on a growing collection of hard disks.
Tim