New Feature : SharpSolve - SharpCap's new built-in plate solver

[Menno555]

Hi Robin

Finally had the chance to test this with the latest version ... and it works great and indeed lightning fast!
Even with my small FOV is works just fine. Tested it with the Folder Monitor Camera on multiple captures made with FL 2100mm (FOV 0,6448x0,4283 degrees) and no problems at all.
Testing it for real has to wait for a while though: now a storm hitting and predictions for coming week(s) is really no good.

Menno

[carlomuccini]

This morning I tested SharpSolving on numerous photos, including very old ones (many cameras and many different scope), file raw FITs, processed FITs, JPG and PNG files, leaving the telescope focal length box unchecked.
It takes two seconds more but the result is FORmidable.

Congratulations, I don't know how you did it, but it works really well!

Carlo

[soulearth]

I'm looking into the possibility of a downloadable extended plate solving index that would take the minimum field of view down to 0.33 degrees or maybe even 0.25 degrees.

Hi,
Very good news !
Observing distant galaxies does not necessarily require a very wide field. Many of us use relatively small sensors. I have an imx290 which gives me a field of 0.49° x 0.28°. Downloading a large catalog is not a problem when you really need it.

The D50 catalog from astap weighs 800MB and is sufficient up to 0.2°. It's not huge either.

Regards,

[OregonSky]

+1 for 0.25 FOV .
My 6Se with .reducer and Player One Uranus C camera is 0.34 and my 11SCT with same camera is 0.22.

I have to use binning X 2 to get fairly consistent solving with ASTAP now. ( D80 database )
Would be so nice to have it work right within Sharpcap .

Thanks so much - You're Amazing !!
MIKE

[admin]

Hi,

I had some promising results testing yesterday with a database that comes in at about 160Mb. I think that will give me >99% success rate at 0.25x0.25 with a bit of luck (running a full test of 300 large frames and all sub-areas at 0.25x0.25 takes a long time!).

I'm currently thinking that the file for that is too big to install with SharpCap, but I could have a button to download it in the plate solve settings.

cheers,

Robin

[Jean-Francois]

Hello Robin,

Nice.
Please think to the people using off-line computer system.
So, the file has to be "simple" to find and easy to copy on the off-line computer.
And when you finish the 0.25° database, then you can start the 0.1° database

One question ... was it not possible to use the database from ASTAP or other software ?
Maybe a reason of copyright, but it could be possible to ask the permission.

Regards,
Jean-Francois

[han59]

Hello Robin,

I have experimented with your SharpSolve in simulation. It works very well. Congratulation with the very good performance!

Out of curiosity I have experimented with the SharpSolve performance at very short exposure times v.s. ASTAP (my creation). They perform very similar. So the star detection is good but I noticed it extracts more stars out of noisy images. Does it use quads (4 star figures) for a hash code creation? In ASTAP you can switch from quads to triplets but that gives only some improvement for noisy images but slows done for star rich images and it usage it not recommended.


Sometimes when SharpSolve fails to solve it doesn't report solve failure with a red text window. The yellow window with "Solving: Searching at 21.4 degrees......" stays:

See this log:

Code: Select all

[size=85]Debug   15:18:25.348458 #18 Searching at 20.4 degrees from RA=23:03:05,Dec=+47:08:39                                                                in void SharpCap.ImageProcessing.PlateSolving.SharpSolvePlateSolver._tileSolver_Progress(object sender, GenericEventArgs<MultiTileProgress> e)
Debug   15:18:25.366711 #26 Searching at 20.6 degrees from RA=23:03:05,Dec=+47:08:39                                                                in void SharpCap.ImageProcessing.PlateSolving.SharpSolvePlateSolver._tileSolver_Progress(object sender, GenericEventArgs<MultiTileProgress> e)
Debug   15:18:25.381196 #10 Searching at 20.6 degrees from RA=23:03:05,Dec=+47:08:39                                                                in void SharpCap.ImageProcessing.PlateSolving.SharpSolvePlateSolver._tileSolver_Progress(object sender, GenericEventArgs<MultiTileProgress> e)
Info    15:18:25.381416 #10 Plate solving : Searching at 20.6 degrees from RA=23:03:05,Dec=+47:08:39                                                in Task<bool> SharpCap.ViewModels.PlateSolveAndResync.SolveImplAsync(bool gotFrame, IFrameToSolve tmpFile, CancellationTokenSource tokenSource, IMount model, SizeF? fieldOfView, PostSolveActions action)+(object _, SolveProgressEventArgs a) => { }
Debug   15:18:25.416992 #18 Searching at 21.0 degrees from RA=23:03:05,Dec=+47:08:39                                                                in void SharpCap.ImageProcessing.PlateSolving.SharpSolvePlateSolver._tileSolver_Progress(object sender, GenericEventArgs<MultiTileProgress> e)
Debug   15:18:25.676415 #1  Notification (Status=Error): Plate solve failed due to : Failed to find solution                                        in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Debug   15:18:25.677721 #1  Notification (Status=Warning): Solving: Searching at 17.7 degrees from RA=23:03:05,Dec=+47:08:39                        in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Debug   15:18:25.678037 #1  Notification (Status=Warning): Solving: Searching at 20.6 degrees from RA=23:03:05,Dec=+47:08:39                        in void SharpCap.UI.NotificationViewModel.DisplayMessage(NotificationMessage message)
Debug   15:19:44.935719 #1  User clicked on notification button Cancel                                                                              in void SharpCap.UI.NotificationViewModel.HandleButtonPress(NotificationButton button)
Warning 15:19:44.936432 #1  Cancellation token source already disposed? : Exception of type 'ObjectDisposedException' : The CancellationTokenSource has been disposed. 
Stack Trace:   at void System.Threading.CancellationTokenSource.Cancel()
   at void SharpCap.Base.ExtensionMethods.SafeCancel(CancellationTokenSource cts) in C:/Documents/Source Code/SharpCap/src/SharpCap.Base/ExtensionMethods.cs:line 127                    in void SharpCap.Base.ExtensionMethods.SafeCancel(CancellationTokenSource cts)[/size]

Thanks, Han

[admin]

Hi Jean-Francois,

0.1 degree - you're kidding right!

Good point on the off-line installation - I will think about a way to make that easy (perhaps a small installer?).

Right now I am stuck because the 300x10s test images I took a week or two back aren't long enough exposures to be sure of getting a decent selection of stars when I cut them down to ROIs below about 0.3 to 0.4 degrees. I can't tell if the problems I am having getting things to work reliably at smaller scales are down to that problem or something more fundamental.

With luck I will have some clear sky Monday or Tuesday and I will set up to take some test images at 20s, 30s and 60s of a fairly decent size patch of sky. Hopefully I will get better results with the longer exposures, which would indicate that my code and database are working OK but just need a better selection of stars from the image.

The issue with using the Astap data is that I am pre-processing the star positions and pre-selecting the stars in a different way to Astap. The Astap D50 database runs to about 900Mb when installed at 5 bytes per star (180 million stars). I believe it contains RA, Dec and magnitude for each star in a slightly compressed form.

I'm estimating that I will need about 40 million stars at 4 bytes per star to get to 0.25 degrees (~10 million stars gets me to 0.5 degrees). It takes about 10 minutes of CPU time to built my index file from a large source list of ~120 million stars, but only I have to do that - once the index is built it's quick to use because most of the spherical geometry calculation has been pre-calculated.

cheers,

Robin

[admin]

Hi Han,

glad you like it I have been wanting to write this code for about 6 months while I was finishing SharpCap 4.1 and updating the documentation - I knew that I had a lot of the solution already due to the polar alignment and wanted to find out if I could make the whole sky system work.

As I explained to Jean-Francois above, I think the speed is helped by the fact that I am doing quite a bit of the spherical geometry calculations before writing the index. One of the biggest parts of what is left is calculating the invariant properties (hash code values) of the quads - this requires more 2D geometry than I think is strictly necessary. I do have an idea for an alternate way to get the hash codes calculated, but haven't had a chance to try it out yet.

Star detection is fortunately something that I have had for a long time in SharpCap for other features (polar alignment, live stacking, focus detection, etc), so I had code for that ready to use. I am still working on fine tuning the sensitivity for best results - I might need to add an end-user adjustment in the end for the sensitivity.

I do have some triangle alignment code in SharpCap that is still used for live stacking alignment - it works, but does produce false positives at a higher frequency than the quad alignment, so I only use it for live stacking where the issue can be dealt with easily (there is always likely to be a good match and you can filter to only look at matches that have a scaling factor of ~1).

I think that the issue that you see with the failure message not showing is down to the use of multiple threads (up to 4) to parallelize the solving. Each thread sends a notification to the UI when it finishes checking a particular area of the sky, and that becomes the 'Searching at ...' message. Sometimes one of those message gets displayed after the 'Plate solve failed' message, so the failed message is lost. I think I have fixed this in 4.1.11239 by synchronizing the way that the messages are forwarded from the worker threads to the UI.

cheers,

Robin

PS. Do you include non-stars in your database (I am mostly thinking of small objects that could be mistaken for stars like distant galaxies). I can't decide if this would be a good idea or not...

[han59]

As I explained to Jean-Francois above, I think the speed is helped by the fact that I am doing quite a bit of the spherical geometry calculations before writing the index. One of the biggest parts of what is left is calculating the invariant properties (hash code values) of the quads - this requires more 2D geometry than I think is strictly necessary. I do have an idea for an alternate way to get the hash codes calculated, but haven't had a chance to try it out yet.

In ASTAP the stars database contains the RA, DEC and magnitude. The magnitude is not really required since the star are sorted in magnitude. From the RA, DEC equatorial coordinates ASTAP calculates the standard coordinates according the rigid method. Maybe it could be simplified by just correcting the RA by the cos(Dec), I never tried. But I assume you will get quicker problems at the celestial pole.

For years I have considered to preprocess the stars directly in hash codes. I assume the database will grow significantly but if sorted/indexed maybe blind solve and normal solve will be executed in about the same speed. Likely then the FOV doesn't play a role either. The argument against it is that blind solve is rare.

Star detection is fortunately something that I have had for a long time in SharpCap for other features (polar alignment, live stacking, focus detection, etc), so I had code for that ready to use. I am still working on fine tuning the sensitivity for best results - I might need to add an end-user adjustment in the end for the sensitivity.

In my experience there is somewhere an optimum for star detection sensitivity. Too sensitive the solve success goes down due to too many ghost stars. In my program the limit is set at SNR 7 but it depend probably on the SNR definition.

I do have some triangle alignment code in SharpCap that is still used for live stacking alignment - it works, but does produce false positives at a higher frequency than the quad alignment, so I only use it for live stacking where the issue can be dealt with easily (there is always likely to be a good match and you can filter to only look at matches that have a scaling factor of ~1).

Yes quads seems to be the best setup. According some old Astrometry.net documentation using 5 stars figures results in more processing time and did not bring any real benefits.

For my documented setup I use the distances between the four stars. Astrometry.net uses a very different (complicated) quad hash code. I assume they can do that because their hash codes are preprocessed and If I remember well are stored in FITS files.

I think that the issue that you see with the failure message not showing is down to the use of multiple threads (up to 4) to parallelize the solving. Each thread sends a notification to the UI when it finishes checking a particular area of the sky, and that becomes the 'Searching at ...' message. Sometimes one of those message gets displayed after the 'Plate solve failed' message, so the failed message is lost. I think I have fixed this in 4.1.11239 by synchronizing the way that the messages are forwarded from the worker threads to the UI.

How do you split the threads? Are they all doing the same but a different part of the sky? I assume it depends if the star database is preloaded in memory. Then you can split the sky in e.g four parts. If the database is not preloaded in memory using threads is likley more complicated since all running threads will like to access the database on disk. I assume the star database of SharpSolve is the file psindex.bin and preloaded? Threaded processing is still on my wish list but I'm not sure if it will be much beneficial for short offsets.

ps. I use GAiA DR3 for the star database. The UCAC4 is fine but doesn't go that deep.

PS. Do you include non-stars in your database (I am mostly thinking of small objects that could be mistaken for stars like distant galaxies). I can't decide if this would be a good idea or not...

In the past I tried to avoid detecting small galaxies in the image by the ovality. I practice it didn't bring a benefit. Currently I just assume there are much more stars then galaxies detected and the galaxies do not disturb the solving process.

With Gaia (space telescope), the star density can be much higher then the UCAC4 (earth based telescope). This could be a problem for images where M13 fills most of the images but will be less of a problem for a database with a specific star density like you have now.


Cheers, Han