Custom frame transform?

Discussions on extending SharpCap using the built in Python scripting functionality
furrysocks2
Posts: 14
Joined: Wed Jan 03, 2018 1:28 pm

Custom frame transform?

#1

Post by furrysocks2 »

I have implemented Adaptive Digital Pixel Binning in C++ and wondered whether I am able to port to Python and have SharpCap apply to the previewed image?
User avatar
admin
Site Admin
Posts: 13122
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Custom frame transform?

#2

Post by admin »

eek,

you can't change the resolution of the frame in a frame transform or in response to any of the events on the Camera class (FrameCaptured, BeforeFrameDisplay). If you are happy to run the algorithm and then leave the image at the same resolution either with 2x2 pixel blocks or only the image in the top-left corner then maybe (just maybe) you can hang off of the FrameCaptured event.

If you are interested in that approach then I can supply more details,but it's on the bleeding edge of what is possible!

cheers,

Robin
furrysocks2
Posts: 14
Joined: Wed Jan 03, 2018 1:28 pm

Re: Custom frame transform?

#3

Post by furrysocks2 »

:)

Conveniently, ADPB maintains spatial resolution - there is no change in image size, just pixel values.

https://stargazerslounge.com/topic/3022 ... l-binning/


I'd be interested to see what's possible in SharpCap. It may be scripted performance that prevents it.
User avatar
admin
Site Admin
Posts: 13122
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Custom frame transform?

#4

Post by admin »

Well, it may be possible to load a native DLL and call it from the scripting (I've never tried, but you can do it from .NET and IronPython is based on .NET), so that would make the performance less of an issue - and save having to re-write your algorithm in Python

Here's the stuff you will need to know to have a chance of intercepting frames as they are captured. This is all in C#, so will need translating to IronPython syntax.

On the Camera object, hook the FrameCaptured event

Code: Select all

public event EventHandler<FrameCapturedEventArgs> FrameCaptured;

public class FrameCapturedEventArgs : EventArgs
{
    public BufferFrame Frame { get; set; }
}
This lets you get the frame object for each captured frame before it is passed to any processing code

Important parts of buffer frames

Code: Select all

var frame = eventArgs.Frame;
frame.Info //  useful stuff like width, height, bits per pixel, etc
frame.BufferSize // size of the memory buffer holding the image data

var lease = frame.GetBufferLease()
byte* frameData = lease.Buffer.ToPointer()
// do stuff with data
lease.Dispose()
// don't try to access frame buffer after disposing of lease
So as you can see you can basically get the memory buffer for the frame and then do what you need to it in place.

Please note that these events and methods are not part of the main, fairly stable, API available for scripting, so may change without warning in new versions.

cheers,

Robin
furrysocks2
Posts: 14
Joined: Wed Jan 03, 2018 1:28 pm

Re: Custom frame transform?

#5

Post by furrysocks2 »

Thanks, Robin.

Yes, calling out to native code was a thought. Hooking the event works, I'll try accessing and modifying the frame data.

Code: Select all

>>> def handler(sender, args):
...     print "got frame";
... 
>>> SharpCap.SelectedCamera.FrameCaptured += handler;
>>> got frame
got frame
got frame

I did get an "Operation could destabilise the runtime..." error pop up just now, though.
furrysocks2
Posts: 14
Joined: Wed Jan 03, 2018 1:28 pm

Re: Custom frame transform?

#6

Post by furrysocks2 »

I'm uncertain how to interact with the <System.Reflection.Pointer> object that I get from lease.Buffer.ToPointer().

dir(frameData) give me the following:
['Box', 'Equals', 'GetHashCode', 'GetObjectData', 'GetType', 'MemberwiseClone', 'ReferenceEquals', 'ToString', 'Unbox', '__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']
User avatar
admin
Site Admin
Posts: 13122
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Custom frame transform?

#7

Post by admin »

Hmm,

try

Code: Select all

lease.Buffer.ToInt32()
instead - that will give you the memory address in an integer. If there is a 64 bit version of SharpCap in future then you would need ToInt64()

cheers,

Robin
furrysocks2
Posts: 14
Joined: Wed Jan 03, 2018 1:28 pm

Re: Custom frame transform?

#8

Post by furrysocks2 »

This works for me...

Code: Select all

def handler( sender, eventArgs):
	frame          = eventArgs.Frame;
	bufsize        = int(frame.BufferSize)
	lease          = frame.GetBufferLease()
	frameData_addr = lease.Buffer.ToInt64()
	frameData_BUF  = (ctypes.c_char * bufsize).from_address(frameData_addr)

	# ... modify frameData_BUF[N]

	lease.Dispose()
Presumably, ToInt64() works as it will currently return Int32-range addresses.
User avatar
admin
Site Admin
Posts: 13122
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Custom frame transform?

#9

Post by admin »

Yes, ToInt64() will be fine.

Did you manage to load and use your C++ DLL?

cheers,

Robin
furrysocks2
Posts: 14
Joined: Wed Jan 03, 2018 1:28 pm

Re: Custom frame transform?

#10

Post by furrysocks2 »

No, not yet.
Post Reply