How to access on the "cutout" data ?

[Jean-Francois]

Hello Robin,

I have a script using one of your small code (here with some modification rom me).

Code: Select all

def framehandler(sender, args):
    global i, T_meas, W_meas, t0
    if (dumpdata):
        dt = (datetime.datetime.now() - t0).total_seconds()
        cutout = args.Frame.CutROI(SharpCap.Transforms.SelectionRect)
        Stat = cutout.GetStats()
        mean = Stat.Item1
        T_meas.append(dt)
        W_meas.append(mean)
        i = i + 1
        cutout.Release()

Normally it is possible to see the available functions in the console.
But here for the "cutout", I do not find a way to find how to access the data.

Is it possible to perform "image A + image B" directly ? or it is necessary to iterate over all the pixels ?
I wish to calculate the average image of a sequence and later perform some interpolation on the 2D image pixels.

Regards,
Jean-Francois

[admin]

Hi Jean-Francois,

not quite sure why the intellisense is not working, as the cut out is just another frame object (should have the same methods available on it as the original frame).

In particular, you might find these useful

Code: Select all

frameA.Multiply(scaleFactor);
// and ...
frameA.Add(frameB, scaleFactorForB);

If you use a scale factor of -1 in the second then you are effectively subtracting B from A, leaving the result in A. If you use a scale factor of 1, then it is A=A+B

hope this helps,

Robin

[Jean-Francois]

Hello,

OK, thanks.

And how can I generate a frame object ?
I know how to generate a bitmap object ... but the function ".Add" does not work with a bitmap and the cutout buffer.

Jean-Francois

[admin]

Hi Jean-Francois,

probably easiest to take an existing frame (like you get from the event) and then call .Clone() on it to make a copy. The copy will have the same size, bit depth and data, but you can reset the data from an array using frame.CopyFrom(array) (the array should be the right size).

cheers,

Robin

[Jean-Francois]

Hello Robin,

No, it is not the easiest way ... I have the following code:

Code: Select all

# *******************************************************************************************
# SharpCap script "WIN_Transmission.py"
# 2024/08/08 Jean-Francois
#
# Version: 1.0.0:   First version
#
# The script works only with a SharpCap PRO version.
# *******************************************************************************************

import os
import time
import datetime
import math
import clr
clr.AddReference("Oxyplot")
clr.AddReference("OxyPlot.WindowsForms ")
clr.AddReference("System.Drawing")
clr.AddReference("System.Windows.Forms")

import OxyPlot
import System.Drawing
import System.Windows.Forms

from SharpCap.Base import BufferFrame

from System import Array
from System.Drawing import *
from System.Windows.Forms import *
from System.Threading import Thread, ApartmentState, ParameterizedThreadStart

i = 1
T_meas = []
W_meas = []
t0 = 0
image = 0
frameA = 0
pos = 0


def framehandler(sender, args):
    global i, T_meas, W_meas, t0, frameA, pos
    if (dumpdata):
        try:
            dt = (datetime.datetime.now() - t0).total_seconds()
            cutout = args.Frame.CutROI(SharpCap.Transforms.SelectionRect)
            Stat = cutout.GetStats()
            mean = Stat.Item1
            T_meas.append(dt)
            W_meas.append(mean)
            i = i + 1
            cutout.Release()
        except:
            print("Problem")


# *****************************************************************************
def WIN_measurement(self):
    global dumpdata
    global i, T_meas, W_meas, t0, frameA

    self.WIN_Monitoring.Text = "Monitoring running"
    self.WIN_Monitoring.BackColor = Color.Red
    self.WIN_Monitoring.Enabled = False

    SharpCap.Transforms.SelectionRect = Rectangle(300, 300,600, 600)

    i = 1
    T_meas = []
    W_meas = []

    expos_ms = SharpCap.SelectedCamera.Controls.Exposure.ExposureMs
    wait_time = 0.1       # waiting time between 2 refresh of Oxyplot graphic

    print("Waiting Time: %5.3f" % (wait_time))
    print()
    print("WIN Transmission:")
    print("******************")

    dumpdata = True
    SharpCap.SelectedCamera.FrameCaptured += framehandler

    try:
        if (float(self.textBoxTime.Text) > 0.0):
            Max_time = float(self.textBoxTime.Text) - wait_time
            t0 = datetime.datetime.now()
            dt = 0

            while  (dt < Max_time):
                dt = (datetime.datetime.now() - t0).total_seconds()
                time.sleep(wait_time)
                self.ShowOxyGraphic()

            print("Time:", T_meas)
            print("WIN:", W_meas)
            print()
            print("Number: ", i - 1)

            mean = sum(W_meas) / len(W_meas)
            var  = sum(pow(x-mean,2) for x in W_meas) / len(W_meas)
            std  = math.sqrt(var)
            print("Standard deviation 1: " + str("%.6f" % round(std,6)))
        else:
            print("Measurement time need to be larger than 0 s")
    except:
        print("Problem 1, Problem 1, Problem 1, Problem 1, Problem 1")

    dumpdata = False
    SharpCap.SelectedCamera.FrameCaptured -= framehandler

    print()
    print("Measurement done")

    self.WIN_Monitoring.Text = "Start Measurement"
    self.WIN_Monitoring.BackColor = Color.Gainsboro
    self.WIN_Monitoring.Enabled = True

    file_path  = os.path.dirname(__file__)
    file_name = file_path + "\\" + self.textBoxFileName.Text + ".txt"
    file = open(file_name, "w")
    for index in range(len(T_meas)):
        file.write(str("%.4f" % T_meas[index]) + " " + str("%.4f" % W_meas[index]) + "\n")
    file.close()

    return()

# *****************************************************************************

class MonitoringWINMenuForm(Form):
    def __init__(self):
        self.SuspendLayout()
        self.InitializeComponent()
        self.setupCheckButtons()
        self.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Dpi
        self.AutoScaleDimensions = SizeF(96, 96)
        self.ResumeLayout()

    def InitializeComponent(self):
        self.Text = "WIN Transmission"
        self.ClientSize = System.Drawing.Size(700, 400)
        self.TopMost = True

        self.labelTime = Label()
        self.textBoxTime = TextBox()
        self.labelFile = Label()
        self.textBoxFileName = TextBox()

        self.plot1 = OxyPlot.WindowsForms.PlotView()
        self.plot1.Location = System.Drawing.Point(10, 35)
        self.plot1.Size = System.Drawing.Size(680, 350)
        self.plot1.AutoSize = True

        self.Controls.Add(self.labelTime)
        self.Controls.Add(self.textBoxTime)
        self.Controls.Add(self.labelFile)
        self.Controls.Add(self.textBoxFileName)
        self.Controls.Add(self.plot1)

        self.labelTime.AutoSize = True
        self.labelTime.Location = Point(150, 14)
        self.labelTime.Name = "labelTime"
        self.labelTime.Text = "Measurement time [s]:"

        self.textBoxTime.AutoSize = True
        self.textBoxTime.Location = Point(280, 13)
        self.textBoxTime.Name = "textBoxTime"
        self.textBoxTime.Size = Size(35, 20)
        self.textBoxTime.Text = "1"

        self.labelFile.AutoSize = True
        self.labelFile.Location = Point(320, 14)
        self.labelFile.Name = "labelFile"
        self.labelFile.Text = "File:"

        self.textBoxFileName.AutoSize = True
        self.textBoxFileName.Location = Point(350, 13)
        self.textBoxFileName.Name = "textBoxFileName"
        self.textBoxFileName.Size = Size(250, 20)
        self.textBoxFileName.Text = "test"

    def setupCheckButtons(self):
        self.WIN_Monitoring = Button()
        self.WIN_Monitoring.Text = "Start Measurement"
        self.WIN_Monitoring.Location = Point(20, 10)
        self.WIN_Monitoring.Click += self.WIN_monitor
        self.WIN_Monitoring.AutoSize = True

        self.button_Exit = Button()
        self.button_Exit.Text = "Exit"
        self.button_Exit.Location = Point(615, 10)
        self.button_Exit.Click += self.exit
        self.button_Exit.AutoSize = True

        self.Controls.Add(self.WIN_Monitoring)
        self.Controls.Add(self.button_Exit)

    def ShowOxyGraphic(self):
        global T_meas, W_meas
        self.plotmodel1 = OxyPlot.PlotModel()
        self.YAxis1 = OxyPlot.Axes.LinearAxis(Position = OxyPlot.Axes.AxisPosition.Left, Minimum = 0.999*min(W_meas), Maximum = 1.001*max(W_meas))
        self.points_start = []

        for i in range(0,len(T_meas)):
            self.points_start.Add(OxyPlot.DataPoint(T_meas[i],W_meas[i]))

        self.serie_1 = OxyPlot.Series.LineSeries()
        self.serie_1.Color  = OxyPlot.OxyColors.Blue
        self.serie_1.MarkerSize = 2
        self.serie_1.MarkerType = OxyPlot.MarkerType.Circle
        self.serie_1.StrokeThickness = 1
        self.serie_1.ItemsSource = self.points_start
        self.plotmodel1.Axes.Add(self.YAxis1)
        self.plotmodel1.Series.Add(self.serie_1)
        self.plot1.Model = self.plotmodel1

    def WIN_monitor(self, sender, event):
        th = Thread(ParameterizedThreadStart(WIN_measurement))
        th.SetApartmentState(ApartmentState.STA)
        th.Start(self)

    def exit(self, sender, event):
        print("Stop Monitoring script")
        self.Close()


#def launch_form():
form = MonitoringWINMenuForm()
form.StartPosition = FormStartPosition.CenterScreen
form.TopMost = True
form.Show()

#SharpCap.AddCustomButton(" WIN__Monitoring  |", None, "WIN__Monitoring", launch_form)

It calculates the average on the red rectangle area and show it in a graphic.
It saves the time + average in a text file. You can change the measurement duration and the export file name.

I will modify the script, so that it generates an average image of all the image during the measurement time.
One of the problem is ... it is not possible to generate the sum in an image if the image has the same bit depth.
So for a camera bit depth of 8, then it is necessary to have the average on 16 bit. For 16 bit image, then on 32 bit or floating.

The framehandler function is active only during the measurement. I need an image in memory before and after the function framehandler call.
I tried with the following bitmap generation:

Code: Select all

import System.Drawing
from SharpCap.Base import BufferFrame

frameA = Bitmap(1260, 940, Imaging.PixelFormat.Format16bppGrayScale) 
or
frameA = BufferFrame.CreateBitmap(1260, 940, System.Drawing.Imaging.PixelFormat.Format8bppIndexed)

But, the function .Add() does not work with with the bitmap and the buffer together.

Best regards,
Jean-Francois

[admin]

Hi Jean-Francois,

I suspect that using the 'Add' method is more trouble than it is work in this case, particularly because it will only work adding two frames of the same bit depth, so it cannot really help with adding an 8 bit frame to a 16 bit sum, etc.

It's probably easier just to get the data out of the frame into an array - you can use .ToBytes() (8 bit) .ToUshorts() (16 bit) and .ToFloats() (either) which all return a 1d array of the image data (starting first row, left to right, down to last row, finishing at bottom right). RGB pixel data is actually in the order blue, green, red, but that's just Windows for you.

You can then use one of the BufferFrame.CopyFrom...() methods to copy data back into a frame when ready to do that.

cheers,

Robin

[Jean-Francois]

Hello Robin,

OK, I will try as proposed.

Regards,
Jean-Francois

[Jean-Francois]

Hello Robin,

That is really not easy (for me).
I tried with different mathematical libraries ... but an array is a list and used () or {} or [] but not from Array type.
I searched fast everywhere, but I do not find what I wish. Sometimes I found an "add" function, but it is not a number addition, but a list concatenation.

Here my code. It can generate an average image. But the full array iteration in Python is relatively slow.

Code: Select all

# *******************************************************************************************
# SharpCap script "xxxxxxxxx.py"
# 2024/08/18 Jean-Francois
#
# Version: 1.0.0:   First version
#
# The script works only with a SharpCap PRO version.
# *******************************************************************************************

import os
import time
import datetime
import math
import ctypes
import clr
clr.AddReference("Oxyplot")
clr.AddReference("OxyPlot.WindowsForms ")
clr.AddReference("System.Drawing")
clr.AddReference("System.Windows.Forms")

import OxyPlot
import System.Drawing
import System.Windows.Forms

from SharpCap.Base import BufferFrame

from System import Array
from System.Drawing import *
from System.Windows.Forms import *
from System.Threading import Thread, ApartmentState, ParameterizedThreadStart

i = 1
T_meas = []
W_meas = []
t0 = 0
image = 0
frameAverage = Array.CreateInstance(System.Double, 300 * 300)
frameAverage_byte = Array.CreateInstance(System.Byte, 300 * 300)
frameB = 0
pos = 0

# .ToBytes() (8 bit) .ToUshorts() (16 bit) and .ToFloats()
# BufferFrame.CopyFrom...()

#dll=ctypes.CDLL(dllfile)

def framehandler(sender, args):
    global i, T_meas, W_meas, t0, frameAverage, pos, frameB
    if (dumpdata):
        try:
            dt = (datetime.datetime.now() - t0).total_seconds()
            cutout = args.Frame

            Stat = cutout.GetStats()
            T_meas.append(dt)
            W_meas.append(Stat.Item1)
            i = i + 1

            frameB = cutout.ToUshorts()     # 1D array

            for index in range(frameB.Length):
                frameAverage[index] += frameB[index]


#            info           = cutout.Info
#            bufsize        = int(cutout.BufferSize)
#            lease          = cutout.GetBufferLease()
#            frameData_addr = lease.Buffer.ToInt64()
#            frameData_ptr  = ctypes.cast(frameData_addr, ctypes.POINTER(ctypes.c_ubyte))
#            #dll.do_foo(frameData_ptr, info.Width, info.Height, bufsize)
#            lease.Dispose()


        except:
            print("Problem")


# *****************************************************************************
def WIN_measurement(self):
    global dumpdata
    global i, T_meas, W_meas, t0, frameAverage

    self.WIN_Monitoring.Text = "Monitoring running"
    self.WIN_Monitoring.BackColor = Color.Red
    self.WIN_Monitoring.Enabled = False

    SharpCap.Transforms.SelectionRect = Rectangle(300, 300,600, 600)

    i = 1
    T_meas = []
    W_meas = []

    expos_ms = SharpCap.SelectedCamera.Controls.Exposure.ExposureMs
    wait_time = 0.1       # waiting time between 2 refresh of Oxyplot graphic

    print("Waiting Time: %5.3f" % (wait_time))
    print()
    print("WIN Transmission:")
    print("******************")

    dumpdata = True
    SharpCap.SelectedCamera.FrameCaptured += framehandler

    try:
        if (float(self.textBoxTime.Text) > 0.0):
            Max_time = float(self.textBoxTime.Text) - wait_time
            t0 = datetime.datetime.now()
            dt = 0

            while  (dt < Max_time):
                dt = (datetime.datetime.now() - t0).total_seconds()
                time.sleep(wait_time)
                self.ShowOxyGraphic()

            print("Time:", T_meas)
            print("WIN:", W_meas)
            print()
            i = i - 1
            print("Number: ", i)

            mean = sum(W_meas) / len(W_meas)
            var  = sum(pow(x-mean,2) for x in W_meas) / len(W_meas)
            std  = math.sqrt(var)
            print("Standard deviation 1: " + str("%.6f" % round(std,6)))
        else:
            print("Measurement time need to be larger than 0 s")
    except:
        print("Problem 1")

    dumpdata = False
    SharpCap.SelectedCamera.FrameCaptured -= framehandler

    for index in range(frameAverage.Length):
        frameAverage[index] = frameAverage[index] / i

    try:
        for index in range(frameAverage.Length):
            frameAverage_byte[index] = frameAverage[index] / 256
        flag = Bitmap(300,300)
        for i in range(300):
            for j in range(300):
                index = i + (j * 300)
                value = frameAverage_byte[index]
                flag.SetPixel(i, j, System.Drawing.Color.FromArgb(value, value, value))
        view.pictureBox1.Image = flag
    except:
        print()
        print("picture problem")

    print()
    print("Show Average")

    print()
    print("Measurement done")

    self.WIN_Monitoring.Text = "Start Measurement"
    self.WIN_Monitoring.BackColor = Color.Gainsboro
    self.WIN_Monitoring.Enabled = True

    file_path  = os.path.dirname(__file__)
    file_name = file_path + "\\" + self.textBoxFileName.Text + ".txt"
    file = open(file_name, "w")
    for index in range(len(T_meas)):
        file.write(str("%.4f" % T_meas[index]) + " " + str("%.4f" % W_meas[index]) + "\n")
    file.close()

    return()

# *****************************************************************************

class MonitoringWINMenuForm(Form):
    def __init__(self):
        self.SuspendLayout()
        self.InitializeComponent()
        self.setupCheckButtons()
        self.AutoScaleMode = System.Windows.Forms.AutoScaleMode.Dpi
        self.AutoScaleDimensions = SizeF(96, 96)
        self.ResumeLayout()

    def InitializeComponent(self):
        self.Text = "WIN Transmission"
        self.ClientSize = System.Drawing.Size(700, 400)
        self.TopMost = True

        self.labelTime = Label()
        self.textBoxTime = TextBox()
        self.labelFile = Label()
        self.textBoxFileName = TextBox()

        self.plot1 = OxyPlot.WindowsForms.PlotView()
        self.plot1.Location = System.Drawing.Point(10, 35)
        self.plot1.Size = System.Drawing.Size(680, 350)
        self.plot1.AutoSize = True

        self.Controls.Add(self.labelTime)
        self.Controls.Add(self.textBoxTime)
        self.Controls.Add(self.labelFile)
        self.Controls.Add(self.textBoxFileName)
        self.Controls.Add(self.plot1)

        self.labelTime.AutoSize = True
        self.labelTime.Location = Point(150, 14)
        self.labelTime.Name = "labelTime"
        self.labelTime.Text = "Measurement time [s]:"

        self.textBoxTime.AutoSize = True
        self.textBoxTime.Location = Point(280, 13)
        self.textBoxTime.Name = "textBoxTime"
        self.textBoxTime.Size = Size(35, 20)
        self.textBoxTime.Text = "1"

        self.labelFile.AutoSize = True
        self.labelFile.Location = Point(320, 14)
        self.labelFile.Name = "labelFile"
        self.labelFile.Text = "File:"

        self.textBoxFileName.AutoSize = True
        self.textBoxFileName.Location = Point(350, 13)
        self.textBoxFileName.Name = "textBoxFileName"
        self.textBoxFileName.Size = Size(250, 20)
        self.textBoxFileName.Text = "test"

    def setupCheckButtons(self):
        self.WIN_Monitoring = Button()
        self.WIN_Monitoring.Text = "Start Measurement"
        self.WIN_Monitoring.Location = Point(20, 10)
        self.WIN_Monitoring.Click += self.WIN_monitor
        self.WIN_Monitoring.AutoSize = True

        self.button_Exit = Button()
        self.button_Exit.Text = "Exit"
        self.button_Exit.Location = Point(615, 10)
        self.button_Exit.Click += self.exit
        self.button_Exit.AutoSize = True

        self.Controls.Add(self.WIN_Monitoring)
        self.Controls.Add(self.button_Exit)

    def ShowOxyGraphic(self):
        global T_meas, W_meas
        self.plotmodel1 = OxyPlot.PlotModel()
        self.YAxis1 = OxyPlot.Axes.LinearAxis(Position = OxyPlot.Axes.AxisPosition.Left, Minimum = 0.999*min(W_meas), Maximum = 1.001*max(W_meas))
        self.points_start = []

        for i in range(0,len(T_meas)):
            self.points_start.Add(OxyPlot.DataPoint(T_meas[i],W_meas[i]))

        self.serie_1 = OxyPlot.Series.LineSeries()
        self.serie_1.Color  = OxyPlot.OxyColors.Blue
        self.serie_1.MarkerSize = 2
        self.serie_1.MarkerType = OxyPlot.MarkerType.Circle
        self.serie_1.StrokeThickness = 1
        self.serie_1.ItemsSource = self.points_start
        self.plotmodel1.Axes.Add(self.YAxis1)
        self.plotmodel1.Series.Add(self.serie_1)
        self.plot1.Model = self.plotmodel1

    def WIN_monitor(self, sender, event):
        th = Thread(ParameterizedThreadStart(WIN_measurement))
        th.SetApartmentState(ApartmentState.STA)
        th.Start(self)

    def exit(self, sender, event):
        print("Stop Monitoring script")
        self.Close()

class MainForm(Form):
    def __init__(self):
        self.InitializeComponent()
        self.pen = System.Drawing.Pen(System.Drawing.Color.Black);

    def InitializeComponent(self):
        self.pictureBox1 = System.Windows.Forms.PictureBox()
        self.SuspendLayout()

        self.pictureBox1.Location = System.Drawing.Point(3, 3)
        self.pictureBox1.Name = "pictureBox1"
        self.pictureBox1.Size = System.Drawing.Size(300, 300)
        self.pictureBox1.TabIndex = 0
        self.pictureBox1.TabStop = False

        self.ClientSize = System.Drawing.Size(306, 306)
        self.Controls.Add(self.pictureBox1)
        self.Name = "MainForm"
        self.Text = "Average"
        self.ResumeLayout()


form = MonitoringWINMenuForm()
form.StartPosition = FormStartPosition.CenterScreen
form.TopMost = True
form.Show()

view = MainForm()
view.StartPosition = FormStartPosition.CenterScreen
view.TopMost = True
view.Show()

If somebody want to test:
- select the Test Camera 2 (Hight Speed)
- ROI = 300 x 300
- 16 bit mono
- 1 or 2 ms exposure with gain = 4000
- add noise on the image ~ 50

The code does not change the size or accept other image parameters.
One error can produce a crash of SharpCap.

Here an example of the result:

cutout_average.jpg (196.9 KiB) Viewed 3151 times

On my computer, with the settings like above, SharpCap generates ~333 image/s.
If I disable the image average, then my script can calculate ~ 325 image/s.
If I activate the image average, the script calculates only ~ 41 image/s.

I really need to improve the calculation time of the image average.
I found somewhere in the forum under "script" topics ...

Code: Select all

dll=ctypes.CDLL(dllfile)

info           = cutout.Info
bufsize        = int(cutout.BufferSize)
lease          = cutout.GetBufferLease()
frameData_addr = lease.Buffer.ToInt64()
frameData_ptr  = ctypes.cast(frameData_addr, ctypes.POINTER(ctypes.c_ubyte))
#dll.do_foo(frameData_ptr, info.Width, info.Height, bufsize)
lease.Dispose()

It will be painful to code a dll in C/C++ for doing the calculation.
What is missing (I search, but I do not find a solution) is how to find the address of the Array().


Regards,
Jean-Francois

[admin]

Hi Jean-Francois,

ah, OK. I suspect the performance issue is the use of python (interpreted language) for the loop to add the values.

Do you have a c# library of your own that you include in your add-in? You could define an 'AddArray' function in that to do the addition and reference it from the python. In C#, something like

Code: Select all

    public static class Utils
    {
        public static void AddArray(int[] left, int[] right)
        {
            if (left.Length != right.Length)
                throw new ArgumentException("Array lengths differ");

            for (int i=0; i < left.Length; i++)
            {
                left[i] += right[i];
            }

        }

    }

The quick way to test whether that is actually the performance pain point is to comment out the addition in the script, so that it doesn't bother with the loop and see if the frame rate rises enough to be useful.

cheers,

Robin

[Jean-Francois]

Hello Robin,

Yes, the interpreted Python language is slower as a compiled program (if doing the same thing).

I have no C# library. That is what I will try to code.
The difficulties will be with the pointers. First to find and transfer the pointer from Python script to the dll,
and then to interpret these pointers correctly in the C# code.

I saw somewhere in the internet that some variable are in Python not fix.
The address of "a" in "a = 5" is not the same after doing "a = 6". That could be a problem.

The other problem is to be very quick, the code should be done without additional conversion.

I tested my actual script ...

On my computer, with the settings like above, SharpCap generates ~333 image/s.
If I disable the image average, then my script can calculate ~ 325 image/s.
If I activate the image average, the script calculates only ~ 41 image/s.

I will redo the test after a working dll-library.

Regards,
Jean-Francois