Automatic adjustment of the LED (QHY-174-GPS)

procyon12
Posts: 253
Joined: Tue Jan 14, 2020 11:32 am

Re: Automatic adjustment of the LED (QHY-174-GPS)

#11

Post by procyon12 »

Hello Jean-Francois,

That's all very amazing !
I've tested under W7-64 with the latest SC 3.2 and 3.3 (both 32 bit) - both scripts work very well.

Example script 2 with usual times for occultation work (a CSV output might be good ?).

Code: Select all

Time_List = [5,10,20,30,40,50,60,70,80,90,100,150,200,250,300,350,400,450,500,600,800,1000,1300,2000,3000] # [ms]

Camera Settings:
========================================================
Colour Space :  MONO16
Binning      :  1x1
USB Traffic  :  3

Calculation time:  12.56  minutes

 [ms]  Cal_Start    Cal_End       Exp.   LED-time    Delta   SC_Start     SC_End 
================================================================================
    5       9661     384895       5.00      5.003   -0.003       9661     381827
   10       9661     759601      10.00      9.999    0.001       9661     756827
   20       9661    1510036      20.00     20.005   -0.005       9661    1506827
   30       9666    2259451      30.00     29.997    0.003       9661    2256827
   40       9716    3009886      40.00     40.002   -0.002       9661    3006827
   50       9656    3759298      50.00     49.995    0.005       9661    3756827
   60       9661    4509736      60.00     60.001   -0.001       9661    4506827
   70       9661    5260168      70.00     70.007   -0.007       9661    5256827
   80       9606    6009584      80.00     80.000    0.000       9661    6006827
   90       9606    6760018      90.00     90.005   -0.005       9661    6756827
  100       9666    7509434     100.00     99.997    0.003       9661    7506827
  150       9661   11259566     150.00    149.999    0.001       9661   11256827
  200       9661   15009697     200.00    200.000    0.000       9661   15006827
  250       9606   18759830     250.00    250.003   -0.003       9661   18756827
  300       9606   22509966     300.00    300.005   -0.005       9661   22506827
  350       9661   26260098     350.00    350.006   -0.006       9661   26256827
  400       9606   30009210     400.00    399.995    0.005       9661   30006827
  450       9661   33759342     450.00    449.996    0.004       9661   33756827
  500       9661   37509478     500.00    499.998    0.002       9661   37506827
  600       9661   45009741     600.00    600.001   -0.001       9661   45006827
  800       9606   60009254     800.00    799.995    0.005       9661   60006827
 1000       9716   75009786    1000.00   1000.001   -0.001       9661   75006827
 1300       9606   97509561    1300.00   1299.999    0.001       9661   97506827
 2000       9666  150009381    2000.00   1999.996    0.004       9661  150006827
 3000       9716  225010001    3000.00   3000.004   -0.004       9661  225006827
Cheers

Christian
Jean-Francois
Posts: 361
Joined: Sun Oct 13, 2019 10:52 am
Location: Germany

Re: Automatic adjustment of the LED (QHY-174-GPS)

#12

Post by Jean-Francois »

Hello Christian,

Sorry for giving a bad news ... your values are not correct.
It is good that you test with the 16 bit image ... that shows the problems .

For long integration time, the Start Position is all the time the same ... but you have 9661, 9716 or 9606 ... that is not correct.
When I set with my camera the same parameters (MONO16, 1x1 and USB = 3) then SharpCap sets the Start Pos automatically to 9661.
That is what you have too, but it is not correct.
The Start Pos has to be for 20 ms exposure time 14299. Start the (single) script several time ... you will have different results.
Sometimes the correct value is around 14299, sometimes something completely wrong, or the script take image only from the "dark" time and the result will be the same as the starting value from SharpCap (mean here 9661).

So Robin, can you have a look on the calculation of the Start Pos for MONO16 ? (for exposure above 13 ms)
Note ... the MONO8 start values are all OK.


Today my script starts the search of the End Pos from the SC value.
I already think to change this, so that the End Pos is calculated from the Start Pos (as Robin writes in the message from 05 Feb 2020, 19:48, page 4 of the topic "QHY 174 GPS Calibration LED issue // USB Traffic weirdness") ... End Pos = Start Pos + 75000 * Exposure (ms) - 1903 - 277 * USB Traffic

Here a quick check of the equation ... End Pos = 14299 + 75000*20 -1903 - 277*3 = 1511565
Compare to the result on my camera (and my script) : 1510034 (diff = 1531)

=> the script is not stable for the MONO16 case

I try to improve the correct search (I add a starting value with a "*10 max digit" ... also for 9661, the script add a position by 19661 so that a "light" image is taken in any case). That is not necessary for the MONO8 images.

For example for the MONO8 End-Pos:
The deviation between the SC start values and the result with my camera/script is between -150 and 300.
Result_MONO8_End-Pos_10-100ms.png
Result_MONO8_End-Pos_10-100ms.png (64.11 KiB) Viewed 2226 times

For the MONO16 Start-Pos:
The deviation between the SC start values and the result with my camera/script is between -200 and 600.
Result_MONO16_Start-Pos_1-10ms.png
Result_MONO16_Start-Pos_1-10ms.png (19.95 KiB) Viewed 2226 times

The problem is for MONO16 with exposure time above 13 ms.

I think to write two additional script ... one for performing the calibration of the LED at different settings (8/16, USB Traffic and exposure time) and saving this in a separate file. The second script will open this file and perform an interpolation of the Start/End Position for the 8/16, USB Traffic and exposure time, and then to start the LED calibration.


Regards,
Jean-Francois
Jean-Francois
Posts: 361
Joined: Sun Oct 13, 2019 10:52 am
Location: Germany

Re: Automatic adjustment of the LED (QHY-174-GPS)

#13

Post by Jean-Francois »

Hello,

In the past week, I do a lot of modification in my scripts and some tests.

Here the last version with the following changes:
- new calculation of the Start/End values with a least square fit algorithm (after linearisation of the Fermi/exponential function)
- add the export of the results in a text file (the file location can be changed ... code line 351/352)

I improve the search in the 16 bit mode, but it is sometimes difficult to find the correct values (because the Start/End value from SharpCap are sometimes far away from the correct values).

Code: Select all

import time
import math
import System

from System.IO import Directory

directory = Directory.GetCurrentDirectory()

# (Begin) - Parameter to be changed by the user
search_loop = 20						# Number of fine search
power_threshold = 4						# (10**power_threshold) digit
slope = 1.2								# value between 1.0 and 1.3

Time_List = [5] # [ms]

#Time_List = [1.0,1.05,1.1,1.15,1.2,1.25,1.3,1.35,1.4,1.45,1.5,1.55,1.6,1.65,1.7,1.75,1.8,1.85,1.9,1.95,2.0] # [ms]
#Time_List = [1.0,1.2,1.4,1.6,1.8,2.0,2.2,2.4,2.6,2.8,3.0,3.2,3.4,3.6,3.8,4.0,4.2,4.4,4.6,4.8,5.0,5.2,5.4,5.6,5.8,6.0,6.2,6.4,6.6,6.8,7.0,7.2,7.4,7.6,7.8,8.0,8.2,8.4,8.6,8.8,9.0,9.2,9.4,9.6,9.8,10.0] # [ms]
#Time_List = [7.05,7.1,7.15,7.2,7.25,7.3,7.35,7.4,7.45,7.5,7.55,7.6,7.65,7.7,7.75,7.8,7.85,7.9,7.95] # [ms]
#Time_List = [1,1.5,2,2.5,3,3.5,4,5,6,7,7.2,7.4,7.6,7.8,8,9,10,11,12,13,14,15,20,40,60,80,100,150,200,250,500,750,1000] # [ms]
#Time_List = [10,20,30,40,50,60,70,80,90,100]
#Time_List = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,32,34,36,38,40,42,44,46,48,50] # [ms]
#Time_List = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50] # [ms]
#Time_List = [51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100] # [ms]
#Time_List = [12.1,12.2,12.3,12.4,12.5,12.6,12.7,12.8,12.9] # [ms]
#Time_List = [5,10,20,30,40,50,60,70,80,90,100,150,200,250,300,350,400,450,500,600,800,1000,1300,2000,3000] # [ms]

# (End)   - Parameter to be changed by the user

SharpCap.SelectedCamera.Controls.Exposure.ExposureMs = Time_List[0]
Cal_Start_Pos = SharpCap.SelectedCamera.Controls[20].Value
Cal_End_Pos = SharpCap.SelectedCamera.Controls[19].Value
Colour_Space = SharpCap.SelectedCamera.Controls.ColourSpace.Value

LED_status = SharpCap.SelectedCamera.Controls[21].Value
if (LED_status == "Off"):
    SharpCap.SelectedCamera.Controls[21].Value = "On"

print "Date:",  time.ctime()
print SharpCap.SelectedCamera.Controls[18]
print("GPS Calibration LED on/off  : %s") % (SharpCap.SelectedCamera.Controls[21].Value)
print("GPS Freq Stabilization      : %s") % (SharpCap.SelectedCamera.Controls[22].Value)
print("GPS                         : %s") % (SharpCap.SelectedCamera.Controls[23].Value)

cloneRect = SharpCap.Transforms.SelectionRect
print("Rectangle Selection         : "), cloneRect
print

def framehandler(sender, args):
    if (dumpdata):
        global Mean
        cutout = args.Frame.CutROI(SharpCap.Transforms.SelectionRect)
        Stat = cutout.GetStats()
        Mean = Stat.Item1
        cutout.Release()

def evthandler(sender, args):
    if (SharpCap.SelectedCamera != None):
        SharpCap.SelectedCamera.FrameCaptured -= framehandler

def monitorFrames():
    SharpCap.SelectedCamera.FrameCaptured += framehandler

def Search_next_L(M, a):
    k = 0
    while (M[k][1] < a):
        k = k + 1
    return(math.floor((M[k][0] + M[k-1][0]) / 2.0))

def Search_next_R(M, a):
    k = 0
    while (M[k][1] > a):
        k = k + 1
    return(math.floor((M[k][0] + M[k-1][0]) / 2.0))

def LED_Start_search(Pos, LED, n):
    P = Pos
    L = LED
    for i in range(0,n):
        average = (min(L) + max(L)) / 2.0
        M = zip(P,L)
        M.sort()
        Pnew = Search_next_L(M,average)
        SharpCap.SelectedCamera.Controls[20].Value = int(Pnew)
        time.sleep(wait_time)
        P.append(Pnew)
        L.append(Mean)
#        print("%i   %7.3f") % (P[-1], L[-1])
        if (abs((L[-1] - average)/average) < 0.05):
            break
    M = zip(P,L)
    M.sort()
    return(M)

def LED_End_search(Pos, LED, n):
    P = Pos
    L = LED
    for i in range(0,n):
        average = (min(L) + max(L)) / 2.0
        M = zip(P,L)
        M.sort()
        Pnew = Search_next_R(M,average)
        SharpCap.SelectedCamera.Controls[19].Value = int(Pnew)
        time.sleep(wait_time)
        P.append(Pnew)
        L.append(Mean)
#        print("%i   %7.3f") % (P[-1], L[-1])
        if (abs((L[-1] - average)/average) < 0.05):
            break
    M = zip(P,L)
    M.sort()
    return(M)


# LED Calibration Start Position Adjustment
def LED_Start(expos_ms):
    Cal_Start_Pos = SharpCap.SelectedCamera.Controls[20].Value
    if (Colour_Space == "MONO16"):
        cal_step = math.floor(math.log10(Cal_Start_Pos)) - 0
    if (Colour_Space == "MONO8"):
        cal_step = math.floor(math.log10(Cal_Start_Pos)) - 0

    Pos = [Cal_Start_Pos]
    if (cal_step > power_threshold):
        cal_step = power_threshold
    print
    print("Digit position selection:  "),
    for p in range(int(cal_step), 0, -1):
        print 10**p,
        Pos.append(Cal_Start_Pos + 10**p)
        Pos.append(Cal_Start_Pos - 10**p)

    if (Colour_Space == "MONO16"):
        Pos.append(Cal_Start_Pos + 10**(cal_step+1))

    Pos.sort()
    print
    print("Cal_Start_Pos list: "), Pos

    LED = []
    loop = 1000
    for L in range(0,len(Pos)):
        SharpCap.SelectedCamera.Controls[20].Value = int(Pos[L])
        
        time.sleep(wait_time)
        LED.append(Mean)
        
        if (L==0):
            print("%i   %8.3f") % (int(Pos[L]), LED[L])
        else:
            print("%i   %8.3f") % (int(Pos[L]), LED[L]),
            average = (LED[L]+LED[L-1])/2.0
            diff = LED[L]-LED[L-1]
            print
            if (abs(diff / average) > slope):
                loop = L
        if (loop == (L - 1)):
            print "Break"
            break

    print("Min / Max  : %6.3f   %6.3f") % (min(LED), max(LED))
    print("Half height: %6.3f") % ((min(LED) + max(LED)) / 2.0)
    Pos = Pos[0 : len(LED)]
    M = zip(Pos, LED)
    M.sort()
    time.sleep(wait_time)
    Result = LED_Start_search(Pos, LED, search_loop)
    Pos, LED = zip(*Result)
    print
    l_min = min(LED) * 1.1
    l_max = max(LED) * 0.95
    Pos_m = []
    LED_m = []
    Y_m = []
    for i in range(0,len(Pos)):
        if (LED[i]>l_min) & (LED[i]<l_max):
            Pos_m.append(Pos[i])
            LED_m.append(LED[i])
            Y_m.append(-(math.log((max(LED) - min(LED)) / (LED[i] - min(LED)) - 1.0)))
            print("%i %10.4f") % (Pos_m[-1], LED_m[-1])
    if (len(Pos_m) < 2):
#        SharpCap.SelectedCamera.Controls[20].Value = int(Pos[-1])
        print("Direct value:")
        print("%i %10.4f") % (Pos[-1], LED[-1])
        return(Pos[-1])
    print
    if (len(Pos_m) > 1.5):
        Sx2 = 0
        Sx = 0
        Vx2 = []
        Vx = []
        n = len(Pos_m)

        for i in range(0,len(Pos_m)):
            xi = Pos_m[i] - Pos_m[0]
            Vx.append(xi)
            Vx2.append(xi**2)
            Sx = Sx + xi
            Sx2 = Sx2 + xi**2

        Det = float(Sx2 * n - Sx * Sx)
        Inv = [[n / Det,-Sx / Det],[-Sx / Det,Sx2 / Det]]

        a = 0
        b = 0
        for i in range(0,len(Pos_m)):
            a = a + (Inv[0][0]*Vx2[i] + Inv[0][1]) * LED_m[i]
            b = b + (Inv[1][0]*Vx2[i] + Inv[1][1]) * LED_m[i]
        k = a
        m = -b / a
        Cal_Start_Pos = round(Pos_m[0] + m)
        print("Optimisation: n = %i, k = %10.6f, m = %10.6f, Cal Start Pos = %i") % (n,k,m,Cal_Start_Pos)
        SharpCap.SelectedCamera.Controls[20].Value = int(Cal_Start_Pos)
        return(Cal_Start_Pos)


# LED Calibration End Position Adjustment
def LED_End(expos_ms):
    Cal_End_Pos = SharpCap.SelectedCamera.Controls[19].Value
    SharpCap.SelectedCamera.Controls[19].Value = Cal_End_Pos + 1
    SharpCap.SelectedCamera.Controls[19].Value = Cal_End_Pos - 1

    cal_step = math.floor(math.log10(Cal_End_Pos)) - 1
    Pos = [Cal_End_Pos]
    if (cal_step > power_threshold):
        cal_step = power_threshold
    print
    print("Digit position selection:  "),
    for p in range(int(cal_step), 0, -1):
        print 10**p,
        Pos.append(Cal_End_Pos + 10**p)
        Pos.append(Cal_End_Pos - 10**p)

    Pos.sort()
    print
    print("Cal_End_Pos list: "), Pos

    LED = []
    loop = 1000
    for L in range(0,len(Pos)):
        SharpCap.SelectedCamera.Controls[19].Value = int(Pos[L])

        time.sleep(wait_time)
        LED.append(Mean)

        if (L==0):
            print("%i   %8.3f") % (Pos[L], LED[L])
        else:
            print("%i   %8.3f") % (Pos[L], LED[L]),
            average = (LED[L]+LED[L-1])/2.0
            diff = LED[L]-LED[L-1]
            print
            if (abs(diff / average) > slope):
                loop = L
        if (loop == (L - 1)):
            print "Break"
            break

    print("Min / Max  : %6.3f   %6.3f") % (min(LED), max(LED))
    print("Half height: %6.3f") % ((min(LED) + max(LED)) / 2.0)
    Pos = Pos[0 : len(LED)]
    M = zip(Pos, LED)
    M.sort()

    time.sleep(wait_time)
    Result = LED_End_search(Pos, LED, search_loop)
    Pos, LED = zip(*Result)
    print
    l_min = min(LED) * 1.1
    l_max = max(LED) * 0.95
    Pos_m = []
    LED_m = []
    Y_m = []
    for i in range(0,len(Pos)):
        if (LED[i]>l_min) & (LED[i]<l_max):
            Pos_m.append(Pos[i])
            LED_m.append(LED[i])
            Y_m.append((math.log((max(LED) - min(LED)) / (LED[i] - min(LED)) - 1.0)))
            print("%i %10.6f") % (Pos_m[-1], LED_m[-1])
    print
    if (len(Pos_m) < 2):
#        SharpCap.SelectedCamera.Controls[19].Value = int(Pos[-1])
        print("Direct value:")
        print("%i %10.4f") % (Pos[-1], LED[-1])
        return(Pos[-1])

    if (len(Pos_m) > 1.5):
        Sx2 = 0
        Sx = 0
        Vx2 = []
        Vx = []
        n = len(Pos_m)

        for i in range(0,len(Pos_m)):
            xi = Pos_m[i] - Pos_m[0]
            Vx.append(xi)
            Vx2.append(xi**2)
            Sx = Sx + xi
            Sx2 = Sx2 + xi**2

        Det = float(Sx2 * n - Sx * Sx)
        Inv = [[n / Det,-Sx / Det],[-Sx / Det,Sx2 / Det]]

        a = 0
        b = 0
        for i in range(0,len(Pos_m)):
            a = a + (Inv[0][0]*Vx2[i] + Inv[0][1]) * LED_m[i]
            b = b + (Inv[1][0]*Vx2[i] + Inv[1][1]) * LED_m[i]
        k = a
        m = -b / a
        Cal_End_Pos = round(Pos_m[0] + m)
        print("Optimisation: n = %i, k = %10.6f, m = %10.6f, Cal End Pos = %i") % (n,k,m,Cal_End_Pos)
        SharpCap.SelectedCamera.Controls[19].Value = int(Cal_End_Pos)
        return(Cal_End_Pos)


# *****************************************************************************
print
print "Time sequence  [ms] :", Time_List
Cal_Start = []
Cal_End = []
SC_Start = []
SC_End = []
time_start = time.clock()

dumpdata = True
SharpCap.CaptureEvent += evthandler
monitorFrames()

for T in range(0,len(Time_List)):
    SharpCap.SelectedCamera.Controls.Exposure.ExposureMs = Time_List[T]
    expos_ms = SharpCap.SelectedCamera.Controls.Exposure.ExposureMs
    wait_time = 2.1 *(expos_ms / 1000.0) + 0.2
    Cal_End_Pos = SharpCap.SelectedCamera.Controls[19].Value
    Cal_Start_Pos = SharpCap.SelectedCamera.Controls[20].Value
    SC_End.append(int(Cal_End_Pos))
    SC_Start.append(int(Cal_Start_Pos))
    print
    print
    print "Exposure Time: ", Time_List[T], " ms"
    print("=================================")
    Cal_Start.append(int(LED_Start(Time_List[T])))
    Cal_End.append(int(LED_End(Time_List[T])))
    SharpCap.SelectedCamera.Controls[19].Value = Cal_End[-1]
    SharpCap.SelectedCamera.Controls[20].Value = Cal_Start[-1]

time_end = time.clock()
date = time.localtime()
file_date = time.strftime("_%Y%m%d_%H%M%S",date)

file_name = directory + '\\LED_'
#file_name = 'D:\\Scripts\\GPS_control\\LED_'

file_name = file_name + "_" + str(SharpCap.SelectedCamera.Controls.ColourSpace.Value)
file_name = file_name + "_" + str(SharpCap.SelectedCamera.Controls.Resolution.Value)
file_name = file_name + "_" + str(SharpCap.SelectedCamera.Controls.Binning.Value)
file_name = file_name + "_USB" + str(SharpCap.SelectedCamera.Controls.Usb.Value)
file_name = file_name + file_date + '_.txt'

print
print
with open(file_name, "w") as myfile:
    print("Camera Settings:")
    myfile.writelines("Camera Settings:" + "\n")
    print("========================================================")
    print("Colour Space : "), SharpCap.SelectedCamera.Controls.ColourSpace.Value
    myfile.writelines(("Colour Space :, %s \n") % (str(SharpCap.SelectedCamera.Controls.ColourSpace.Value)))
    print("Binning      : "), SharpCap.SelectedCamera.Controls.Binning.Value
    myfile.writelines(("Binning      :, %s \n") % (str(SharpCap.SelectedCamera.Controls.Binning.Value)))
    print("USB Traffic  : "), SharpCap.SelectedCamera.Controls.Usb.Value
    myfile.writelines(("USB Traffic  :, %s \n") % (str(SharpCap.SelectedCamera.Controls.Usb.Value)))
    print
    print "Calculation time: ", round((time_end - time_start)/60, 2), " minutes"
    myfile.writelines(("Calculation time:, %s \n") % (str(round((time_end - time_start)/60, 2))))
    print
    print(" [ms]  Cal_Start    Cal_End       Exp.   LED-time    Delta   SC_Start     SC_End ")
    myfile.writelines("[ms],Cal_Start,Cal_End,Exp.,LED-time,Delta,SC_Start,SC_End" + "\n")
    print("================================================================================")
    for T in range(0,len(Time_List)):
        LED_time = (Cal_End[T] - Cal_Start[T]) / 75000.0
        Diff_time = Time_List[T] - LED_time
        print("%5s %10s %10s %10.2f %10.3f %8.3f") % (Time_List[T],Cal_Start[T],Cal_End[T],Time_List[T],LED_time,Diff_time),
        print("%10s %10s") % (SC_Start[T],SC_End[T])
        myfile.writelines(("%5s,%10s,%10s,%10.2f,%10.3f,%8.3f,%10s,%10s") % (Time_List[T],Cal_Start[T],Cal_End[T],Time_List[T],LED_time,Diff_time,SC_Start[T],SC_End[T]) + "\n")

dumpdata = False

myfile.close()

if (LED_status == "Off"):
    SharpCap.SelectedCamera.Controls[21].Value = "Off"

Here some results summarized in an Excel file:
LED_calibration_with_script.xlsx
(460.42 KiB) Downloaded 115 times

Regards,
Jean-Francois
procyon12
Posts: 253
Joined: Tue Jan 14, 2020 11:32 am

Re: Automatic adjustment of the LED (QHY-174-GPS)

#14

Post by procyon12 »

Hello Jean-Francois,

Due to limited time I was only able to do a few spot checks: It works in principle.
because the Start/End value from SharpCap are sometimes far away from the correct values
Are you sure here?

With the "autocalibration" I fulfill normally the criteria I wrote somewhere (Locked state + Right exp. time + GPS time near system time ...).

Also if I get a locked state after calibration with your script, and esp. startPos. differ like following

Code: Select all

 [ms]  Cal_Start    Cal_End       Exp.   LED-time    Delta   SC_Start     SC_End 
================================================================================
  250      14292   18759834     250.00    249.941    0.059       9661   18756827
when I switch on the LED after the script calibration, the LED should lit - but I get then very often "BadCalibrationData"?

In general, I think the script must be absolutely stable - and then integrated into a SC-testversion, ideally with both calibration options.


Cheers, Christian
Jean-Francois
Posts: 361
Joined: Sun Oct 13, 2019 10:52 am
Location: Germany

Re: Automatic adjustment of the LED (QHY-174-GPS)

#15

Post by Jean-Francois »

Hello Christian,

Concerning the Start/End values from SharpCap ... look in the Excel file ...
In the cells region (V58;AE97), you can see the difference between the script results and the SharpCap start values.
In the sheets "MONO8_xxx" : the differences are ~ +/- 2000 max.
In the sheet "MONO16_xxx" : the differences are larger ... look at the Start value above with the "horizontal" value, the difference is 4638.

For the calculation of the End value from the the Start value ... yes it works fine (at least for 8 bit, I do not check for 16 bit).
I calculate the difference in the sheet "MONO8_1x1_1920x1200" (after the row 100).

At the beginning, my script was taking the SC value and modify the value "up" and "down". For a value of 6000, then the script was adding +100 or -100, then +10 and -10. It was working good for all the first test with the 8 bit images (with full resolution). Later when I test the 16 bit or I change the USB traffic or the image height, then it was no more working with so small value. Now for 16 bit, the script add 10000 for a starting value of 6301.

Need to take care not to find the "third transition", in my case (with 16 bit, 1920x1200, USB=0, 250 ms) at 4200.

I think that the stability of the script can be improved for the 16 bit images when the starting values are improved.
For an integrated version in SC ... why not, but only Robin can implement it in SC.

Regards,
Jean-Francois
Jean-Francois
Posts: 361
Joined: Sun Oct 13, 2019 10:52 am
Location: Germany

Re: Automatic adjustment of the LED (QHY-174-GPS)

#16

Post by Jean-Francois »

Hello Christian,

I test you last (not working) example ... the script find a wrong value (for 16 bit, 1920x1200, USB = 3 and 250 ms).
I habe the same behavior as you show ... the problem is that the script found a good value at the first try, but it gives at the end the last value after sorting all the tries.

Here the last part of the output ...

Code: Select all

18756927   9388.783
18757827   9376.538
18766827    471.442
Min / Max  : 471.442   9403.278
Half height: 4937.360

18759830 5034.550424

Direct value:
18766827   471.4425


Camera Settings:
========================================================
Colour Space :  MONO16
Binning      :  1x1
USB Traffic  :  3

Calculation time:  0.45  minutes

 [ms]  Cal_Start    Cal_End       Exp.   LED-time    Delta   SC_Start     SC_End 
================================================================================
  250      14290   18766827     250.00    250.034   -0.034       9661   18756827

The first sequence is a fix value with -10000, -1000, -100, -10, 0, +10, +100, +1000 and +10000 from the start value.
Then if a transition (dark to light for Start Pos or light to dark for End Pos), the script search with several iteration the value near the average.
In my case here, the first try gives a good result: 5034.55 (for 18759830) is near the average value of 4937.36.
But in place to take this value, the script gives the last value from the preceding search (for the +10000 case).
That is not correct.
Note, if the iterative search has 2 or more values, then the result is correct.
Also Christian ... you find a case with the first value near the average ... so the problem here.

It explains the problem with the "BadCalibrationData" ... that is clear, while the correct value is 18759830.

But ... when I switch-off and then switch-on the LED ... I have "BadCalibrationReduceEndPos" with the values 14290/18759830.
With the LED on ... I click the Start Pos +10 and -10, then the End Pos +10 and -10 ... and the data is locked.
I remark the following :
- when the last click is on the End Pos (for example +10 and -10), LED off and on again, then "BadCalibrationReduceEndPos"
- when the last click is on the Start Pos (for example +10 and -10), LED off and on again, then "Locked"

(... I like this camera ... a lot of interesting things happens)


Regards,
Jean-Francois
andyho
Posts: 16
Joined: Wed Apr 07, 2021 7:12 pm

Re: Automatic adjustment of the LED (QHY-174-GPS)

#17

Post by andyho »

SITUATION

The auto-calibration script "calib_qhy.py" does not produce calibration settings that work with my QHY174M-GPS camera. This script was kindly provided to me by C. Weber on 28 May 2021.

A statistical experiment was conducted. For a SharpCap exposure of 1000ms, the auto-calibration script gives results that do not agree with settings found by me using manual calibration.

MANUAL CALIBRATION

My QHY174M-GPS camera (SN 110007) was calibrated manually by me on 8 April 2021, for exposures ranging from 10ms to 4000ms. I used the manual calibration method described in detail by K. Getrost (https://www.youtube.com/watch?v=1myioXAEBtQ) and M. Buie (https://www.youtube.com/watch?v=tNaToOa-Kh0&t=3s). When SharpCap's exposure was set to 1000ms, manual calibration yielded the following calibration settings:

Calibration Start Pos Adjust = 158,120
Calibration End Pos Adjust = 75,107,060

The camera's other settings are as follows: Mono16, 1920x1200, 1x1 binning, maximum frame rate, amp noise reduction ON, gain 300, offset 100, USB traffic = 0, GPS ON, GPS frequency stabilization OFF, Cooler Temp = 0.0 C, histogram stretch ON, banding suppression = 0.

My camera has the so-called "3rd transition," a phenomenon first described by K. Getrost ("ibid"). While calibrating the camera, I discovered that the camera's electronic shutter is open from zero (0) to 46,290 when SharpCap's exposure >=250ms. The 3rd transition at position 46,290 is where the camera's electronic shutter changes from open to closed. It must not be confused with the camera's start of exposure nor with the camera's end of exposure that occurs later.

AUTO-CALIBRATION EXPERIMENT

On the afternoon of 29 May 2021, my QHY174M-GPS camera (SN = 110007) was connected to a USB3.0 port on the docking station of a Dell E6420 laptop computer running SharpCap 4.0.7806 (beta). After opening SharpCap, the previously saved 1000ms calibration profile (from the manual calibration of 8 April 2021) was loaded.

Lock was immediate. (GPS Status window of the attached screenshot)

To prepare for the auto-calibration experiment, the camera's ROI was then reduced from 1920x1200 to 600x1200, while setting Pan = 1136 and Tilt = 0. This restricted the view to the right-most part of the sensor where the LED's illumination is brightest. No other settings were changed.

Next, the "calib_qhy.py" script was invoked using the Run Script command from SharpCap's menu. To assess the script's repeatability, it was run ten times in succession.

At the start of each run, the script initialized the calibration counters to the following values: Start Position = 10839 and End Position = 4190. The script then iterated to find the updated ("calibrated") Start Position, which took 15-20 seconds. When that was accomplished, the script iterated again to find the updated ("calibrated") End Position, which required another 15-20 seconds.

At the completion of each run, results were recorded:

(1) Start = 10763; End = 4188; GPS Exp. (microseconds) = 1001349.8
(2) Start = 10832; End = 4176; GPS Exp. (microseconds) = 1001348.8
(3) Start = 10706; End = 4176; GPS Exp. (microseconds) = 1001350.5
(4) Start = 10907; End = 4188; GPS Exp. (microseconds) = 1001347.9
(5) Start = 10939; End = 4203; GPS Exp. (microseconds) = 1001347.6
(6) Start = 10755; End = 4226; GPS Exp. (microseconds) = 1001350.5
(7) Start = 10924; End = 4191; GPS Exp. (microseconds) = 1001347.7
(8) Start = 11092; End = 4178; GPS Exp. (microseconds) = 1001345.2
(9) Start = 10829; End = 4165; GPS Exp. (microseconds) = 1001348.5
(10) Start = 11460; End = 4174; GPS Exp. (microseconds) = 1001340.3

STATISTICAL SUMMARY:

Using MS Excel, sample means and standard deviations were calculated.

Calibration Start Pos Adjust = 10921 +/- 220
Calibration End Pos Adjust = 4187 +/- 18
GPS Exp. (microseconds) = 1001348 +/- 3.0

START POSITION IS A PROBLEM

All ten runs of the auto-calibration script settled on Start Positions in the range of 10,706 to 11,460 whose sample mean = 10,921. These positions are between 0 and the 3rd transition at 46,290. In this regime, the camera's shutter is open, and the LED's illumination of the sensor is more or less the same at all positions.

The problem is that there are no transitions (closed to open, or open to closed) at position 10,921 or anywhere near that position. Why did the auto-calibration script settle on a solution where there are no transitions?

For my camera, the true Start Position is at 158,120. This Start Position works with SharpCap exposure settings of 250ms, 500ms, 1000ms, 2000ms, and 4000ms. At each of these exposure settings, the camera's shutter always changes from closed to open at Start Position = 158,120.

Why did auto-calibration not find the closed to open transition at position 158,120?

END POSITION IS A PROBLEM, TOO

Each of the ten experimental runs produced End Position < Start Position. Now, how is it possible for an exposure to end before it begins?

Logically, auto-calibration should search positions in the vicinity of Start Position + (75,000,000 / second) * Exposure (seconds). (Note: 1 second = 75 million ticks of the counter.)

Setting Start Position = 10,921 and Exposure = 1 sec yields 75,010,921 as the expected location of End Position.

Indeed, manual calibration found the actual End Position = 75,107,060, which differs by only a small amount from the expected value.

Why did the auto-calibration script not search in the vicinity of 75,010,921 to find the exposure's open to closed transition?

GPS EXPOSURE IS NO SMALL MATTER

Ten runs of the auto-calibration experiment yielded a mean GPS Exposure = 1,001,348 microseconds. This is 0.13% larger than SharpCap's indicated exposure = 1,000,000 microseconds.

In contrast, manual calibration using Start Position = 158,120 and End Position = 75,107,060 yields GPS Exposure = 999,327.6 microseconds. (GPS Status window of the attached screenshot). This is 0.067% smaller than SharpCap's indicated exposure.

Although the absolute values of the deviations (0.13% versus 0.067%) differ by only a factor of two, it is another indicator that something might be amiss with the auto-calibration script.

SUMMARY

The auto-calibration script fails to generate calibration settings that work with my camera. The auto-calibration's settings would produce timing errors that affect the accuracy of data taken by my camera.

Issues:
1. Auto-calibration's Start Position is not located at a shutter closed to open transition.
2. Auto-calibration's End Position < Start Position, which doesn't make sense.
3. Auto-calibration's End Position = 4187, but it should be in the neighborhood of 75,010,921.
4. Auto-calibration's GPS Exposure is 0.13% larger than than nominal, but it could be better.

The auto-calibration script is a "black box" which has assumptions that appear to limit where it searches for exposure transitions. The script's assumptions need to be relaxed so that the the script works for all cameras, not just the limited subset of cameras that was used to build the script.

Ideally, the script should be a "blind solver" that quickly and efficiently identifies the two key transitions (closed to open, open to closed) where the state of the camera's electronic shutter changes.

ADDENDUM: HARDWARE & SOFTWARE

Dell Latitude E6420, x64 based PC, Intel Core i5-2520M CPU @2.50 Ghz. 8GB RAM, 500GB SD. Computer docking station includes USB3.0, serial, and parallel ports.

QHY174M-GPS camera (SN = 110007) is connected to the docking station's USB3.0 port using the QHY supplied USB3.0 cable. This camera's firmware was upgraded by me to include Altitude and NMEA data in the GPS Status window. Camera receives power from a Radio Shack 3A power supply. The GPS port of the camera is connected to the QHY supplied GPS antenna, which is housed inside a white plastic dome 100mm in diameter.

MS Windows Home 10.0.19041. SharpCap v4.0.7806 (beta)
Attachments
GPS Status SharpCap Capture.PNG
GPS Status SharpCap Capture.PNG (666.65 KiB) Viewed 1455 times
procyon12
Posts: 253
Joined: Tue Jan 14, 2020 11:32 am

Re: Automatic adjustment of the LED (QHY-174-GPS)

#18

Post by procyon12 »

Hi,

Thanks for this work, Andy - I think there is something to discuss, not only regarding the script itself but also (and much more important) the current autocalibration.
FYI, I attach the script. It is based on work by Jean-Francois and was modified in some kind by others, not by me.

Cheers, Christian
Attachments
calib_qhy.zip
(3.74 KiB) Downloaded 67 times
User avatar
admin
Site Admin
Posts: 13177
Joined: Sat Feb 11, 2017 3:52 pm
Location: Vale of the White Horse, UK
Contact:

Re: Automatic adjustment of the LED (QHY-174-GPS)

#19

Post by admin »

Hi,

the current belief is that the end calibration value that you mention (about 75,000,000) is incorrect - the values for end calibration should typically be small (often smaller than the start calibration value, although that may seem contradictory).

There is a long discussion in this thread - viewtopic.php?p=18229#p18229 - including some diagrams from QHY that explain how things are supposed to work.

thanks,

Robin
Jean-Francois
Posts: 361
Joined: Sun Oct 13, 2019 10:52 am
Location: Germany

Re: Automatic adjustment of the LED (QHY-174-GPS)

#20

Post by Jean-Francois »

Hello,

I did a lot of work in the past ... and that is in the past ... means that I need some time for remember the topic :-)
Your message is very long with a lot of tests and questions ... so, I need some days to have a look at it.
During the past months I start a different project with the 3D printing of a solar spectrometer.

The "QHY-174-GPS automatic LED setting" script is discussed in 2 different thread. And yes, it is confusing, while at the beginning the common understanding was wrong (and so my first scripts were wrong too).

I need some time to have a look at the "calib_qhy.zip" files ... also to understand how much is coming from my script and what is new.

Concerning my script ... I try to find a generic equation for the starting LED values.
In parallel, Robin does the same for the SharpCap starting values.
My script search the transition (the mid intensity of the LED illumination) near the starting LEC values.

I will respond in the next days ... I have now some free days ... but I have ~ 300 GB SER film to process (without counting the coming new film).

Regards,
Jean-Francois
Post Reply