0

this is my code and this is my data, and this is the output of the code. I've tried adding one the values on the x axes, thinking maybe values so little can be interpreted as zeros. I've no idea what true_divide could be, and I cannot explain this divide by zero error since there is not a single zero in my data, checked all of my 2500 data points. Hoping that some of you could provide some clarification. Thanks in advance.

import pandas as pd
import matplotlib.pyplot as plt
from iminuit import cost, Minuit
import numpy as np

frame = pd.read_excel('/Users/lorenzotecchia/Desktop/Analisi Laboratorio/Analisi     dati/Quinta Esperienza/500Hz/F0000CH2.xlsx', 'F0000CH2')
data =  pd.read_excel('/Users/lorenzotecchia/Desktop/Analisi Laboratorio/Analisi     dati/Quinta Esperienza/500Hz/F0000CH1.xlsx', 'F0000CH1')
# tempi_500Hz = pd.DataFrame(frame,columns=['x'])
# Vout_500Hz = pd.DataFrame(frame,columns=['y'])
tempi_500Hz = pd.DataFrame(frame,columns=['x1'])
Vout_500Hz = pd.DataFrame(frame,columns=['y1']) 
# Vin_500Hz = pd.DataFrame(data,columns=['y'])

def fit_esponenziale(x, α, β):
    return α * (1 - np.exp(-x / β))

plt.xlabel('ω(Hz)')
plt.ylabel('Attenuazioni')
plt.title('Fit Parabolico')
plt.scatter(tempi_500Hz, Vout_500Hz)
least_squares = cost.LeastSquares(tempi_500Hz, Vout_500Hz, np.sqrt(Vout_500Hz), fit_esponenziale)
m = Minuit(least_squares, α=0, β=0)
m.migrad()
m.hesse()
plt.errorbar(tempi_500Hz, Vout_500Hz, fmt="o", label="data")
plt.plot(tempi_500Hz, fit_esponenziale(tempi_500Hz, *m.values), label="fit")
fit_info = [
f"$\\chi^2$ / $n_\\mathrm{{dof}}$ = {m.fval:.1f} / {len(tempi_500Hz) - m.nfit}",]

for p, v, e in zip(m.parameters, m.values, m.errors):
    fit_info.append(f"{p} = ${v:.3f} \\pm {e:.3f}$")

plt.legend()
plt.show()

input

output and example of data

mikuszefski
  • 3,943
  • 1
  • 25
  • 38
Lorenzo
  • 3
  • 3
  • Hi Lorenzo. A few comments. 1) Try to give a minimum working example, i.e. if possible provide a minimal data set that results in the same error. Images of excel sheets are not helping. 2) Your data is negative, so an offset in the fit function is probably a good idea. 3) the search algorithm in the non-linear fit may test `beta=0`, therefore producing division by zero. better using `y = off + a * ( 1 - exp( - b * x ) )`. The error propagate `b` to `beta`. – mikuszefski Dec 02 '21 at 07:51
  • Hi! really thank you for the answer, so the the offset used in my fit should be the same one I would be putting to adjust my data, in order to be non negative? – Lorenzo Dec 02 '21 at 15:09
  • sure, you can invert the problem like this...it will be positive within error margins – mikuszefski Dec 02 '21 at 15:51
  • unfortunately adding 10 to my values didn't work, and I've got the same output...., I thought doing as I said in my previous reply was the same thing you said in your first replay, evidently I didn't understand in the first place – Lorenzo Dec 03 '21 at 14:20
  • just try with `def fit( x, a, b, c) : return a * ( 1- exp( - b * x ) ) + c` – mikuszefski Dec 03 '21 at 14:54
  • doesn't work either.... at this point I'm clueless. Nevertheless, thanks for your answers – Lorenzo Dec 03 '21 at 15:02
  • do you need to use iminuit or would scipy.optimize suffice. Can you check the shape of your input data – mikuszefski Dec 03 '21 at 15:20
  • I think for my level of analysis the two methods are equivalent, although I've never succeeded using SciPy due to the unclarity of the use of the sigma, which I'm trying to understand in the other question I've posted, for which I thank you again. I can check the shape of my input data, I'm pretty sure is unidimensional – Lorenzo Dec 03 '21 at 15:33
  • is Vout the y value? in the form posted above I guess the third entry in cost.LeastSquares is the error. but you put negative values into a root? – mikuszefski Dec 03 '21 at 15:35
  • I'd also say that beta=0 in Minuit() was a problem initially. – mikuszefski Dec 03 '21 at 15:41
  • yes it is. You're right about the root, but so far I've changed the third entry into cost.LeastSquares now it's the 3% of Vout with is the proper error of the y values. – Lorenzo Dec 03 '21 at 15:45
  • I'm so sorry, when you said Beta = 0 you meant β = 0 I thought it was a Minuit check in fact I was scavenging the documents for this.... I feel so stupid hahah, thanks a lot for the support, the code doesn't give my any error right now but it's still an horizontal line, maybe the function for the fit itself is wrong. thanks a lot for you're help, this was so important to me because it's a school project, I'm doing the data analysis of a lab class with python but my professor (an experimental physicist ) doesn't know how to use it lol – Lorenzo Dec 03 '21 at 15:51
  • Well, today I do not have time as the docs for this package are lacking some clarity. Maybe next week I have a look. But if it is only about fitting...use scipy.optimize.curve_fit with absolute_sigma=True – mikuszefski Dec 03 '21 at 15:51
  • @mikuszefski could you show me a code example for the fit with SciPy, since with iminuit the output fit is an horizontal line? I'm suspecting it could be the bidimensional size of my input arrays.... thanks in advance – Lorenzo Dec 04 '21 at 17:35

2 Answers2

0

Here is a working Minuit vs curve_fit example. I scaled the function such that the decay in the exponential is in the order of 1 (generally a good idea for non linear fits ). Eventually, both methods give very similar results.

Note:I leave it open whether the error makes sense like this or not. The starting values equal to zero was definitively a bad idea.

import pandas as pd
import matplotlib.pyplot as plt
from iminuit import cost, Minuit
from scipy.optimize import curve_fit
import numpy as np

def fit_exp(x, a, b, c):
    return a * (1 - np.exp(- 1000 * b * x) ) + c

nn = 170
xl = np.linspace( 0, 0.001, nn )
yl = fit_exp( xl, 15, 5.3, -8.1 ) + np.random.normal( size=nn, scale=0.05 )

#######################
### Minuit
#######################
least_squares = cost.LeastSquares(xl, yl, np.sqrt( np.abs( yl ) ), fit_exp )
print(least_squares)
m = Minuit(least_squares, a=1, b=5, c=-7)
print( "grad: ")
print( m.migrad() ) ### needs to be called to get fit values
print( m.values )### gives slightly different output
print("Hesse:")
print( m.hesse() )

#######################
### curve_fit
#######################

opt, cov = curve_fit(
    fit_exp, xl, yl, sigma=np.sqrt( np.abs( yl ) ),
    absolute_sigma=True
)
print( " curve_fit: ")
print( opt )
print( " covariance ")
print( cov )

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1 )
ax.plot(xl, yl, marker ='+', ls='' )

ax.plot(xl, fit_exp(xl, *m.values), ls="--")
ax.plot(xl, fit_exp(xl, *opt), ls=":")


plt.show()
mikuszefski
  • 3,943
  • 1
  • 25
  • 38
0

There is actually a way to fit this completely linear. See e.g.here

import matplotlib.pyplot as plt
import numpy as np

from scipy.integrate import cumtrapz



def fit_exp(x, a, b, c):
    return a * (1 - np.exp( -b * x) ) + c

nn = 170
xl = np.linspace( 0, 0.001, nn )
yl = fit_exp( xl, 15, 5300, -8.1 ) + np.random.normal( size=nn, scale=0.05 )

"""
with y = a( 1- exp(-bx) ) + c
we have Y = int y = -1/b y + d x + h ....try it out or see below
so we get a linear equation for b (actually 1/b ) to optimize
this goes as:
"""

Yl = cumtrapz( yl, xl, initial=0 )
ST = [xl, yl, np.ones( nn ) ]
S = np.transpose( ST )

eta = np.dot( ST, Yl )
A = np.dot( ST, S )

sol = np.linalg.solve( A, eta )
bFit = -1/sol[1]
print( bFit )
"""
now we can do a normal linear fit
"""

ST = [ fit_exp(xl, 1, bFit, 0), np.ones( nn ) ]
S = np.transpose( ST )
A = np.dot( ST, S )
eta = np.dot( ST, yl )
aFit, cFit = np.linalg.solve( A, eta )

print( aFit, cFit )
print(aFit + cFit, sol[0] ) ### consistency check

fig = plt.figure()
ax = fig.add_subplot(1, 1, 1 )
ax.plot(xl, yl, marker ='+', ls='' )
## at best a sufficient fit, at worst a good start for a non-linear fit
ax.plot(xl, fit_exp( xl, aFit, bFit, cFit ) ) 

plt.show()

"""
a ( 1 - exp(-b x)) + c = a + c - a exp(-b x) = d - a exp( -b x )
int y = d x + a/b exp( -b x ) + g
    = d x +a/b exp( -b x ) + a/b - a/b + c/b - c/b + g
    = d x - 1/b ( a - a exp( -b x ) + c ) + c/b + a/b + g
    = d x + k y + h
with k = -1/b and h = g + c/b + a/b.
d and h are fitted but not used, but as a+c = d we can check 
for consistency
"""


mikuszefski
  • 3,943
  • 1
  • 25
  • 38
  • thanks again for the help, although I don't understand the consistency check since my output offers a different result from the one on the line exactly above – Lorenzo Dec 09 '21 at 08:38
  • The first fit is a trick to get the exponential decay as linear fit. Usually this introduces other linear fit parameter that are of no use for the original problem. They are discarded. After knowing `b` fitting `a` and `c` is a simple linear task. In this case, interestingly, one of the additional parameters popping up in the first fit is `a + c`. So for our peace of mind, we could check if this parameter is consistent with `a + c` from the second fit. No need to do so. I just wanted to mention this for completeness. – mikuszefski Dec 09 '21 at 09:52
  • thanks, you're my saver !! – Lorenzo Dec 09 '21 at 16:13