Suggestion for curve fitting

Question

I have this set of data:

X:              Y:       
0.          0.
0.001417162 0.0118
0.002352761 0.0128
0.003123252 0.0135
0.003866221 0.0138
0.004045083 0.0147
0.005544762 0.0151
0.006260197 0.0156
0.007195755 0.0157
0.007883656 0.0158
0.008805432 0.0159
0.009314465 0.0165
0.010566391 0.0168
0.011047891 0.0186
0.011666955 0.0177
0.012341036 0.0225
0.013193938 0.0399
0.013854235 0.087
0.014500764 0.1479
0.015381122 0.198
0.015601208 0.2586
0.01638525  0.3111
0.016976706 0.3693
0.017691939 0.42
0.018338382 0.4737
0.018861027 0.5223
0.01963122  0.5691
0.021625353 0.6183
0.020923988 0.6684
0.021377815 0.711
0.021927895 0.7551
0.022574222 0.7938
0.023633053 0.8382
0.023646804 0.8742
0.024279325 0.912
0.025131822 0.9495
0.0256543   0.9891
0.026094271 1.0215
0.026685464 1.0596
0.027345378 1.098
0.028101497 1.1328
0.028513912 1.1739
0.029077528 1.1997
0.029723601 1.2339
0.030355902 1.2741
0.031056901 1.3041
0.031428005 1.3383
0.032087723 1.3665
0.032692438 1.3983
0.033242157 1.4262
0.033846824 1.4589
0.034410239 1.4877
0.035248448 1.5222
0.035729364 1.5534
0.036430096 1.5861
0.037034618 1.6179
0.037694064 1.6536
0.038408425 1.6842
0.039067798 1.7121
0.039521096 1.7427
0.040207877 1.7763
0.04071607  1.8075
0.041279177 1.8381
0.04129291  1.8711
0.042707418 1.9065
0.043366544 1.9332
0.043860863 1.9659
0.044368889 1.9959
0.045055371 2.0202
0.045700624 2.0487
0.04626347  2.0796
0.047059639 2.1105
0.047540055 2.1339
0.048308673 2.1618
0.048857648 2.1849
0.049557546 2.2203
0.050229948 2.2425
0.052082233 2.2716
0.051355084 2.2983
0.051945039 2.3193
0.052466363 2.3475
0.053371748 2.3718
0.053851839 2.3937
0.054647359 2.4189
0.055072521 2.4372
0.055675941 2.4633
0.056306742 2.4882
0.057060898 2.5131
0.057691594 2.5332
0.058582712 2.5527
0.059007671 2.5755
0.059597094 2.5941
0.060172767 2.6115
0.065187502 2.6403
0.06131028  2.6592
0.061968042 2.6808
0.062598344 2.6991
0.063173791 2.7246
0.063790292 2.7441
0.064393043 2.7633
0.065091624 2.7795
0.065502522 2.8011
0.066433804 2.8212
0.066598135 2.8368
0.067351271 2.8545
0.067981104 2.8665
0.068610879 2.8845
0.069309041 2.9046
0.069870256 2.9214
0.070253498 2.9355
0.070828319 2.9499
0.07159467  2.9691
0.072046228 2.9856
0.072620893 2.9982
0.07326391  3.0108
0.073893183 3.0255
0.074467682 3.0387
0.075165218 3.0531
0.075862676 3.0654
0.076395973 3.0735
0.077230012 3.0879
0.077571798 3.0996
0.077968246 3.1116
0.078720058 3.1251
0.079485442 3.1332
0.080168736 3.1473
0.080797297 3.1524
0.081671703 3.1647
0.082518656 3.1761
0.082737205 3.1794
0.083242565 3.1911
0.083843476 3.207
0.084594523 3.2157
0.083993693 3.2217
0.086028058 3.2313
0.086806105 3.2412
0.087515804 3.2475
0.087979788 3.2562
0.089221428 3.2658
0.089289641 3.2727
0.090053572 3.2823
0.090599168 3.285
0.091485638 3.2988
0.092508299 3.3006
0.092794607 3.309
0.093585278 3.3177
0.094443969 3.3255
0.095029975 3.336
0.095752165 3.3381
0.096378886 3.3468
0.097182604 3.3498
0.097781901 3.3573
0.098585385 3.3612
0.099238967 3.3702
0.102192566 3.372
0.100464189 3.3816
0.101226388 3.3879
0.101961246 3.396
0.102519116 3.4038
0.103335387 3.4113
0.103920292 3.4134
0.104777107 3.4206
0.105416212 3.423
0.105932869 3.4308
0.106952407 3.4353
0.107550426 3.4431
0.108284246 3.4479
0.108759804 3.4539
0.109439081 3.4587
0.110118251 3.4662
0.11096027  3.4701
0.111802122 3.4749
0.112182257 3.4857
0.113132445 3.4902
0.113878868 3.4929
0.114530181 3.5028
0.11465229  3.5076
0.116063069 3.513
0.116619106 3.5181
0.117378448 3.5247
0.118246089 3.5295
0.118869589 3.5286
0.119479439 3.5364
0.120292424 3.5451
0.120807227 3.5511
0.121728283 3.5565
0.122283519 3.5625
0.123068837 3.5622
0.123745705 3.5691
0.124544254 3.5775
0.125247919 3.5796
0.125924395 3.5868
0.126614273 3.5892
0.128683133 3.5958
0.127993643 3.5991
0.128602023 3.6057
0.129507645 3.6093
0.130115772 3.6132
0.130669753 3.6156
0.131682911 3.6222
0.132263658 3.6234
0.132911821 3.6288
0.133870342 3.6291
0.136717348 3.6336
0.13504451  3.6396
0.135692157 3.6372
0.13621828  3.6432
0.137068001 3.6465
0.141030546 3.6522
0.138470251 3.6501
0.139009423 3.6567
0.140181824 3.6615
0.140532116 3.6633
0.141124835 3.6588
0.141717448 3.6675
0.14257924  3.6711
0.143319661 3.6741
0.143844585 3.6729
0.144813453 3.6753
0.145243967 3.678
0.1472209   3.6849
0.14672342  3.6879
0.147301565 3.6888
0.147866163 3.6933
0.148403782 3.6927
0.149277216 3.6972
0.149962355 3.7011
0.151922897 3.7077
0.151426165 3.7023
0.152070559 3.7089
0.153251595 3.7107
0.153587034 3.7134
0.15429804  3.7191
0.154861361 3.7191
0.155706144 3.7221
0.157086785 3.7206
0.157086785 3.7272
0.156925966 3.7269
0.158480178 3.7335
0.15912306  3.7272
0.159618519 3.7356
0.160381629 3.7359
0.161171304 3.7386
0.161853731 3.7377
0.162415608 3.7398
0.163151233 3.7437
0.163993625 3.7443
0.165062969 3.7533
0.165330242 3.7509
0.166065112 3.7512
0.166866568 3.7569
0.167494215 3.7635
0.168108371 3.7569
0.168829162 3.7638
0.169536423 3.7626
0.173429745 3.7581
0.171070418 3.7617
0.171803762 3.7653
0.172790131 3.7635
0.172936723 3.7644
0.174029244 3.7695
0.174761783 3.7701
0.175334355 3.7668
0.176173017 3.7704
0.176705362 3.7722
0.177490375 3.7701
0.178261854 3.774
0.178979922 3.7797
0.178793775 3.78
0.180774223 3.7887
0.182394664 3.8301
0.18451827  3.8169
0.186282158 3.8067
0.186772642 3.7731
0.188058046 3.7782
0.188587916 3.7734
0.189395747 3.7749
0.189885613 3.7707
0.190560674 3.7734
0.191447234 3.7752
0.192280569 3.7713
0.193007849 3.7704
0.193946376 3.7701
0.194329613 3.7734
0.195188367 3.7647
0.195716677 3.7644
0.196416507 3.7683
0.197142527 3.7695
0.197709986 3.7596
0.198923623 3.7662
0.199714786 3.7617
0.200189355 3.7626
0.2011382   3.7683
0.201915434 3.7665
0.202310538 3.7599
0.203258508 3.7656
0.204153451 3.7569
0.204903352 3.753
0.205311089 3.7512
0.206192075 3.753
0.207033292 3.7461
0.207966146 3.7539
0.208478393 3.7473
0.209318747 3.7425
0.210119413 3.7455
0.21077548  3.7443
0.21140512  3.7425
0.21234924  3.7443
0.213122589 3.7461
0.213751552 3.7443
0.214498207 3.7422
0.215192234 3.7425
0.216030006 3.7395
0.216762788 3.7437
0.217508397 3.7377
0.218096851 3.7446
0.223098604 3.7383
0.219547672 3.7386
0.220187807 3.7395
0.220906094 3.7386
0.221872127 3.7368
0.222381072 3.7404
0.223111647 3.7374
0.223789808 3.7365
0.224207027 3.7278
0.225288796 3.7359
0.225862032 3.7326
0.226487197 3.7368
0.226396039 3.7302
0.227775995 3.7308
0.228426593 3.7287
0.229115997 3.7278
0.229063975 3.7269
0.230312139 3.726
0.23101389  3.7104

plot(x,y)

Can you please give me suggestions of models (I've tried polynomials and I either get a bad fit or a overfit). Thanks!

Since I'm sharpening my whip-out-the-plots skills, I took the trouble to copy your data to a file. But for others, it would be nice to show a plot. — DarenW, Jun 12 '13 at 03:34
Have you tried this: http://creativemachines.cornell.edu/eureqa for a symbolic regression approach? Not `R` but looks cool... There is an `R` method that I've sketched [here](http://stackoverflow.com/questions/14190883/fitting-a-curve-to-specific-data/15050715#15050715) — Ben, Jun 12 '13 at 05:06
There's an important choice you should make first. Do you want to see if your data fit a proposed shape (e.g. exponential or x^n), or do you want to create a fit function that lets you predict future behavior? And in the second case, do you need to extrapolate or just interpolate? If you just want a good interpolation fit, use splines. If you want a good extrapolation, you're headed for trouble :-( . And obviously, in the first case, use the fit function you "hope for" and evaluate the error parameters. My point is: how you fit depends strongly on what your are trying to find out. — Carl Witthoft, Jun 12 '13 at 11:48
@CarlWitthoft: Yes, I agree with you. In my case I just want to get a function that fits the data and which I can use, so I have to know the equation of the function. — jpcgandre, Jun 12 '13 at 17:00

score 1 · Accepted Answer · answered Jun 12 '13 at 03:33

Just monkeying around in Python/Numpy for a few minutes, it looks like you want a formula like

Yfit(x) = Ymax * (1 - exp(-(x-x0)/a) )

x0 is where the data starts to take off from zero. Looks like x0 = 0.012 give or take a little. Ymax is the maximum value. The parameter a sets how fast the curve rises, and it look like you want a = 0.007 or so.

Polynomials are bad for any data that levels off and holds steady before or after the interesting parts. Polynomials like to wiggle, like a snake trying to go through lined-up croquet wickets. Even fitting loosely with least-squares or whatever, polynomials don't like flatness. But the shape sure looks like a constant minus a decaying exponential - very common in electronics and physics.

The initial zero values, I take to be meaningless and not needing fitting. The Yfit values you get could be clipped to zero when negative, for plotting and comparison.

If exp(-(x-x0)/a) doesn't work well enough, you could try other functions that quickly fade to zero, such as 1/(1+x^p) for some power p>=2, or use a Gaussian exp(-(x-x0)^2 / a^2)

I actually see a slight curve - the Y values go up to max, and then slightly back down. Maybe add a quadratic term to your model, like:

Y_extra_term(x) = ((x-xmax)/b)^2

where xmax is the x value where y is maximum. (BTW, I'm no expert on R, so use the correct syntax not whatever I write.)

Well, it does depend on whether one wishes to use all the data or apply a weight set (referring to your comment about ignoring (0,0) ); and for that matter whether one wishes to force the origin. — Carl Witthoft, Jun 12 '13 at 11:49

Suggestion for curve fitting

1 Answers1