0

I'm using the BaselineRemoval function with IModPoly, but getting different results despite using the same code and datasets. I had a friend run the code and they get the same results that I do with visual studio. Any idea why the results are different in spyder? My code is below:

Edited to simplify the code

from BaselineRemoval import BaselineRemoval

polynomial_degree = 3 #only needed for IModPoly algorithm
input_array_train = data.copy()

print(input_array_train.shape)
print(input_array_train)

baseline_correction_train=BaselineRemoval(input_array_train).IModPoly(polynomial_degree)
    
train_raman_proc = baseline_correction_train

print(train_raman_proc.shape)
train_raman_proc

I ran this same code on Spyder and Visual Studio, output is below:

Spyder:

(2042,)

[336167.00616779 335865.93248741 335585.7852715  ... 133592.04534896
 133576.80460859 133517.41650705]

(2042,)

array([40893.84981077, 40650.58294278, 40428.28300527, ...,
       39654.3524087 , 39840.20855015, 39982.01744302])

Visual Studio:

(2042,)

[336167.00616779 335865.93248741 335585.7852715  ... 133592.04534896
 133576.80460859 133517.41650705]

(2042,)

array([32338.54977607, 32129.43948405, 31941.23853284, ...,
       39768.88844912, 39896.21948952, 39979.46997397])

Notice how the input data arrays are the same, but the output baseline corrected arrays are different.

It seems to have something to do with the size of the array. If I take a subsection of the array [0:1200], I get the same results from both. And if I take the back half of the subsection [1201:2042] I also get the same results from both. But if I run the whole array (or just a larger subsection, like [0:1300]), then they give different results.

Here's the plots of the baseline corrected output from spyder (top) and visual studio (bottom), note around 1300 it seems like the imodpoly restarts for spyder.

Spyder IModPoly Ouput

Visual Studio Output

nick
  • 1
  • 2
  • Different in what way? – planetmaker Dec 21 '22 at 02:15
  • Welcome to SO. You need to provide a [mre] with debugging details including sample data, software versions, logs, etc. And why are you comparing results of the two? Check each result is a good baseline removal. Are you satisfied with both? Or only one? Or none? – relent95 Dec 21 '22 at 03:46
  • I modified the post to simplify the code (removed the loop), and added the results from spyder and visual studio. I'm comparing the results between the two because I'm trying to troubleshoot why there's a difference between what my friend is getting and what I'm getting. – nick Dec 21 '22 at 07:32
  • Any idea why the imodpoly seems to be messing up when the arra is >1300? – nick Dec 22 '22 at 00:35
  • Are Spyder and Visual Studio both using exactly the same Python interpreter and BaselineRemoval module? – nekomatic Dec 22 '22 at 12:03
  • For Visual Studio I'm using: VS: 1.74.2 Python: 3.8.8 Numpy: 1.20.1 Pandas: 0.24.1 sklearn: 1.2.4 scipy: 1.6.2 For Spyder I'm using: Spyder: 4.2.5 Python: 3.8.8 Numpy: 1.20.1 Pandas: 0.24.1 sklearn: 1.2.4 scipy: 1.6.2 – nick Dec 22 '22 at 18:21
  • What's the result you get when running your file in a system terminal with `python my_file.py`? – Carlos Cordoba Dec 22 '22 at 19:29
  • If I run it in the terminal I get the same as spyder (the incorrect results) – nick Dec 22 '22 at 20:10
  • So after some more troubleshooting, I've found the following: 1) Using the parameter polynomial = 3 for ModPoly and IModPoly, the results appear to be inaccurate when using an array greater than 1290 on a Windows machine (works fine on Mac and Linux). 2) Using the parameter polynomial = 2 for ModPoly and IModPoly, the results appear to be CORRECT (regardless of array size and operating system). – nick Dec 24 '22 at 18:07

0 Answers0