2

I am trying to get pearson corrleation of two lists and its works quite well, but i am getting this error also. 

/usr/lib/python2.7/dist- packages/scipy/stats/_stats_mstats_common.py:93: RuntimeWarning: invalid  value encountered in sqrt
t = r * np.sqrt(df / ((1.0 - r + TINY)*(1.0 + r + TINY)))
/usr/lib/python2.7/dist-packages/scipy/stats/_stats_mstats_common.py:97: RuntimeWarning: invalid value encountered in sqrt
sterrest = np.sqrt((1 - r**2) * ssym / ssxm / df)

and thus in the results, i am unable to get pvalue and stderr values like this. 

LinregressResult(slope=0.99429193644763636, intercept=-0.12339201101613284,
rvalue=0.99998196438744391, pvalue=nan, stderr=nan)

my code is as follow

import csv
import datetime
import numpy as np
import pandas as pd
from collections import defaultdict
from datetime import datetime
import mysql.connector
from mysql.connector import errorcode
from scipy.stats import pearsonr
from scipy.stats import linregress

# for temp
dat1 = []
cnx = mysql.connector.connect(user='roby', password='xxxx', database='rob')
cursor = cnx.cursor()
cursor.execute('select  usec ,`temp.tab.1` \
           from robin_tab1 WHERE usec >= 1469833200000000 \
           AND usec <= 1469916000000000') 
rows = cursor.fetchall()
rows_arr = np.array(rows)
dat1.append([i[1] for i in rows_arr])


# for pressure
dat2 = []
cnx = mysql.connector.connect(user='roby', password='xxxx', database='rob')
cursor = cnx.cursor()
cursor.execute('select  usec ,`pres.tab1` \
           from robin_tab2 WHERE usec >= 1469833200000000 \
           AND usec <= 1469916000000000')
rows = cursor.fetchall()
rows_arr = np.array(rows)
dat2.append([i[1] for i in rows_arr])
print linregress(dat1, dat2)

the dat1 and dat2 looks like 

dat1 . .[24.335809999999999, 24.194980000000001, 24.06935, 23.915240000000001,
         23.75497,...... 23.621960000000001, 23.578779999999998]

dat2 . .[24.088159999999998, 23.918780000000002, 23.817270000000001, 23.660900000000002,
         23.505980000000001. . . .. .  24.128170000000001, 24.098600000000001]

Can someone helps me or guide me how to remove this error or deal with it if i have negative numbers in the lists because i cannot remove those negative numbers as they are a part of analysis. Thanks

robbin
  • 313
  • 1
  • 5
  • 14
  • I see your errors, but what is the relation with the posted code? Please post the code with the calls which contain these functions. – Norbert Jun 19 '17 at 00:41
  • @NorbertvanNobelen that is the code I am using. using this line produces this error. .........,print linregress(dat1, dat2). . – robbin Jun 19 '17 at 05:51
  • Got negative values in your data by accident? Or an r value >1? (sqrt of negative values) – Norbert Jun 19 '17 at 05:55
  • @NorbertvanNobelen no they are not by accident. these values are sensor readings and some values are like -4, -3 , -2.343 etc. If i am not mistaken, this error is produced due to negative values in data?. – robbin Jun 19 '17 at 06:01
  • @NorbertvanNobelen As i mention that i am getting the corrleation, but not getting pvalue and stderr. and also this warning. .I am trying to remove this warning and get pvalue also. – robbin Jun 19 '17 at 06:03
  • 1
    You will have to read the interface or code a bit: If the negative values end up in the `r` value, then you have an issue. A not unusual approach is to first normalize your data and then run it through such an algorithm. That prevents outliers and possible (for the algorithm) troublesome data to stop the calculations. – Norbert Jun 19 '17 at 15:42
  • Does this answer your question? [I am getting a warning ](https://stackoverflow.com/questions/39123766/i-am-getting-a-warning-runtimewarning-invalid-value-encountered-in-sqrt) – iacob Mar 28 '21 at 12:12

0 Answers0