2

I am currently struggling with plotting my linear regression output. I found a similar problem and the recommendation was to make sure that the data type was set to int. I have made sure to incorporate that into my code.

I have gone through the code a number of times and the structure seems sound to me. I am open to any and all feedback! Thank you so much for your help!

Please note that the columns (Accident_Severity and Number_of_Casualties) are simply numbers. (i.e. The severity of the accident was 3 and 1 casualty was involved).

-------------------Step1-------------------

import numpy as np
import pandas as pd
from sklearn.linear_model import LinearRegression
%pylab inline
import matplotlib.pyplot as plt

-------------------Step2-------------------

raw_data = pd.read_csv("/Users/Maddco12/Desktop/1-6m-accidents-traffic-flow-over-16-years/accidents_2005_to_2007.csv")
dtype={'Number_of_Casualties': int,'Accident_Severity': int}
raw_data.head(4)

-------------------Step3-------------------

filtered_data = raw_data[~np.isnan(raw_data["Accident_Severity"])] #removes rows with NaN in them
filtered_data.head(4)

filtered_data = raw_data[~np.isnan(raw_data["Number_of_Casualties"])] #removes rows with NaN in them
filtered_data.head(4)

-------------------Step4-------------------

npMatrix = np.matrix(filtered_data)
X, Y = npMatrix[:,0], npMatrix[:,1]
mdl = LinearRegression().fit(filtered_data[['Number_of_Casualties']],
filtered_data.Accident_Severity)
m = mdl.coef_[0]
b = mdl.intercept_
print "formula: y = {0}x + {1}".format(m, b)

-------------------Step5------------------- (I get the Value Error here)

plt.scatter(X,Y, color='blue')
plt.plot([0,100],[b,m*100+b],'r')
plt.title('Linear Regression Example', fontsize = 20)
plt.xlabel('Number of Casualties', fontsize = 15)
plt.ylabel('Accident Severity', fontsize = 15)
plt.show()

Error as follows---->

ValueError                                Traceback (most recent call last)
<ipython-input-10-5bf84a35de3d> in <module>()
----> 1 plt.scatter(X,Y, color='blue')
      2 plt.plot([0,100],[b,m*100+b],'r')
      3 plt.title('Linear Regression Example', fontsize = 20)
      4 plt.xlabel('Number of Casualties', fontsize = 15)
      5 plt.ylabel('Accident Severity', fontsize = 15)

/Users/Maddco12/Documents/Python/anaconda/lib/python2.7/site-packages/matplotlib/pyplot.pyc in scatter(x, y, s, c, marker, cmap, norm, vmin, vmax, alpha, linewidths, verts, edgecolors, hold, data, **kwargs)
   3256                          vmin=vmin, vmax=vmax, alpha=alpha,
   3257                          linewidths=linewidths, verts=verts,
-> 3258                          edgecolors=edgecolors, data=data, **kwargs)
   3259     finally:
   3260         ax.hold(washold)

/Users/Maddco12/Documents/Python/anaconda/lib/python2.7/site-packages/matplotlib/__init__.pyc in inner(ax, *args, **kwargs)
   1817                     warnings.warn(msg % (label_namer, func.__name__),
   1818                                   RuntimeWarning, stacklevel=2)
-> 1819             return func(ax, *args, **kwargs)
   1820         pre_doc = inner.__doc__
   1821         if pre_doc is None:

/Users/Maddco12/Documents/Python/anaconda/lib/python2.7/site-packages/matplotlib/axes/_axes.pyc in scatter(self, x, y, s, c, marker, cmap, norm, vmin, vmax, alpha, linewidths, verts, edgecolors, **kwargs)
   3836 
   3837         # c will be unchanged unless it is the same length as x:
-> 3838         x, y, s, c = cbook.delete_masked_points(x, y, s, c)
   3839 
   3840         scales = s   # Renamed for readability below.

/Users/Maddco12/Documents/Python/anaconda/lib/python2.7/site-packages/matplotlib/cbook.pyc in delete_masked_points(*args)
   1846         return ()
   1847     if (is_string_like(args[0]) or not iterable(args[0])):
-> 1848         raise ValueError("First argument must be a sequence")
   1849     nrecs = len(args[0])
   1850     margs = []

ValueError: First argument must be a sequence.
Maddco12
  • 21
  • 1
  • 3

2 Answers2

1

I suggest check the X and Y values before plotting them. Rest of your code looks straight forward so most likely the issue is over there.

The Scatter plot expect an array of values for X and Y

https://matplotlib.org/api/_as_gen/matplotlib.pyplot.scatter.html

try this and see if it works

plt.scatter([X],[Y], color='blue')
Manjit Ullal
  • 106
  • 8
  • Unfortunately this didn't create the result expected. Here is the following error produced. ValueError: invalid literal for float(): 200797UD71405 – Maddco12 Feb 24 '18 at 07:33
1

Perhaps you should check your csv file. If you generated it with an old Excel version, you might get this kind of error. I solved this problem by loading up my csv to Googlespreadsheets and then exporting it again as (a better) csv file. There seems to be a weird incompatibility with some csv file types and some versions of python. Here you have a valuable discussion on this problem: Excel to CSV with UTF8 encoding. Hope it helps.

meereslicht
  • 23
  • 3