3

I recently stumbled upon a similar question about how to generate a heatmap of frequencies in Python using the MatPlotLib module.

This post was very useful and I could get the individual scripts to run and create heatmaps for the random test data that was inherently generated by the code. However, I am having trouble adapting the code to create a heatmap for the data I am working with. The data is in comma-delineated format (.csv).

I currently have 3788 pairs of average quality ratings saved in this .csv file. These average quality ratings both have a range from 0 - 5. I am trying to create a heatmap that bins the data by .5 increments on both the x and y axis (0-.499, .5-.999, 1-1.499 etc).

I would like to import the first column of the .csv file (webqualityratings) to be the x-values of the heatmap and the second column of the .csv file (inpersonqualityratings) to be the y-values of the heatmap.

The code I was attempting to adapt posted by "ptomato" and edited by Mike Graham is as follows:

import numpy as np   
import numpy.random   
import matplotlib.pyplot as plt  

# Generate some test data  
x = np.random.randn(8873)  
y = np.random.randn(8873)  

heatmap, xedges, yedges = np.histogram2d(x, y, bins=50)  
extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]]  
plt.clf()  
plt.imshow(heatmap, extent=extent)  
plt.show()

If anyone could aid me in adapting this code to read in the data from my .csv file as specified, I would be eternally grateful!

Community
  • 1
  • 1
Ryan Schork
  • 31
  • 1
  • 2

2 Answers2

5

Since you have numpy at your disposal and assuming your csv files are well behaved, you can use numpy.loadtxt(),

import numpy as np   
import matplotlib.pyplot as plt  

dat = np.loadtxt('mydata.csv')

x, y = dat[:,0], dat[:,1]

heatmap, xedges, yedges = np.histogram2d(x, y, bins=50)  
extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]]  
plt.clf()  
plt.imshow(heatmap, extent=extent)  
plt.show()
lafras
  • 8,712
  • 4
  • 29
  • 28
-2

Python has an absolutely spectacular csv file library:

http://docs.python.org/library/csv.html

While I don't know the details of matplotlib, the following code will loop over a tab delimited csv file and execute your code on each first and second column.

import numpy as np
import numpy.random
import matplotlib.pyplot as plt
import csv

with open(yourInputFile, "rb") as mycsv:
    reader = csv.DictReader(mycsv, dialect='excel-tab')

    for row in reader:
        x = row['name of first column']
        y = row['name of second column']
        heatmap, xedges, yedges = np.histogram2d(x, y, bins=50)
        extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]]
        plt.clf()
        plt.imshow(heatmap, extent=extent)

plt.show()

Note that I am using the DictReader variant, which needs a header to work. Either at the beginning of the file, or as input to the constructor. You could also use the regular reader, but it uses column numbers rather than names, and that's unwieldy at 100+ fields.

You can change the dialect for your specific csv file, or even create your own, if necessary.

Finally, note that I know nothing about matplotlib, and that chunk of code is very likely wrong.

Spencer Rathbun
  • 14,510
  • 6
  • 54
  • 73