1

I have a pandas data frame that looks like:

X,Y,VAL
3,1,1221.231
3,3,121.2
3,4,4354.2
3,...,...
3,1200,12.1
...
5,1,756.3
5,2,12.01
5,...,...
...,...,...
123,110,23.1
123,1119,65.9

Note that x,y values are in the first and second column, different from what pyplot imshow expects (a multi dimensional array).

We have in the first n lines all Y values to the first X coordinate an the after that cames the second X value and all Y values related to that line, the thing goes on until it encounters the latest line. The dots here represents that data continues in more rows.

The values on third column it goes the measured quantity over the map.

I have tried to iterate over or build a mesh using the method "unique" from pandas library to built the coordinates, but the non regularity of source turned thing complicated

IS there any functions able to process this in something "graphable" in imshow or convert it to another kind of table/matrix?

Using one of the proposed solutions isn't viable because the coloring can't be interpolated. My source mesh isn't so sparse, but unfortunately I can't garantee that is regular. (but it can be in some cases).

But let's suppose that I have some data like

x         y      value

   64        4     2743
   64        8     3531
   64       16     4543
   64       32     5222
   64       64     5730
  128        4     2778
  128        8     3500
  128       16     4657

Is there any function able to convert this table to imshow compatible? (line/column based on x/y values) or I need to iterate over it ?

After try the proposed solution, I came to a optimization issue, also Iwas not able to create the v_mesh on attempt to use pcolormesh. It can be seen here Optimizing non regularized data reading to image as another question.

Community
  • 1
  • 1
Lin
  • 1,145
  • 11
  • 28
  • For image (imshow) you have to have equidistant steps in X and Y. – Severin Pappadeux Feb 03 '16 at 22:19
  • @SeverinPappadeux lets by hypotesis create 2 scenarios: 1. My actual scenario, is there any function able to plot an image based on that? 2. I can extract the data, build a new table based on min/max values and filling it with NaNs where data isn't found. Is there any function that can be used to convert this format to another one that is easily parsable by matplotlib? In fact im trying to avoid any kind of for iteraction over this data source. Any hint is welcome. – Lin Feb 03 '16 at 22:27
  • 1
    http://stackoverflow.com/questions/27004422/contour-imshow-plot-for-irregular-x-y-z-data – M4rtini Feb 03 '16 at 22:31
  • @M4rtini thank you for the link, but I want to avoid to plot gradients. I just want to see the information that I want (right now ;-), no kind of interpolation cannot be used in this case. – Lin Feb 03 '16 at 22:33
  • 1
    Would matplotlib.pyplot.scatter work? You can adjust the size or colour of each point based on the value http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.scatter – Jezzamon Feb 03 '16 at 23:22
  • There are many ways to convert your dataframe to something imshow-compatible. What is the expected output of the function you are looking for when applied to your sample data? – Stop harming Monica Feb 03 '16 at 23:44
  • I expect a image on the output, not a plot. I'm evaluating values over a sensor mesh. – Lin Feb 04 '16 at 15:34
  • Something like this: http://matplotlib.org/1.5.1/examples/pylab_examples/colorbar_tick_labelling_demo.html but with real data (not gausian noise). My two main issues is: Data formating, often functions require a two dimensional x,y array and the possibility of non regularized data. To the first issue, I'm looking if there is a functions that convert what I have, to the second issue is how to insert a non regularized data to a function (but I can try regularize it – Lin Feb 04 '16 at 15:40

2 Answers2

1

But let's suppose that I have some data like

Ok, second dataset could be easily fit into image array. Basically, if you use log2(x) and log2(y) instead of x and y, it fits easily into image. So fill image with data and use log scales.

Something along the line (untested code!)

x = ...
y = ...
v = ...

xmin = x[0]
xmax = x[-1]
ymin = y[0]
ymax = y[-1]

nx = len(x.unique())
ny = len(y.unique())

img = np.zero((nx, ny))
for ix in range(0, nx):
    for iy in range(0, ny):
        v_idx = ix*ny + iy
        img[ix, iy] = v[v_idx]

plt.imshow(img, extent=[xmin, xmax, ymin, ymax], cmap=plt.cm.Reds, interpolation='none')
plt.xscale('log')
plt.yscale('log')

plt.show()

UPDATE

BTW, is using imshow() a hard requirement? You might want to take a look at pcolormesh()

UPDATE II

When to use imshow over pcolormesh?

Community
  • 1
  • 1
Severin Pappadeux
  • 18,636
  • 3
  • 38
  • 64
  • no imshow isn't hard requirement, but looks like it was the right way to do it. I'm really looking for directions, from there I can walk. I just don't want to code a lot of stuff and discover that there is a python function that does everything I did (and better ) :-) – Lin Feb 04 '16 at 15:41
  • @Lin Well, I would try code I proposed, then look at `pcolor` or `pcolormesh` (`pcolormesh` is better, IMHO). As far as I know, three mentioned functions together with, maybe, `contourf()` are the only game in town, unless you're willing to go to 3D. – Severin Pappadeux Feb 04 '16 at 15:54
  • After looking to pcolormesh() looks like the right way to go. And no, no 3d. – Lin Feb 04 '16 at 16:34
-1

enter image description here

from __future__ import division
from mpl_toolkits.mplot3d.axes3d import Axes3D
from pylab import *

buf = '''64 4   2743
64  8   3531
64  16  4543
64  32  5222
64  64  5730
128 4   2778
128 8   3500
128 16  4657'''
data = np.array([map(int, l.split('\t')) for l in buf.splitlines()])
x = data[:,0]
y = data[:,1]
z = data[:,2]


clf()
ax = subplot(projection='3d')
c = (z/z.max()*256)
# ex1
ax.scatter(x,y,z,c=cm.jet(c), marker='s')
ax.view_init(0,0)
draw()

Axes3D takes (x,y,z) input so you can get away with no interpolation and no iteration. Is this overkill? Actually just normal 2d scatter with color should also be enough... If you use square marker 's' and do axis('off'), it becomes more similar to imshow.

otterb
  • 2,660
  • 2
  • 29
  • 48