0

Say I have an array of numpy arrays df and I want to list the pairs of consecutive rows? How do I do that?

I only know how to list each row one at a time and combine them into a list like this: z=[x for x in df]. Then I'd do [np.vstack(z[i],z[i+1]) for i in range(len(z))].

I'm just wondering if there's a Pythonic way to do this instead.

My df is for example

array([[1., 1., 1., 0., 0., 0., 1., 0., 1.],
       [0., 2., 0., 0., 6., 0., 0., 2., 0.],
       [0., 2., 0., 0., 6., 0., 0., 2., 0.],
       [0., 3., 4., 9., 1., 2., 2., 1., 8.],
       [0., 0., 5., 0., 0., 0., 5., 9., 0.]])

And I'd like to return pairs of consecutive arrays at a time.

*** UPDATE

Okay so now I'm trying to return a dataframe where the columns are (first six entries in the row, the 8th entry, the 7th entry, the 9th entry, and the first six entries in the subsequent row).

When I do pd.DataFrame([[np.array([df1[1,:,:][j][:6],df1[1,:,:][j][7],df1[1,:,:][j][6],df1[1,:,:][j][8],df1[1,:,:][j+1][:6]])] for j in range(len(df1[1,:,:])-1)]) it looks fine and I get

                                                    0
0   [[1.0, 1.0, 1.0, 0.0, 0.0, 0.0], 0.0, 1.0, 1.0...
1   [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 0.0, 0.0, 0.0...
2   [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 0.0, 0.0, 0.0...
3   [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 0.0, 0.0, 0.0...
4   [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0], 0.0, 0.0, 0.0...

.

However, when I do

pd.DataFrame([[np.array([df1[1,:,:][j][:6],df1[1,:,:][j][7],df1[1,:,:][j][6],df1[1,:,:][j][8],df1[1,:,:][j+1][:6]])] for j in range(len(df1[1,:,:])-1)],columns=['s','a','r','absorb','ns']) (the only difference is that I provide column names now), I get the error

AssertionError: 5 columns passed, passed data had 1 columns.

How do I work around this? Thanks.

Kashif
  • 3,063
  • 6
  • 29
  • 45
  • 3
    What causes the rows to be consecutive, is there an index? Can you provide an example df with columns? – dasvootz Aug 15 '18 at 20:46
  • So the array is inside a dataframe or it is numpy array prior to being appended to the dataframe? Also what do you want your end output to look like? How many columns in the end? – dasvootz Aug 15 '18 at 21:04
  • No dataframe. It's just a numpy array of numpy arrays. – Kashif Aug 15 '18 at 21:06
  • On your output, I only see one column "0" that is why you are getting an Assertion Error. Need to create 5 columns in pd.DataFrame. Can you show on your desired output something with the columns and output below them. Thanks – dasvootz Aug 16 '18 at 13:22
  • Yeah I ended up fixing this. `pd.DataFrame([np.array([df1[i,:,:][j][:6],df1[i,:,:][j][7],df1[i,:,:][j][6],df1[i,:,:][j][8],df1[i,:,:][j+1][:6]]) for j in range(len(df1[i,:,:])-1)],columns=['state','action','reward','absorb','next state'])` – Kashif Aug 16 '18 at 21:33

0 Answers0