How do I shift all elements in a row in a python dataframe over by one column?

Question

Let's say I have the following dataframe.

   one  two   ...  ten
a  1.0  1.0   ...  1.0
b  2.0  2.0   ...  NaN
c  3.0  3.0   ...  NaN
d  4.0  4.0   ...  4.0

I want to iterate (maybe I don't have to?) over the dataframe, moving all rows with a Null in the far-right column one to the right, so that the NaN is instead in the first column. Specifically, I want all the values to move one over to the right, so that only the NaN in the far-right column is overwritten.

   one  two   ...  ten
a  1.0  1.0   ...  1.0
b  NaN  2.0   ...  2.0
c  NaN  3.0   ...  3.0
d  4.0  4.0   ...  4.0

Thank you for any and all answers, I'm pretty new to coding so I appreciate it.

Looks like a duplicate of [this question](https://stackoverflow.com/questions/44558215/python-justifying-numpy-array/44559180#44559180) albeit that one is for numpy, you can still use it here. — Quang Hoang, Nov 11 '21 at 17:54

Jacopo Repossi · Accepted Answer · 2021-12-01T22:28:15.157

If I understood your request correctly, you can exploit the shift and update methods like this:

# df

    one     two     ten
a   1.0     1.0     1.0
b   2.0     2.0     NaN
c   3.0     3.0     NaN
d   4.0     4.0     4.0

df.update(
          df.loc[df.iloc[:, -1].isna(), :].shift(axis=1).replace(np.nan, 'na')
)
df.replace('na', np.nan, inplace=True)
print(df)

# Output
    one     two     ten
a   1.0     1.0     1.0
b   NaN     2.0     2.0
c   NaN     3.0     3.0
d   4.0     4.0     4.0

Let's break it down:

#### Step1: Filter NaN in the last column

df.loc[df.iloc[:, -1].isna(), :]

# Ouput 
    one     two     ten
b   2.0     2.0     NaN
c   3.0     3.0     NaN


#### Step2: Shift the rows to the right

df.loc[df.iloc[:, -1].isna(), :].shift(axis=1)

# Output
    one     two     ten
b   NaN     2.0     2.0
c   NaN     3.0     3.0


#### Step3: Replace NaN with "na"
# This is because the update function doens't replace values with NaN

df.loc[df.iloc[:, -1].isna(), :].shift(axis=1).replace(np.nan, 'na')

# Output
    one     two     ten
b   na      2.0     2.0
c   na      3.0     3.0


#### Step4: Update the orginal dataframe with this new one (in place method)

df.update(df.loc[df.iloc[:, -1].isna(), :].shift(axis=1).replace(np.nan, 'na'))

# Output
    one     two     ten
a   1.0     1.0     1.0
b   na      2.0     2.0
c   na      3.0     3.0
d   4.0     4.0     4.0

#### Step5: Replace back the original NaN

df.replace('na', np.nan, inplace=True)

How do I shift all elements in a row in a python dataframe over by one column?

1 Answers1