Pandas dataframe with concatenated column

Question

I have a Pandas dataframe that looks like the below code. I need to add a dynamic column that concatenates every value in a sequence before a given line. A loop sounds like the logical solution but would be super inefficient over a very large dataframe (1M+ rows).

user_id=[1,1,1,1,2,2,2,3,3,3,3,3]
variable=["A","B","C","D","A","B","C","A","B","C","D","E"]
sequence=[0,1,2,3,0,1,2,0,1,2,3,4]
df=pd.DataFrame(list(zip(ID,variable,sequence)),columns =['User_ID', 'Variables','Seq'])

# Need to add a column dynamically 
df['dynamic_column']=["A","AB","ABC","ABCD","A","AB","ABC","A","AB","ABC","ABCD","ABCDE"]

I need to be able to create the dynamic column in an efficient way based on the user_id and the sequence number. I have played with the pandas shift function and that just results in having to create a loop. Looking for some easy efficient way of creating that dynamic concatenated column.

What is this "given seqeuence" you talk about? – Celius Stingher Oct 01 '19 at 17:34 — Celius Stingher, Oct 01 '19 at 17:34

score 2 · Accepted Answer · answered Oct 01 '19 at 17:35

2

This is cumsum:

df['dynamic_column'] = df.groupby('User_ID').Variables.apply(lambda x: x.cumsum())

Output:

0         A
1        AB
2       ABC
3      ABCD
4         A
5        AB
6       ABC
7         A
8        AB
9       ABC
10     ABCD
11    ABCDE
Name: Variables, dtype: object

answered Oct 01 '19 at 17:35

Quang Hoang

146,074
10
56
74

Thank you this is perfect and super fast. – Droid Oct 01 '19 at 17:50

score 0 · Answer 2 · answered Oct 01 '19 at 17:32

0

Your question is a little vague, but would something like this work?

df['DynamicColumn'] = df['user_id'] + df['sequencenumber']

answered Oct 01 '19 at 17:32

jamzsabb

1,125
2
18
40

I doubt this is what he is looking for. – Celius Stingher Oct 01 '19 at 17:34
The above answer got me what I needed – Droid Oct 01 '19 at 17:51

Pandas dataframe with concatenated column

2 Answers2