0

I started to work with pandas very recently, and my issue is the following: I have two loops, that generates each 10 values. What I want to do is to insert at the bottom of my data frame the generated values, in such a way that the index is the same for both loops.

Here is a mock-up example, that is quite close of what I'm trying to do:

import pandas as pd
import random

randint = {'rand': [10,52,99,8],'rand2': [541,632,789,251], 'rand3': [1,3,4,1]}
df = pd.DataFrame(randint, columns = ['rand','rand2', "rand3"])

i = j = len(df)
for x in range(10):
    rand = random.randint(1,101)
    rand2 = random.randint(1,1001)
    df.loc[df.index[i], "rand"] = rand
    df.loc[df.index[i], "rand2"] = rand2
    i = i + 1

for y in range(10):
    rand3 = random.randint(1,11)
    df.loc[df.index[j], "rand3"] = rand3
    j = j + 1

print(df)  

So, what I would like is to have for instance at row 5 the first set of rand, rand2, rand3 at the same row, and so forth (e.g.: for x and y = 1, I would have the three values at the same row, for x and y = 2, same thing, etc...). The issue is that I have read that it was not a good idea to iterate with pandas (and obviously, pandas is raising me the error "index 4 is out of bounds for axis 0 with size 4"), but I really have trouble to understand the pandas syntax and I'm a bit lost on how I am supposed to tackle this issue. Thank you for your help.

Expected output:

At first, my dataframe would look like this:

rand rand2 rand3
10 541 1
52 632 3
99 789 4
8 251 1

Now, let us imagine that the first time in the first loop (so for x = 1) , rand = 8, rand2 = 455, and for the first time of the second loop (so for y=1), rand3 = 7.

So now, I would like to add the values obtained to the last row, in such a way that my dataframe would look like this:

rand rand2 rand3
10 541 1
52 632 3
99 789 4
8 251 1
8 455 7

The issue is that I don't really know to indicate to pandas that I want to have the same index for the two loops. Let me know if it is still not clear.

smci
  • 32,567
  • 20
  • 113
  • 146
Jauhnax
  • 95
  • 1
  • 10
  • Can you post some expected output? Is it that you expect `row5` to be equal to be `10` + `541` + `1`? [Check this link](https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples) – Danail Petrov Jan 03 '21 at 11:53
  • For instance, if rand = 8, rand2= 455 and rand3= 7, I would like to have at row 5: 8, 455, 7. And so forth for the next set of values of rand, rand2 and rand3. – Jauhnax Jan 03 '21 at 11:55
  • **Use `df.concat()`.** You never need to do this iterative indexing with `for x in range(len(df))` and `.loc[...]`. – smci Jan 03 '21 at 13:06
  • 1
    This answers your question, see the `concat` section. [Pandas Merging 101](https://stackoverflow.com/questions/53645882/pandas-merging-101) – smci Jan 03 '21 at 13:07

1 Answers1

0

I am not sure if this is the case, but in your example the second loop is working on rand2 column and not rand3. Also, when you use df.loc use the value of i or j (not df.index[i/j])

i = j = len(df)
for x in range(10):
    rand = random.randint(1,101)
    rand2 = random.randint(1,1001)
    df.loc[i, "rand"] = rand
    df.loc[i, "rand2"] = rand2
    i = i + 1
for y in range(10):
    rand3 = random.randint(1,11)
    df.loc[j, "rand3"] = rand3
    j = j + 1
print(df)  

    rand  rand2  rand3
0   10.0  541.0    1.0
1   52.0  632.0    3.0
2   99.0  789.0    4.0
3    8.0  251.0    1.0
4   37.0  902.0    6.0
5   65.0  717.0   11.0
6   95.0  345.0    6.0
7   81.0  218.0    9.0
8   90.0  233.0   10.0
9   15.0  918.0    6.0
10  62.0  775.0   10.0
11  27.0  955.0    4.0
12  43.0   17.0    2.0
13  69.0   41.0    8.0

Pandas way of doing this is like that:

>>> import numpy as np
>>> pd.DataFrame({
    'rand': np.random.randint(1,101,10), 
    'rand2': np.random.randint(1,1001,10), 
    'rand3': np.random.randint(1,11,10)})

   rand  rand2  rand3
0    50    877      5
1     9    929      5
2    23    605      7
3    52    205      4
4    39    341      6
5    17    455      7
6    11    505      7
7    68    647     10
8    66    920      6
9    63    386      9
Danail Petrov
  • 1,875
  • 10
  • 12
  • Oh shoot, the rand2 was a typo! I'm not at my place but I'll try your code as soon as I can. – Jauhnax Jan 03 '21 at 12:36
  • The OP clearly just needs `df.concat()`. This should be a one-liner. – smci Jan 04 '21 at 09:20
  • 100%. I agree this is not the case, but the problem in his question was the actual code. Whether or not it's a good/bad code - there is no doubt about it. – Danail Petrov Jan 04 '21 at 11:01