0

I'm preparing a dataset right now and I need it to have a certain format, such as this:

Hand Pose
No Seating Back
Yes Seating Back
No Seating Back
Yes Seating Back
No Seating Back
Yes Seating Back

However, currently it's producing

0
Hand ['No', 'No', 'No', 'No', 'No', 'No', 'No']
Pose ['Seating Back', 'Seating Back', 'Seating Back']

As you can see the values inside the array stored in one cell while I need it to be un-nested in a where one value occupies one cell.

My code:

array_all = {'Hand': alldata, 'Pose': keypoints}
df = pd.Series(array_all)

# Keypoint detection
df.to_csv('test.csv',
          mode='w',
          header=True,
          index=True)
df.transpose()

To give context to the object, here is a snippet on one of them

for data_point in results.face_landmarks.landmark:
    if 0.6 <= data_point.x < 0.8:
        face_position.append('Straight')
    else:
        face_position.append('Angled')

All arrays are being appended in this manner.

Dinuli Thalakumbura
  • 111
  • 1
  • 11
  • Thanks for adding the actual text. However, now `"Face Pose"` is missing – wjandrea Nov 21 '21 at 19:09
  • What's your desired output for the jagged data? Do you want to fill with the empty string or another sentinel value, or omit the rows missing data? – wjandrea Nov 21 '21 at 19:23
  • 1
    The jagged data should not have any kind of value, it can present itself as an empty string – Dinuli Thalakumbura Nov 21 '21 at 19:31
  • Does this answer your question? [Creating dataframe from a dictionary where entries have different lengths](/q/19736080/4518341). What worked for me is `df = pd.DataFrame.from_dict(array_all, orient='index').T` (from [dezzan's answer](/a/25217425/4518341)), which fills with `None`, which get converted to the empty string in the CSV. – wjandrea Nov 21 '21 at 19:40

1 Answers1

1

You're creating a Series instead of not a DataFrame. You want to create a DataFrame.

Change

df = pd.Series(array_all)

to

df = pd.DataFrame(array_all)

Edit: If you can't create a DataFrame because the arrays are not all of equal length, add empty strings to them until they all are as long as the longest of them, like this:

array_all = {'Hand': alldata, 'Pose': keypoints, 'Face Pose': face_position}

# Pad the arrays with '' so that they are all the same length
max_size = max([len(array) for array in array_all.values()])
for array in array_all.values():
    array.extend([''] * (max_size - len(array)))

df = pd.DataFrame(array_all)

Or, a better solution:

array_all = {'Hand': alldata, 'Pose': keypoints, 'Face Pose': face_position}
array_all = {k: pd.Series(v) for k, v in array_all.items()}
df = pd.DataFrame(array_all)

Last snippet partially credited to @Jeff's answer here.

  • Why are you using the empty string instead of say, `None`? – wjandrea Nov 21 '21 at 19:18
  • I'm assuming that Dinuli wants the rows to be empty. I'm assuming if I use `np.nan` or `None` they'll show up as `NaN` or `None` in the CSV file. Am I wrong? –  Nov 21 '21 at 19:19
  • Thank you, Your solution works to an extent, for making it in columns, however, I'm still having the issue of unnesting the array where one value of the array populates one cell each rather than the entire array populating one cell (cell in CSV as shown at the bottom of the question above). Do you have any idea about this? Thank you! – Dinuli Thalakumbura Nov 21 '21 at 19:22
  • Can you share a link to a screenshot? –  Nov 21 '21 at 19:23
  • https://i.stack.imgur.com/p0fud.png this is how it looks right now where the entire object just populates a cell – Dinuli Thalakumbura Nov 21 '21 at 19:25
  • Are you doing it like `'Hand': [alldata]`? Don't do the brackets around `alldata` like that. Use my code exactly. It should do what you want. –  Nov 21 '21 at 19:38
  • I'm not sure if it is unconventional but @user17242583, can you please check https://stackoverflow.com/questions/70069614/pandas-add-additional-column-to-an-existing-csv-file , appreciate it, Thanks! – Dinuli Thalakumbura Nov 22 '21 at 17:17
  • 1
    It's unconventional to link other questions around link this, but it's ok here. Try @Corralien's solution. –  Nov 22 '21 at 17:18