4

How do you add data to a Vaex DataFrame?

I can see there is add_column(), but no add/append_row()

I'm looking to use Vaex instead of Pandas.

Amit Amola
  • 2,301
  • 2
  • 22
  • 37
sten
  • 7,028
  • 9
  • 41
  • 63

2 Answers2

2

I believe these is a concat method for this.

df = df_1.concat(df_2)

It's mentioned in the API Documentation as well.

Amit Amola
  • 2,301
  • 2
  • 22
  • 37
  • thanks ... so you are saying i have to create a dataframe every time i need to add row... isnt this too expensive an operation ? – sten Dec 16 '20 at 18:59
  • So that's how I did it when I wanted to do the same actually. Sadly never found a row append thing. It does sound operation expensive but umm... that's all I know. I'd actually love to know if you found something on the same lines. – Amit Amola Dec 17 '20 at 06:02
  • the more i read vaex&dask they seem like readonly fast whole-set processing of large dataset! – sten Dec 17 '20 at 14:55
  • 1
    the other idea i had was to create 10-20% large than required df/array and then convert it and save it and later reopen this new empty dataset. Then edit/update the data incrementally, BUT i couldnt find a way to EDIT/UPDATE specific cells or rows either. Vaex seem like readonly + computed cols large dataset lib – sten Dec 17 '20 at 14:59
  • 1
    another option is if we can trick vaex to read from "lazy' .csv stream-file !? i.e. you add the data at the end of the stream. but how do you make receive lazy stream and reinterpret it !! – sten Dec 17 '20 at 15:11
1

This does not answer your question, but wanted to correct concat A To concatenate two dataframes, the API is similar to pandas.

to assign it to the same dataframe variable

df1 = vaex.concat([df1, df_new])

or

to assign it to a new dataframe variable

df3 = vaex.concat([df1, df2])
MacJava
  • 11
  • 2