create a dask dataframe from a dictionary

Question

I have a dictionary like this:

d = {'Caps': 'cap_list', 'Term': 'unique_tokens', 'LocalFreq': 'local_freq_list','CorpusFreq': 'corpus_freq_list'}

I want to create a dask dataframe from it. How do I do it? Normally, in Pandas, is can be easily imported to a Pandas df by:

df = pd.DataFrame({'Caps': cap_list, 'Term': unique_tokens, 'LocalFreq': local_freq_list,
                               'CorpusFreq': corpus_freq_list})

Should I first load into a bag and then convert from bag to ddf?

Does this answer your question? [convert dask.bag of dictionaries to dask.dataframe using dask.delayed and pandas.DataFrame](https://stackoverflow.com/questions/55298442/convert-dask-bag-of-dictionaries-to-dask-dataframe-using-dask-delayed-and-pandas) — rpanai, Dec 17 '19 at 19:05
@rpanai I did see the link, but, the role of k (sequence...?) and the whole thing seems convoluted. Wanted a simple example. Plus, I have not used 'delayed' before. I always create the task dag and run compute() finally...which has worked well for me. — user1717931, Dec 17 '19 at 19:41

score 6 · Answer 1 · answered Dec 21 '19 at 04:10

If your data fits in memory then I encourage you to use Pandas instead of Dask Dataframe.

If for some reason you still want to use Dask dataframe then I would convert things to a Pandas dataframe and then use the dask.dataframe.from_pandas function.

import dask.dataframe as dd
import pandas as pd

df = pd.DataFrame(...)
ddf = dd.from_pandas(df, npartitions=20)

But there are many cases where this will be slower than just using Pandas well.

create a dask dataframe from a dictionary

1 Answers1

Linked