Looking for a nicer python convention for this code block

Question

When I'm building out dataframes inside a loop, I often find myself using this convention:

complete_df = None
for data_chunk in data_chunks:
    partial_df = get_partial_df(data_chunk)    
    partial_df = do_some_stuff_to_my_df(partial_df)
    if complete_df is None:
        complete_df = partial_df
    else:
        complete_df = complete_df.append(partial_df)

I'm looking for a better / shorter / more pythonic way to do this. A ternary statement seems like it wouldn't be an improvement.

Knowing what `get_partial_df` is (or at least what it returns) would be helpful. — cs95, May 21 '19 at 04:54
get_partial_df is going to return a pandas dataframe. This is just pseudocoded. — red, May 21 '19 at 05:06

score 1 · Accepted Answer · answered May 21 '19 at 06:03

You can do away with the if else block if you initialize the complete_df to an empty DataFrame like this:

import pandas as pd

complete_df = pd.DataFrame()
for data_chunk in data_chunks:
    partial_df = get_partial_df(data_chunk)    
    partial_df = do_some_stuff_to_my_df(partial_df)
    complete_df = complete_df.append(partial_df)

score 0 · Answer 2 · edited May 21 '19 at 05:31

0

try this

complete_df = None
for data_chunk in data_chunks:
    partial_df = get_partial_df(data_chunk)    
    complete_df = partial_df if complete_df is None else complete_df.append(partial_df)

edited May 21 '19 at 05:31

Jitesh Prajapati

2,533
4
29
51

answered May 21 '19 at 04:58

Radeep Yenni

1
1

score 0 · Answer 3 · answered May 21 '19 at 05:07

0

data_chunks = range(1, 100, 4)

def get_partial_df(num):
    return num

#complete_df = None
complete_df = list()
print(type(complete_df))
for data_chunk in data_chunks:
    partial_df = get_partial_df(data_chunk)
    complete_df.append(partial_df)

    # if complete_df is None:
    #     complete_df = partial_df  ##here complete_df is int
    # else:
    #    complete_df = complete_df.append(partial_df) ## appending to be done on list/tuple.

answered May 21 '19 at 05:07

Kaushik

95
1
8

defined get_partial_df and data_chunks.. just to make points clear – Kaushik May 21 '19 at 05:08
1

Empty lists should be defined by `[]`, not really `list()`. – cs95 May 21 '19 at 05:15
1

@cs95: this is opinion-based suggestion, `list()` is a valid syntax for creating empty lists – Azat Ibrakov May 21 '19 at 05:30
@AzatIbrakov https://stackoverflow.com/questions/30216000/why-is-faster-than-list – U13-Forward May 21 '19 at 05:31
@U9-Forward: if something is "faster" it doesn't mean that it's better, `list()` is explicit and readable enough, also creating single empty `list` should never be a bottleneck – Azat Ibrakov May 21 '19 at 05:32
@AzatIbrakov https://stackoverflow.com/questions/5790860/and-vs-list-and-dict-which-is-better/39078199#39078199 – U13-Forward May 21 '19 at 05:35
@AzatIbrakov And in general, `[]` is used 99% of the time, and `list()` is only used 1% the time, basically you really almost never see `list()` – U13-Forward May 21 '19 at 05:36

Looking for a nicer python convention for this code block

3 Answers3