Create new column in pandas df and assign string values conditionally

Question

I'm new to pandas, trying to create a new column in Pandas Dataframe, and assign a string value based on a function, but the outcome outputs only 1 value ('residential) to all 5,000 columns. Any idea what's wrong with my code? Thank you

def programType(c):
if c['Primary Property Type - Self Selected'] == 'Multifamily Housing' or 'Residence Hall/Dormitory':
    return 'Residential'

elif c['Primary Property Type - Self Selected'] == 'Bank Branch' or 'Hotel' or 'Financial Office' \
or 'Retail Store' or 'Distribution Center' or 'Non-Refrigerated Warehouse' or 'Fitness Center/Health Club/Gym' \
or 'Mixed Use Property' or 'Self-Storage Facility' or 'Wholesale Club/Supercenter' or 'Supermarket/Grocery Store':
    return 'Commercial'  

elif c['Primary Property Type - Self Selected'] == 'Senior Care Community' or 'K-12 School' or 'College/University' \
or 'Worship Facility' or 'Medical Office' or 'Hospital (General Medical & Surgical)':
    return 'Institutional'

elif c['Primary Property Type - Self Selected'] == 'Manufacturing/Industrial Plant': 
    return 'Industrial'

else:
    return 'Other'

The new column is called 'Program Type'

datav3['Program Type'] = datav3.apply(programType, axis=1)

I am a bit curious - do you need loops (apply) for some reason? — jezrael, Nov 29 '18 at 06:59
Nope, if there's a better syntax i'm happy to use it. I also just went through your answer, and understood it now. Thank you. — abigfatcat, Dec 01 '18 at 03:45

Mayank Porwal · Answer 1 · 2018-11-29T06:52:15.690

The issue is with your if loops. The way you are comparing after or is not correct.

Writing or 'Residence Hall/Dormitory' will always be true, hence, only the first if gets evaluated everytime and you get Residential in all rows.

Instead of this:

if c['Primary Property Type - Self Selected'] == 'Multifamily Housing' or 'Residence Hall/Dormitory':

Do this:

if c['Primary Property Type - Self Selected'] == 'Multifamily Housing' or c['Primary Property Type - Self Selected'] == 'Residence Hall/Dormitory':

OR

if any([c['Primary Property Type - Self Selected'] == 'Multifamily Housing', c['Primary Property Type - Self Selected'] == 'Residence Hall/Dormitory']):

Just make the above change, and your code should do what is expected. Hope this is clear.

jezrael · Accepted Answer · 2018-11-29T06:53:12.793

In pandas is best avoid loops (apply are loops under the hood) if exist vectorized solutions, because loops are slow.

I try rewrite your code - create dictionary with output and list of values, swap keys with values and call map, last for not matched values add fillna:

d = {'Residential' :['Multifamily Housing', 'Residence Hall/Dormitory'],
     'Commercial' : ['Bank Branch', 'Hotel' , 'Financial Office' , 'Retail Store', 'Distribution Center', 
                   'Non-Refrigerated Warehouse', 'Fitness Center/Health Club/Gym', 'Mixed Use Property',
                   'Self-Storage Facility', 'Wholesale Club/Supercenter', 'Supermarket/Grocery Store'],
     'Institutional':['Senior Care Community', 'K-12 School', 'College/University', 'Worship Facility',
                     'Medical Office', 'Hospital (General Medical & Surgical)'],
     'Industrial':  ['Manufacturing/Industrial Plant'] }

d1 = {k: oldk for oldk, oldv in d.items() for k in oldv}
print (d1)

{
    'Multifamily Housing': 'Residential',
    'Residence Hall/Dormitory': 'Residential',
    'Bank Branch': 'Commercial',
    'Hotel': 'Commercial',
    'Financial Office': 'Commercial',
    'Retail Store': 'Commercial',
    'Distribution Center': 'Commercial',
    'Non-Refrigerated Warehouse': 'Commercial',
    'Fitness Center/Health Club/Gym': 'Commercial',
    'Mixed Use Property': 'Commercial',
    'Self-Storage Facility': 'Commercial',
    'Wholesale Club/Supercenter': 'Commercial',
    'Supermarket/Grocery Store': 'Commercial',
    'Senior Care Community': 'Institutional',
    'K-12 School': 'Institutional',
    'College/University': 'Institutional',
    'Worship Facility': 'Institutional',
    'Medical Office': 'Institutional',
    'Hospital (General Medical & Surgical)': 'Institutional',
    'Manufacturing/Industrial Plant': 'Industrial'
}

datav3 = pd.DataFrame({'Program':['Medical Office','Hotel',
                                       'Residence Hall/Dormitory',
                                       'Manufacturing/Industrial Plant','House']})
datav3['Program Type'] = datav3['Program'].map(d1).fillna('Other')
print (datav3)
                          Program   Program Type
0                  Medical Office  Institutional
1                           Hotel     Commercial
2        Residence Hall/Dormitory    Residential
3  Manufacturing/Industrial Plant     Industrial
4                           House          Other

Could you please explain a little on this bit if you don't mind, i'm new to this syntax: d1 = {k: oldk for oldk, oldv in d.items() for k in oldv} Thanks! — abigfatcat, Dec 01 '18 at 06:08
@SchratchieMe You can check [this](https://stackoverflow.com/a/31674731) — jezrael, Dec 01 '18 at 09:47
@SchratchieMe and [this](https://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/) — jezrael, Dec 01 '18 at 09:49

Create new column in pandas df and assign string values conditionally

2 Answers2