0

I am doing the below, currently; successfully dropping the entire row, with my if in - but it turns out, i don't need to drop the entire row.. How can I handle cells specifically.

How could I keep the same logic but apply to the CELL... convert the N/A, NaT, and NaN cells to blank?

for row in excel_data.itertuples():
    ids = row.IDS
    total_records += 1

    if ids in ("", " ", "N/A", "NaT", "NaN", None) or math.isnan(ids):
        # print(f"Invalid record: {row}")
        num_invalid_records += 1
        # total_invalid = num_invalid_records + dup_count
        excel_data = excel_data.drop(excel_data.index[row.Index])
        # continue
    else:
        num_valid_records += 1
        continue

My data looks like this: (below) As you can see there are N/As that seem to translate as 'NaTs' or Nan's in Pandas - I would just like to convert these to blank or " " or even skip somehow if possible....

enter image description here

In pandas the dataframe will output like this (i omitted some sensitive entries, but you get the idea).

External_Referral='qweryt', Transitions_Planning='NF', Date_ICP_ICT_Signed_by_Member__Caregiver=datetime.date(2019, 1, 10), Date_Refused_Final_Signature_of_ICP=datetime.date(2019, 1, 10), Refused_Final_Signature_Comments='concern', Intervention_Outcome='Successful Contact', Monthly_Member_Contact_Y_N='Yes', Monthly_Member_Contact_Date=datetime.date(2019, 1, 10), Special_Projects_HHB_program='Baby Box Given', ICT_Meeting_Dates_Additional_Comments=NaT, _20='Yes', Date=datetime.date(2019, 2, 12), Targeted_Case_Management_Referral='Yes')
Dr Upvote
  • 8,023
  • 24
  • 91
  • 204
  • 1
    What does your data look like? Can you add some example data (5-10 rows is sufficient) and add expected output as well? Heres more information on that: https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples – Erfan Apr 04 '19 at 20:32
  • 1
    How is this question related to _Selenium_? Am I missing something? – undetected Selenium Apr 05 '19 at 06:01
  • @Erfan I have added it. As you can see there are N/As that seem to translate as 'NaTs' or Nan's in Pandas - I would just like to convert these to blank..... – Dr Upvote Apr 05 '19 at 13:40
  • 1
    @PeterGibbons did you read the link I gave you? Its 2 min work to make a reproducable dataframe and its for your own benefit, since we can answee your question better. I have the solution. Just waiting for you to provide a dataframe which I can answer your question with – Erfan Apr 05 '19 at 14:32
  • Thanks @Erfan, i just added my dataframe. That helpful? – Dr Upvote Apr 05 '19 at 15:41

1 Answers1

0

I assume that excel_data is a Pandas Dataframe.

If so, you can use the Pandas function fillna() on your IDS column.

tYAt1YBWFY
  • 304
  • 2
  • 8