0

I am trying to simply skip over a record that is found within my relevant excel file as: empty, blank, or 'N/A'. I want the script to continue or skip; not stop if a record is found empty, N/A.

I am trying the below, but the script is still stopping at a blank, or N/A found record in the excel file....

    for row in excel_data.itertuples():
        mrn = row.MRN

        if mrn in ("", " ", "N/A", None) or math.isnan(mrn):
            print(f"Invalid record: {row}")
            excel_data = excel_data.drop(excel_data.index[row.Index])
        else:
            num_valid_records += 1

    print(f"Processing #{num_valid_records} records")

More context:

I'm importing data from an excel file, to pandas, to a web page (web form).

  • I noticed the excel file has a few cells with values as 'N/A'.
  • In the pandas data frame a few values are also outputting as 'NaT'.
  • Currently as is my script is stopping when excepting a Date record from a cell, but instead this cell contains a 'N/A'. This entry is not required, so I just want it to skip if N/A is found, instead of stopping.
  • I want it to simply skip this problem cell (i.e. found blank, or within N/A') and continue with the other records.
Dr Upvote
  • 8,023
  • 24
  • 91
  • 204
  • I've added more details, the script doesn't continue to the next record with my efforts, it just stops. – Dr Upvote Mar 15 '19 at 14:16
  • does it go inside of `if` block? – Enayat Mar 15 '19 at 14:17
  • Please add a snapshot of your data as `example data`. See more information here: https://stackoverflow.com/questions/20109391/how-to-make-good-reproducible-pandas-examples – Erfan Mar 15 '19 at 14:21
  • Can you try exception handling here, if there is na value catch valueerror: – H_J Mar 15 '19 at 14:21
  • What is your expected output? Dont think your question is hard to solve. Its just hard to understand what your data looks like and what your goal is. – Erfan Mar 15 '19 at 14:39
  • I'm importing data from an excel file, to pandas, to a web page (web form). I notice the data excel file has a few values as 'N/A' - and in the pandas data frame a few values are outputting as 'NaT' - currently as is my script is failing when excepting a Date record, but is finding a 'N/A' in that cell in the Excel file - I want it to simply skip this problem cell and continue going. – Dr Upvote Mar 15 '19 at 14:41

2 Answers2

1

Find a true positive and use that record as a test. Alternatively, they may be evaluating to nan in your pandas dataframe, therefore a good ole .dropna() may do the trick for you.

Boris the Barb
  • 51
  • 8
1

I tested your code, and the code does not stop in Blank or NaN or values. There should be something wrong in your data. Just print out the row that stops your code to see what's the value of that column is. Alternately, you can check for null values with python functions like: is None or == None or something like that.

Enayat
  • 3,904
  • 1
  • 33
  • 47
  • In the pandas dataframe, a few of the records are outputting 'NaT' – Dr Upvote Mar 15 '19 at 14:38
  • `NaT` looks weird. Please check the actual values of data and user `UTF8` format. Maybe there are some issues with the encoding of your data – Enayat Mar 15 '19 at 14:45