0

I have a dataframe where I want the value to be NaN if the index and column name match. I have tried using np.fill_diagonal() since that is essentially what I want, but it changes the format of the values. I want to keep the scientific notation format of the values if possible. What is the best way to do so?

    granger_df = None
    
    
    
    data_diff = df.diff()
    data_diff.dropna(inplace=True)
    var_model = VAR(data_diff)
    res = var_model.select_order(maxlags=p)

    results = var_model.fit(maxlags=p, ic='aic')
    
    results_df = pd.DataFrame(results, index = df.columns, columns = df.columns)
    
    
    my_list = []
    for col1 in results_df:
        for col2 in results_df:
            element = results.test_causality(col1, col2, kind='f')
            my_list.append(element.pvalue)
            
    my_array = np.array(my_list)
    
    my_array = my_array.reshape(5,5)
    
    granger_df = pd.DataFrame(my_array, index=df.columns, columns=df.columns)

The output I currently have:

enter image description here

The output when I use

np.fill_diagonal(granger_df.values, np.nan)

It achieves my goal, but changes the values. I need them to stay the same:

enter image description here

Blueboots
  • 49
  • 1
  • 9
  • 1
    What do you mean "it changes the values"? The two dataframes appear identical, it's just that your first screenshot shows floats represented in scientific notation. You say you want the value to be NaN on the diagonal but then complain about values being changed. Can you be more specific? – ddejohn Oct 15 '21 at 01:54
  • It seems since you've removed the smallest value from per column from the diagonal that scientific notation is no longer needed for the column to display. Is that what you mean by "changes the values"? – Henry Ecker Oct 15 '21 at 02:06
  • @HenryEcker yes, that is what I meant, I want to keep the scientific notation for each column of the dataframe. I apologize for the confusion, I will edit my question. – Blueboots Oct 15 '21 at 02:10
  • 1
    Looks like `pd.set_option('display.float_format', '{:.7g}'.format)` [via this answer](https://stackoverflow.com/a/31983844/15497888). This could also be an [option context](https://pandas.pydata.org/docs/reference/api/pandas.option_context.html) if you wanted limited scope `with pd.option_context('display.float_format', '{:.7g}'.format):` – Henry Ecker Oct 15 '21 at 02:14

0 Answers0