Pandas: Unable to remove column from dataframe

Question

In my dataframe, I see a colum that I used for an aggregation and I am unable to remove it. please let me know why and how to remove it please (I am trying to export the columns to CSV)

df['newid']=(df['START']-df['END'].shift()).dt.total_seconds().gt(0).cumsum()
print (df.to_string(index=False))

alarm_object              Alarm_path                                    TEXT               START                 END  newid
         5394  OLT2227-LT3-PON0-ONT03           USECASE1 - ALARM1 -NO OVERLAP 2021-01-19 18:00:00 2021-01-19 19:00:00      0
         5396  OLT2227-LT3-PON0-ONT03          USECASE1 - ALARM2 - NO OVERLAP 2021-01-19 19:10:00 2021-01-19 20:00:12      1
         5394  OLT2227-LT3-PON0-ONT05     USECASE2 - ALARM1 - Fully Contained 2021-01-19 18:00:00 2021-01-19 23:00:00      1
         5396  OLT2227-LT3-PON0-ONT05     USECASE2 - ALARM2 - Fully Contained 2021-01-19 19:00:00 2021-01-19 20:00:12      1
         5394  OLT2227-LT3-PON0-ONT10  USECASE3 - ALARM1 - START-END-RELATION 2021-01-19 22:00:00 2021-01-19 22:30:00      2
         5396  OLT2227-LT3-PON0-ONT10  USECASE3 - ALARM2 - START-END-RELATION 2021-01-19 22:30:00 2021-01-19 23:00:12      2
         5394  OLT2227-LT3-PON0-ONT21                         USECASE3-ALARM1 2021-01-19 22:00:00 2021-01-19 22:10:00      2
         5396  OLT2227-LT3-PON0-ONT21                  USECASE3-ALARM2-NO-END 2021-01-19 22:15:00 2042-01-19 20:00:12      3
         5397   OLT2227-LT3-PON0-ONT4                               USECASE-4 2021-01-19 17:30:00 2042-01-19 20:00:12      3
         5395   OLT2227-LT3-PON0-ONT4                               USECASE-4 2021-01-19 20:00:00 2021-01-19 23:00:00      3
         5397  OLT2227-LT3-PON0-ONT99                               USECASE-5 2021-01-19 17:30:00 2021-01-19 22:00:00      3
         5395  OLT2227-LT3-PON0-ONT99                               USECASE-5 2021-01-19 20:00:00 2042-01-19 20:00:12      3



newdf=df.groupby(['newid','Alarm_path']).agg({'START':'min','END':'max'}).reset_index(level=1)
print (newdf.to_string(index=False))



            Alarm_path               START                 END
 OLT2227-LT3-PON0-ONT03 2021-01-19 18:00:00 2021-01-19 19:00:00
 OLT2227-LT3-PON0-ONT03 2021-01-19 19:10:00 2021-01-19 20:00:12
 OLT2227-LT3-PON0-ONT05 2021-01-19 18:00:00 2021-01-19 23:00:00
 OLT2227-LT3-PON0-ONT10 2021-01-19 22:00:00 2021-01-19 23:00:12
 OLT2227-LT3-PON0-ONT21 2021-01-19 22:00:00 2021-01-19 22:10:00
 OLT2227-LT3-PON0-ONT21 2021-01-19 22:15:00 2042-01-19 20:00:12
  OLT2227-LT3-PON0-ONT4 2021-01-19 17:30:00 2042-01-19 20:00:12
 OLT2227-LT3-PON0-ONT99 2021-01-19 17:30:00 2042-01-19 20:00:12

newdf['Alarm_group']="ACCES_DEGRADE"
newdf['Alarm_Id']="OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_object 9999"
newdf.rename(columns={'START': 'EventTime', 'END':'ClearTime'}, inplace=True)

for col in newdf.columns: 
    print(col) 

Alarm_path
EventTime
ClearTime
Alarm_group
Alarm_Id

final_df = newdf[["Alarm_Id", "Alarm_path", "Alarm_group","EventTime","ClearTime"]]

final_df


Alarm_Id    Alarm_path  Alarm_group EventTime   ClearTime
newid                   
0   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT03  ACCES_DEGRADE   2021-01-19 18:00:00 2021-01-19 19:00:00
1   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT03  ACCES_DEGRADE   2021-01-19 19:10:00 2021-01-19 20:00:12
1   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT05  ACCES_DEGRADE   2021-01-19 18:00:00 2021-01-19 23:00:00
2   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT10  ACCES_DEGRADE   2021-01-19 22:00:00 2021-01-19 23:00:12
2   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT21  ACCES_DEGRADE   2021-01-19 22:00:00 2021-01-19 22:10:00
3   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT21  ACCES_DEGRADE   2021-01-19 22:15:00 2042-01-19 20:00:12
3   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT4   ACCES_DEGRADE   2021-01-19 17:30:00 2042-01-19 20:00:12
3   OPERATION_CONTEXT .OC.HUA_NS_DEGRADE alarm_obj...   OLT2227-LT3-PON0-ONT99  ACCES_DEGRADE   2021-01-19 17:30:00 2042-01-19 20:00:12

The first column 'newid' is a parasite and I am unable to remove it. please let me know what is it and how to remove it when I try to write the dateframe to csv.

It's not a column it's an Index named `'newid'` (See: https://stackoverflow.com/questions/55027108/pandas-rename-index/55028542#55028542). If you're writing `to_csv` you can use `index=False` as an argument — ALollz, Jan 25 '21 at 18:56

Pandas: Unable to remove column from dataframe

0 Answers0