How to remove undesired values from column

Question

I want to drop a few elements from the 'Months' column through loop. After implementation of loop, it's not presenting output.

Please refer this datafile:File:

Please refer this code:

months=pd.Series(['Dec�Jan�Feb','Mar�Apr�May','Jun�Jul�Aug','Meteorological year'])

     for i in months:
        if i in Temp_data2.Months:
           Temp_data2[Temp_data2.Months != months]
           print(Temp_data2)`

"2","Afghanistan","7012","December","7271","Temperature change","°C","0.600000","-0.721000","-0.603000","-3.636000","-1.115000","-0.327000","-0.347000","0.056000","1.985000","-0.186000","2.462000","-2.037000","-0.723000","-1.530000","-0.696000","-1.016000","1.140000","2.340000","1.053000","1.199000","1.619000","-1.850000","-0.588000","-3.167000","-0.262000","-1.094000","1.504000","2.456000","1.918000","-0.573000","1.704000","2.007000","0.660000","0.014000","-0.878000","0.940000","0.290000","2.817000","1.199000","0.967000","2.137000","-1.256000","-0.076000","0.187000","0.867000","-1.976000","-0.736000","0.842000","0.950000","-0.154000","-1.789000","-0.467000","-0.216000","0.005000","0.048000","3.429000","0.441000","1.311000","1.426000"
"2","Afghanistan","7012","December","6078","Standard Deviation","°C","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000","1.516000"
"2","Afghanistan","7016","Dec–Jan–Feb","7271","Temperature change","°C","-0.731000","1.042000","1.981000","-2.012000","-0.621000","1.970000","-0.488000","-0.671000","-1.803000","1.533000","-0.286000","-2.188000","-1.050000","-1.871000","-1.086000","0.031000","-1.395000","0.036000","1.518000","-0.187000","1.337000","0.065000","-0.289000","-1.656000","-0.088000","-0.072000","0.990000","0.941000","-0.514000","0.747000","-0.643000","0.722000","0.867000","0.112000","0.387000","-0.242000","0.803000","0.233000","2.299000","0.621000","0.357000","1.576000","1.013000","1.834000","-0.403000","1.329000","0.033000","-2.874000","1.618000","1.921000","0.449000","-1.858000","0.753000","-0.549000","1.549000","1.905000","1.436000","1.714000","1.449000"
"2","Afghanistan","7016","Dec–Jan–Feb","6078","Standard Deviation","°C","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000","1.335000"
"2","Afghanistan","7017","Mar–Apr–May","7271","Temperature change","°C","0.073000","0.454000","0.211000","0.322000","-0.972000","-0.484000","-1.187000","-0.645000","0.245000","0.796000","1.746000","-1.146000","0.452000","0.746000","-0.275000","-0.958000","1.660000","0.344000","-0.745000","1.321000","1.099000","-0.419000","-0.913000","1.205000","0.277000","-0.839000","0.245000","0.943000","-0.765000","0.161000","-0.695000","-1.930000","-0.134000","0.327000","-0.625000","-0.424000","-0.420000","0.446000","0.328000","2.429000","2.702000","1.402000","-0.151000","1.808000","0.207000","2.376000","1.418000","2.854000","0.842000","2.467000","2.034000","0.600000","1.132000","0.489000","0.985000","2.174000","1.789000","2.238000","0.593000"
"2","Afghanistan","7017","Mar–Apr–May","6078","Standard Deviation","°C","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000","0.932000"
"2","Afghanistan","7018","Jun–Jul–Aug","7271","Temperature change","°C","0.406000","-0.330000","0.535000","0.003000","-0.331000","0.352000","-0.112000","-0.040000","-0.245000","0.626000","0.645000","-1.401000","1.080000","-0.202000","0.092000","0.075000","0.754000","0.089000","-0.069000","0.381000","-0.237000","-0.309000","0.764000","1.062000","0.686000","-0.052000","-0.044000","0.651000","-0.123000","0.906000","0.228000","-0.245000","-0.074000","1.128000","0.989000","0.155000","0.739000","0.600000","0.787000","0.317000","1.036000","0.832000","0.852000","0.556000","0.575000","0.864000","0.527000","1.504000","0.123000","0.558000","1.637000","0.878000","1.312000","0.956000","0.990000","0.992000","1.002000","1.528000","0.790000"
"2","Afghanistan","7018","Jun–Jul–Aug","6078","Standard Deviation","°C","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000","0.562000"
"2","Afghanistan","7019","Sep–Oct–Nov","7271","Temperature change","°C","-0.066000","-1.645000","0.800000","-1.222000","1.123000","-1.135000","0.343000","-0.136000","-0.162000","0.476000","0.362000","0.213000","0.397000","-0.623000","-0.604000","-0.316000","1.098000","0.080000","0.633000","0.817000","-0.324000","-0.736000","0.973000","-0.121000","0.072000","0.544000","0.205000","0.954000","0.735000","1.248000","0.397000","0.173000","0.029000","0.124000","0.701000","0.087000","0.796000","1.398000","1.297000","0.619000","1.164000","1.458000","0.678000","1.070000","1.167000","2.018000","0.488000","1.200000","0.779000","1.311000","1.249000","0.788000","1.751000","1.000000","0.859000","0.821000","1.704000","0.614000","0.739000"
"2","Afghanistan","7019","Sep–Oct–Nov","6078","Standard Deviation","°C","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000","0.783000"
"2","Afghanistan","7020","Meteorological year","7271","Temperature change","°C","-0.080000","-0.120000","0.882000","-0.727000","-0.200000","0.176000","-0.361000","-0.373000","-0.491000","0.858000","0.617000","-1.131000","0.220000","-0.487000","-0.468000","-0.292000","0.529000","0.137000","0.334000","0.583000","0.469000","-0.350000","0.134000","0.123000","0.237000","-0.105000","0.349000","0.872000","-0.167000","0.766000","-0.178000","-0.320000","0.172000","0.423000","0.363000","-0.106000","0.480000","0.669000","1.178000","0.996000","1.315000","1.317000","0.598000","1.317000","0.386000","1.647000","0.616000","0.671000","0.841000","1.564000","1.342000","0.102000","1.237000","0.474000","1.096000","1.473000","1.483000","1.524000","0.893000"
"2","Afghanistan","7020","Meteorological year","6078","Standard Deviation","°C","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000","0.548000"
"3","Albania","7001","January","7271","Temperature change","°C","0.180000","1.414000","-1.783000","-2.319000","0.635000","-1.501000","-1.887000","-2.290000","-0.868000","1.797000","2.397000","1.629000","0.244000","0.743000","-0.648000","-0.057000","1.238000","0.183000","-0.528000","-1.577000","-2.543000","0.336000","0.610000","1.098000","-0.933000","0.668000","-0.167000","2.543000","-0.697000","-0.972000","-0.491000","-0.192000","-0.361000","2.058000","-0.052000","1.158000","2.025000","1.946000","1.038000","-2.435000","3.057000","-1.060000","2.040000","-0.411000","0.431000","-1.060000","2.755000","1.229000","1.152000","1.137000","0.812000","-1.004000","1.760000","3.583000","1.083000","1.530000","-2.470000","2.304000","-0.816000"
"3","Albania","7001","January","6078","Standard Deviation","°C","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000","1.472000"
"3","Albania","7002","February","7271","Temperature change","°C","-0.625000","-2.125000","-0.621000","-0.932000","-4.319000","2.845000","-1.224000","1.316000","0.453000","0.136000","-0.465000","1.442000","-0.105000","1.708000","-1.524000","-0.354000","3.226000","0.897000","1.464000","-0.321000","-1.157000","-1.817000","-2.272000","-0.285000","-3.434000","-0.603000","0.804000","0.194000","1.242000","2.317000","-1.721000","-1.113000","-2.075000","0.405000","2.240000","-0.992000","0.758000","2.260000","-1.408000","-0.042000","1.314000","3.123000","-3.763000","0.486000","-2.478000","-0.379000","2.374000","1.080000","-0.803000","0.879000","0.554000","-2.714000","1.219000","3.502000","0.378000","4.611000","2.622000","0.765000","1.563000"

Raj Singh · Answer 1 · 2021-04-18T05:40:43.317

0

You are actually not replacing the dataframe in the loop thats why its not visible in the result.

You can use :

drop after selecting dataframe based on condition to drop it.

edited Apr 18 '21 at 05:40

answered Apr 18 '21 at 05:30

Raj Singh

1
1

Hey @RajSingh , In my case also Drop function not working to drop rows, I don't know why is this happening. I am figuring out this. Instead of i chose this method to drop element from the column. Please see: Temp_data2 = Temp_data1[Temp_data1.Months != ''Dec�Jan�Feb'']. Temp_data2 – Lavish Isasare Apr 18 '21 at 06:15

Trenton McKinney · Accepted Answer · 2021-04-18T20:27:33.267

There are two issues:
1. The data is loaded with the incorrect encoding, which is why the DataFrame contains .
2. If you are using a loop to filter a DataFrame, you are doing it incorrectly.
Use Boolean indexing and pandas.DataFrame.isin to filter the DataFrame with the 'Months' column
- Also see SO: Logical operators for Boolean indexing in Pandas

import pandas as pd
import calendar

# read the file with the correct encoding
df = pd.read_csv('Environment_Temperature_change_E_All_Data_NOFLAG.csv', encoding='cp1252')

# print the unique values in the Months column before filtering
print(df.Months.unique())

[out]:
['January' 'February' 'March' 'April' 'May' 'June' 'July' 'August'
 'September' 'October' 'November' 'December' 'Dec–Jan–Feb' 'Mar–Apr–May'
 'Jun–Jul–Aug' 'Sep–Oct–Nov' 'Meteorological year']

# create a list of calendar month names
month_names = list(calendar.month_name)[1:]

# filter the dataframe with Boolean selection
filtered = df[df.Months.isin(month_names)]

# show unique values in the Months column
print(filtered.Months.unique())

[out]: 
['January' 'February' 'March' 'April' 'May' 'June' 'July' 'August'
 'September' 'October' 'November' 'December']

`df.iloc[22:35, :7]` without encoding

    Area Code         Area  Months Code               Months  Element Code             Element Unit
22          2  Afghanistan         7012             December          7271  Temperature change   �C
23          2  Afghanistan         7012             December          6078  Standard Deviation   �C
24          2  Afghanistan         7016          Dec�Jan�Feb          7271  Temperature change   �C
25          2  Afghanistan         7016          Dec�Jan�Feb          6078  Standard Deviation   �C
26          2  Afghanistan         7017          Mar�Apr�May          7271  Temperature change   �C
27          2  Afghanistan         7017          Mar�Apr�May          6078  Standard Deviation   �C
28          2  Afghanistan         7018          Jun�Jul�Aug          7271  Temperature change   �C
29          2  Afghanistan         7018          Jun�Jul�Aug          6078  Standard Deviation   �C
30          2  Afghanistan         7019          Sep�Oct�Nov          7271  Temperature change   �C
31          2  Afghanistan         7019          Sep�Oct�Nov          6078  Standard Deviation   �C
32          2  Afghanistan         7020  Meteorological year          7271  Temperature change   �C
33          2  Afghanistan         7020  Meteorological year          6078  Standard Deviation   �C
34          3      Albania         7001              January          7271  Temperature change   �C

`df.iloc[22:35, :7]` with encoding

    Area Code         Area  Months Code               Months  Element Code             Element Unit
22          2  Afghanistan         7012             December          7271  Temperature change   °C
23          2  Afghanistan         7012             December          6078  Standard Deviation   °C
24          2  Afghanistan         7016          Dec–Jan–Feb          7271  Temperature change   °C
25          2  Afghanistan         7016          Dec–Jan–Feb          6078  Standard Deviation   °C
26          2  Afghanistan         7017          Mar–Apr–May          7271  Temperature change   °C
27          2  Afghanistan         7017          Mar–Apr–May          6078  Standard Deviation   °C
28          2  Afghanistan         7018          Jun–Jul–Aug          7271  Temperature change   °C
29          2  Afghanistan         7018          Jun–Jul–Aug          6078  Standard Deviation   °C
30          2  Afghanistan         7019          Sep–Oct–Nov          7271  Temperature change   °C
31          2  Afghanistan         7019          Sep–Oct–Nov          6078  Standard Deviation   °C
32          2  Afghanistan         7020  Meteorological year          7271  Temperature change   °C
33          2  Afghanistan         7020  Meteorological year          6078  Standard Deviation   °C
34          3      Albania         7001              January          7271  Temperature change   °C

`filtered.iloc[22:35, :7]`

The rows where the values in 'Months' that didn't match a value in month_names, are no longer in filtered.

    Area Code         Area  Months Code    Months  Element Code             Element Unit
22          2  Afghanistan         7012  December          7271  Temperature change   °C
23          2  Afghanistan         7012  December          6078  Standard Deviation   °C
34          3      Albania         7001   January          7271  Temperature change   °C
35          3      Albania         7001   January          6078  Standard Deviation   °C
36          3      Albania         7002  February          7271  Temperature change   °C
37          3      Albania         7002  February          6078  Standard Deviation   °C
38          3      Albania         7003     March          7271  Temperature change   °C
39          3      Albania         7003     March          6078  Standard Deviation   °C
40          3      Albania         7004     April          7271  Temperature change   °C
41          3      Albania         7004     April          6078  Standard Deviation   °C
42          3      Albania         7005       May          7271  Temperature change   °C
43          3      Albania         7005       May          6078  Standard Deviation   °C
44          3      Albania         7006      June          7271  Temperature change   °C

@LavishIsasare you're welcome. Also note, I removed the ? because the title is not a question; it's missing an auxiliary verb (e.g. How do I ...?) — Trenton McKinney, Apr 20 '21 at 03:39
Noted. This answer teaches me, How to post questions on 'stackoverflow'? How to write this type questions? How to give an answer? I have seen your stackoverflow/LinkedIn profile, You are inspirational personality. Looking forward to learn from you. Thank you. — Lavish Isasare, Apr 20 '21 at 05:27

How to remove undesired values from column

2 Answers2

df.iloc[22:35, :7] without encoding

df.iloc[22:35, :7] with encoding

filtered.iloc[22:35, :7]

`df.iloc[22:35, :7]` without encoding

`df.iloc[22:35, :7]` with encoding

`filtered.iloc[22:35, :7]`