List comprehension Pandas

Question

I am trying to label a row if they are exist in a dataframe. Here is a snippet of my code:

MissingAATracking = []
    for row in UK.UK:
        if row in links_adobe_uk:
            MissingAATracking.append("NO")
        else:
            MissingAATracking.append("YES")

I get the following error:

> --------------------------------------------------------------------------- TypeError                                 Traceback (most recent call
> last) <ipython-input-94-5b85f0a628ec> in <module>()
>       1 for row in UK.UK:
> ----> 2     if row in links_adobe_uk:
>       3         MissingAATracking.append("NO")
>       4     else:
>       5         MissingAATracking.append("YES")
> 
> /anaconda/lib/python3.6/site-packages/pandas/core/generic.py in
> __contains__(self, key)
>     905     def __contains__(self, key):
>     906         """True if the key is in the info axis"""
> --> 907         return key in self._info_axis
>     908 
>     909     @property
> 
> /anaconda/lib/python3.6/site-packages/pandas/core/indexes/base.py in
> __contains__(self, key)    1588     @Appender(_index_shared_docs['__contains__'] % _index_doc_kwargs)   
> 1589     def __contains__(self, key):
> -> 1590         hash(key)    1591         try:    1592             return key in self._engine
> 
> TypeError: unhashable type: 'list'

I have looked here: Python, TypeError: unhashable type: 'list'

UPDATE

Here is what the head of my UK dataframe looks like:

         UK
0       Link1
1       Link1
2       Link1
3       Link1
4       Link1

and my links_adobe_uk:

        PageURL
0       (null)
1       Link1
2       Link1
3       Link1
4       Link1

I have also tried to drop duplicates from both columns but my UK dataframe throws a

TypeError: unhashable type: 'list'

but I cant seem to follow as Both UK and links_adobe_uk are both dataframes. I will appreciate guidance on this

you are trying to search a row from `UK` dataframe in `links_adobe_uk`,that is not possible. What you should do is come up with some unique key per row, or use index to check if row is present in the other dataframe or not. — Vikash Singh, Jun 29 '17 at 13:57
If you show a few lines of your UK and links_adobe_uk dataframe, you will get a more efficient way of doing what you want in no time. — Allen Qin, Jun 29 '17 at 13:58
Also share sample few rows of your both dataframes if possible. We will be able to help better. — Vikash Singh, Jun 29 '17 at 13:58
I have just tried dropping duplicated on the UK data frame but got the TypeError: unhashable type: 'list' error again — 0xsegfault, Jun 29 '17 at 14:00

Vikash Singh · Accepted Answer · 2017-06-29T14:24:44.110

import pandas as pd

df1 = pd.DataFrame(data = {'col1' : [1, 2, 3, 4, 5]}) 
df2 = pd.DataFrame(data = {'col1' : [1, 2, 3]})

# merge both dataframes on the common column
common = df1.merge(df2,on=['col1'])

# wherever the common column is present in df1 return "YES" else "NO"
missing_tracking = ["Yes" if val else "No" for val in (df1.col1.isin(common.col1)).values ]

output:

['Yes', 'Yes', 'Yes', 'No', 'No']

Credits: pandas get rows which are NOT in other dataframe

score 1 · Answer 2 · answered Jun 29 '17 at 14:40

As the dataframe 'links_adobe_uk' contains only one column, it is treated as a series. Thats the reason why it is showing the error, "Unhashable type:list". So converting the series into list will resolve the issue

list_to_search = links_adobe_UK.PageURL.values.tolist()
MissingAATracking = []
for row in UK.UK:
    if row in list_to_search:
        MissingAATracking.append('No')
    else:
        MissingAATracking.append('Yes')
print(MissingAATracking)

score 0 · Answer 3 · answered Jun 29 '17 at 14:04

The goal of this program can be divided into two parts:

iterate the rows in a data.frame
check whether each of the rows in another data.frame

I believe that it's not that straight forward to do with for row in UK.UK:.

Please check with these two answers with respect to the two parts above.
How to iterate over rows in a DataFrame in Pandas?
Pandas: Check if row exists with certain values

List comprehension Pandas

3 Answers3