Cannot compare types 'ndarray(dtype=int64)' and 'int64'

Question

I have a column in my dataframe with barcodes and created a dictionary to map barcodes to item ids.

I am creating a new column:

df['item_id'] = df['bar_code']

A dictionary (out of a second dataframe - imdb -)

keys = (int(i) for i in imdb['bar_code'])
values = (int(i) for i in imdb['item_id'])
map_barcode = dict(zip(keys, values))

map_barcode (first 5 e.g.)

{0: 1000159, 9000000017515: 11, 7792690324216: 16, 7792690324209: 20, 70942503334: 33}

And then mapping the item id with the dict

df = df.replace({'item_id':map_barcode})

Here I am hoping to obtain the item ids in the column

(Going back to the dict examples:)

df['item_id'][0] = 1000159
df['item_id'][1] = 11
df['item_id'][2] = 16
df['item_id'][3] = 20
df['item_id'][4] = 33

But end up getting this error:

Cannot compare types 'ndarray(dtype=int64)' and 'int64'

I tried to change the type of the dictionary to np.int64

keys = (np.int64(i) for i in imdb['bar_code'])
values = (np.int64(i) for i in imdb['item_id'])
map_barcode = dict(zip(keys, values))

But got the same error.

Is there anything I am missing here?

what are the contents of dict? can you include the first five? if you don't provide data people are less likely to help you — Yuca, Sep 07 '18 at 15:40
Umm... does `df.item_id.map(pd.Series(imdb['keys'], imdb['item_id']))` do what you're after? — Jon Clements, Sep 07 '18 at 15:43
@Yuca I updated it the post, please let me know if it's clearer now. I'll try the pd.Series.map, thanks! — Franco D'Angelo, Sep 07 '18 at 15:53
@Yuca you were right, it worked with `pd.Series.map` thanks! — Franco D'Angelo, Sep 07 '18 at 16:00

jpp · Accepted Answer · 2018-09-07T16:49:29.493

`replace` example

Firstly, I cannot reproduce your error. This works fine:

map_dict = {0: 1000159, 9000000017515: 11, 7792690324216: 16, 7792690324209: 20, 70942503334: 33}

df = pd.DataFrame({'item_id': [0, 7792690324216, 70942503334, 9000000017515, -1, 7792690324209]})

df = df.replace({'item_id': map_dict})

Result:

Use `map` + `fillna` instead

Secondly, manually iterating Pandas series within generator expressions is relatively expensive. In addition, replace is inefficient when mapping via a dictionary.

In fact, creating a dictionary is not even necessary. There are optimized series-based methods for these tasks:

map_series = imdb[['bar_code', 'item_id']].astype(int).set_index('bar_code')['item_id']

df['item_id'] = df['item_id'].map(map_series).fillna(df['item_id'])

Cannot compare types 'ndarray(dtype=int64)' and 'int64'

1 Answers1

replace example

Use map + fillna instead

`replace` example

Use `map` + `fillna` instead