How to get rid of warning "DeprecationWarning generator 'ngrams' raised StopIteration"

Question

While working on a Kaggle notebook I ran into an issue. The following code block:

from nltk import ngrams
def grams(tokens):
    return list(ngrams(tokens, 3))
negative_grams = preprocessed_negative_tweets.apply(grams)

resulted in a red box appearing saying

/opt/conda/bin/ipython:5: DeprecationWarning: generator 'ngrams' raised StopIteration

The variable preprocessed_negative_tweets is a Pandas data frame containing tokens.

Anyone know how to make this go away?

(Full notebook available here)

The cause of the warning is the change in behavior described in [PEP 479](https://www.python.org/dev/peps/pep-0479/). How to best make it go away I can't say exactly, but you may need to just filter and [ignore](http://stackoverflow.com/q/879173/699305) the `DeprecationWarning`... — alexis, Apr 25 '17 at 20:54
Quick question. Which nltk version are you using? Also which Python version are you using? — alvas, Apr 26 '17 at 23:00

score 21 · Accepted Answer · edited Oct 05 '17 at 22:32

21

To anyone else who doesn't want or can't suppress the warning.

This is happening because ngrams is raising StopIteration exception to end a generator, and this is deprecated from Python 3.5.

You could get rid of the warning by changing the code where the generator stops, so instead of raising StopIteration you just use Python's keyword return.

More on: PEP 479

edited Oct 05 '17 at 22:32

CDspace

2,639
18
30
36

answered Oct 05 '17 at 22:18

carla08

226
3
3

3

I have a generator: `while True: yield itertools.chain(...)`. How would you convert that? – OrangeDog Oct 04 '18 at 13:21
4

@OrangeDog You can use this: `yield from itertools.chain(...)`. – Björn Pollex Dec 11 '18 at 09:08

score 4 · Answer 2 · answered Nov 19 '18 at 11:42

4

You can use a wrapper like this one:

def get_data(gen):
    try:
        for elem in gen:
            yield elem
    except (RuntimeError, StopIteration):
        return

and then (according to your example):

data = get_data(ngrams(tokens, 3))

should do the trick

answered Nov 19 '18 at 11:42

Most Wanted

6,254
5
53
70

or just `yield from gen` – Tushar Sadhwani Dec 31 '20 at 11:34

score 0 · Answer 3 · answered Aug 23 '18 at 00:41

0

If you just want to quiet all warnings, you can do:

import warnings

warnings.filterwarnings('ignore')

We should listen to these warnings...

answered Aug 23 '18 at 00:41

duhaime

25,611
17
169
224

1

Nobody do this, you stand to miss all sorts of important and relevant warnings. They wouldn't exist unless the core developers made some sort of mistake. Just fix your code. – KeatsKelleher Dec 27 '20 at 16:41
Sometimes warnings are triggered by third party libraries and one can't simply change that source code. I agree one should be cautious when filtering warnings--as always in programming, with great power comes great responsibility. You should only filter warnings if you know what you're doing... – duhaime Dec 29 '20 at 22:39

How to get rid of warning "DeprecationWarning generator 'ngrams' raised StopIteration"

3 Answers3