3

I have a general question about the syntax used in Python. I am very new to programming so this might seem like a noob question, but I'm using pythons nltk, and in that there is a command which appears as follows...

word_tokens = word_tokenize(example_sent)

filtered_sentence = [w for w in word_tokens if not w in stop_words]

filtered_sentence = []

for w in word_tokens:
if w not in stop_words:
    filtered_sentence.append(w)

Is anyone able to explain the logic behind the "w for w in word_tokens"? I have seen this in several forms, so can anyone break down what is happening here in the "X for X in X"?

Would just like some clarity on the concept used here, thanks in advance!

peterh
  • 11,875
  • 18
  • 85
  • 108
Arod
  • 39
  • 1
  • 3
  • 1
    This is a list comprehension. You should google this as it can be a pretty confusing subject, which is hard to explain in a single answer. That's a 'representative' of the _functional_ programming paradigm. – ForceBru Mar 22 '17 at 14:44

1 Answers1

6
filtered_sentence = [w for w in word_tokens if not w in stop_words]

is equivalent to:

filtered_sentence = []
for w in word_tokens:
    if not w in stopwords:
        filtered_sentence.append(w)
L3viathan
  • 26,748
  • 2
  • 58
  • 81