Split sentences into words and make it into a list (Python)

Question

I have a dataframe named "df" that only have 1 column called "tweet". That dataframe consists of a bunch of sentences like this :

I have a cat
What do you mean by that?
This is my room.
Lorem ipsum dolor sit amet

I want to split all the sentences into words and put all the words into a list.

So far i tried :

def word_split() :
word = []
    for index, row in df.iterrows() :
        words = row['tweet'].split()
        word.append(words)
    return word

word_split()

But rather than a list, i got a list of lists :

[['I', 'have', 'a', 'cat'],
['What', 'do', 'you', 'mean', 'by', 'that?'],
['This', 'is' .....]]

I want it to be a list rather than a list of lists :

['I', 'have', 'a', 'cat', 'What', 'do', 'you', .....]

Any suggestions?

Thank you @jonrsharpe, it works! I've never known that until now — bramd, Jul 27 '20 at 07:28
[Don't use `iterrows`](https://stackoverflow.com/questions/24870953/does-pandas-iterrows-have-performance-issues). Assuming your column name is `word`, use `df["word"].str.split("\s")`. — Henry Yik, Jul 27 '20 at 07:42
What are the advantages of using .str.split("\s") instead of iterrows? @HenryYik — bramd, Jul 28 '20 at 02:30

score 0 · Accepted Answer · answered Jul 27 '20 at 07:33

0

Rather than

word.append(words)

it must be

word.extend(words)

Thank you @jonrsharpe for the answer.

answered Jul 27 '20 at 07:33

bramd

11
4

Split sentences into words and make it into a list (Python)

1 Answers1