How to get 50 most common words in lyrics songs (Python)

Question

I am new in python and I am trying to return the most 50 commons word's in a lyrics of songs and I have a problem that I don't really understand why it's happens.

the name "lyrics" in the code is a string of the song lyrics from a text file. every iteration of the loop is different string of lyrics that I need to include in the total of how much word are shows up in the songs.

if someone know where the problem is and can help it would be very nice.

my output is not with words is in characters: "[(' ', 46), ('o', 24), ('e', 23), ('n', 15), ('t', 15), ('h', 12), ('a', 12), ('w', 8), ('r', 8), ('s', 8), ('\n', 7), ('f', 7), ('d', 6), ('u', 5), ('y', 5), ('m', 5), ('I', 4)..." and i need to get something like: ("the", 555), ("you", 365)... without include white spaces and \n

    count = {}
    for songs in the_dict.values():
        songs = songs[0]
        for lyrics in songs.values():
            lyrics = lyrics[2]
            count = Counter(lyrics)
    return count.most_common(50)

Please include a sample value for `the_dict` and the desired output. — Samwise, May 09 '21 at 16:38
Can you share the lyrics of the song? How have you obtained and stored the lyrics in a variable? like did you extract from a file? — Muhd Mairaj, May 09 '21 at 16:41

Eno Gerguri · Answer 1 · 2021-05-09T16:40:09.570

3

Call the split() method on the lyrics before counting:

split_lyrics = lyrics[2].split()
count = Counter(split_lyrics)

see https://www.geeksforgeeks.org/find-k-frequent-words-data-set-python/

edited May 09 '21 at 16:40

answered May 09 '21 at 16:37

Eno Gerguri

639
5
22

1

You beat me to it :D – fynsta May 09 '21 at 16:38
I would upvote you, but I don't have the Karma yet... – fynsta May 09 '21 at 16:41
Does this get rid of the new line characters as well? as @roee asked in his question? – Muhd Mairaj May 09 '21 at 16:44
@MuhdMairaj Yes it does. – Eno Gerguri May 09 '21 at 16:45
yea, that code know give me common word wthout the \n but it's give me only from the last iteration. and I want to include also from all the songs together because I have a few number of lyrics – roee May 09 '21 at 16:46
@EnoGerguri ok Great, so the .split() without any arguments splits at every whitespace character? – Muhd Mairaj May 09 '21 at 16:47
@MuhdMairaj Yes, into a list, which is then counted – Eno Gerguri May 09 '21 at 16:48
@roee add all of the lyrics into a single string and then use the exact same method in the answer. It's still correct and answers your initial question. – Eno Gerguri May 09 '21 at 16:50
1

@EnoGerguri ok good to know, i thought iit only split at `" "`, probably could've/should've referred the documentation. Thanks though – Muhd Mairaj May 09 '21 at 16:50
I made a big string with all the lyrics together. – roee May 09 '21 at 17:19
and I get that the word "the" is showes only 489 times and the answer is 555 – roee May 09 '21 at 17:19
all_lyrics = re.sub('[\W_]+', ' ', all_lyrics) all_lyrics = all_lyrics.split(" ") count = Counter(all_lyrics) print(count.most_common(50)) – roee May 09 '21 at 17:20
@MuhdMairaj you know why? – roee May 09 '21 at 17:21
im assuming the_dict.values() contains all the lyrics? So do `all_lyrics = " ".join(list(dict.values()))` then `split_all_lyrics = all_lyrics.split()` – Muhd Mairaj May 09 '21 at 17:55

score 2 · Answer 2 · answered May 09 '21 at 16:38

2

You should split your lyrics at every whitespace and newline, so that you get an array of words (instead of parsing in the string immediately like you do now).

So you should use

lyrics = lyrics[2].split()

answered May 09 '21 at 16:38

fynsta

348
3
10

yea, that code know give me common word wthout the \n but it's give me only from the last iteration. and I want to include also from all the songs together because I have a few number of lyrics – roee May 09 '21 at 16:46
So this is basically two questions? – fynsta May 09 '21 at 16:48
You can just add Counters together after each iteration, then you have the words from every iteration combined https://stackoverflow.com/questions/19356055/summing-the-contents-of-two-collections-counter-objects – fynsta May 09 '21 at 16:49

How to get 50 most common words in lyrics songs (Python)

2 Answers2