How to get text within the `p` tag using Beautiful Soup?

Question

I wanna get all the p tags and store it in a list, but unfortunately all of them have a
between.

This is how the content looks like:

<p>Ich halt mir die Pistole an den Kopf
 <br/>Doch drück' nicht ab, denn ich hab zu viel Angst vor Gott</p>,
 <p>Feinde wurden zu Brüdern
 <br/>Und Brüder wurden zu V-Männern
 <br/>Die beste Gang, in der ich jemals war
 <br/>Me, myself und meine DNA</p>,

and i should look like:

[Ich halt mir die Pistole an den Kopf
    Doch drück' nicht ab, denn ich hab zu viel Angst vor Gott, Feinde wurden zu Brüdern
     Und Brüder wurden zu V-Männern
     Die beste Gang, in der ich jemals war
     Me, myself und meine DNA,]

Thats my current code:

url = requests.get("https://www.myzitate.de/suche/farid-bang/")
z = bs(url.content)
cont = z.find("div", attrs={"id":"cont"})
cont.find_all("p")

Does this answer your question? [How to find all text inside
elements in an HTML page using BeautifulSoup](https://stackoverflow.com/questions/10113702/how-to-find-all-text-inside-p-elements-in-an-html-page-using-beautifulsoup) — mpx, Jul 18 '20 at 14:24
xKuramaa, as webmaster of myzitate.de I'd find it interesting why you're trying to scrape content from my page? — steveniclas, Dec 17 '20 at 09:50

score 1 · Answer 1 · answered Jul 18 '20 at 14:13

You can use tag.get_text() with parameters strip=True and separator='\n' to get correct text:

import requests
from bs4 import BeautifulSoup


url = 'https://www.myzitate.de/suche/farid-bang/'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

for p in soup.select('p'):
    print(p.get_text(strip=True, separator='\n'))
    print('-' * 80)

Prints:

Ich halt mir die Pistole an den Kopf
Doch drück' nicht ab, denn ich hab zu viel Angst vor Gott
--------------------------------------------------------------------------------
Feinde wurden zu Brüdern
Und Brüder wurden zu V-Männern
Die beste Gang, in der ich jemals war
Me, myself und meine DNA
--------------------------------------------------------------------------------
Wir sind Public Enemy
Ihr Police Academy
Kann nicht tanzen, meine Schultern sind zu breit
Du willst mich batteln und frisst dann den Bürgersteig
--------------------------------------------------------------------------------
Wir kam'n von unten mit Gangsta-Rap
Und sind im Endeffekt drei Gs wie das Handynetz
--------------------------------------------------------------------------------

...and so on.

score 0 · Accepted Answer · answered Jul 18 '20 at 14:01

0

Is this what you're looking for?

ps = [p.get_text() for p in cont.find_all("p")]

answered Jul 18 '20 at 14:01

Balaji Ambresh

4,977
2
5
17

score 0 · Answer 3 · answered Jul 18 '20 at 14:18

All the items are located under the id of z which is accessible via

page_soup.find_all('div', {'id': 'z'}):

In some website, the are multiple instances where the p tag exist, so I usually will search for upper class just to make sure I get the intended output.

The text you interested are within the p tag and retrievable via

div_tag.find_all('p')

The full code is as below:

append_text=[]
url = requests.get("https://www.myzitate.de/suche/farid-bang/")
page_soup= Soup(url.content)
for div_tag in page_soup.find_all('div', {'id': 'z'}):
    for litag in div_tag.find_all('p'):
        append_text.append(litag.text)

How to get text within the `p` tag using Beautiful Soup?

3 Answers3