1

I want to extract the title of an article using BeautifulSoup 4.

Here's my scrape function:

def scrape(request):
    session = requests.Session()
    session.headers = {"User-Agent": "Googlebot/2.1 (+http://www.google.com/bot.html)"}
    url = "https://www.ynet.co.il/home/0,7340,L-2,00.html"

    content = session.get(url, verify=False).content
    soup = BSoup(content, "html.parser")
    News = soup.find_all('div', {"class":"str3s str3s_small str3s_type_small"})

    for artcile in News: 
        main = artcile.find_all('a')[0]
        link = main['href']
        image_src = str(main.find('img')['src']).split(" ")[0]
        title = main['title']
        new_headline = Headline()
        new_headline.title = title
        new_headline.url = link
        new_headline.image = image_src
        new_headline.save()
    return redirect("../")

As you can see my title is set to main['title'], but the problem is that this ['title'] searches for title elements and what I'm looking for is, for example, a <div class title>title</div>.

How do I do that?

Jongware
  • 22,200
  • 8
  • 54
  • 100
Dat Guy
  • 51
  • 2
  • 6

0 Answers0