4

I try to scrape first name + last name of people on this web page (https://www.meleenumerique.com/scientist_comite) using the code below but it doesn't work. How can I determine what's wrong with it?

This is the code I wrote

from lxml import html  
import csv,os,json
import requests
url="https://www.meleenumerique.com/scientist_comite"
r=requests.get(url)
t=html.fromstring(r.content)

title=t.xpath('/html/head/title/text()')
#Create the list of speaker
speaker=t.xpath('//span[contains(@class,"speaker-name")]//text()')

print(title)
print("Speakers:",speaker)
halfer
  • 19,824
  • 17
  • 99
  • 186
Nico2806
  • 63
  • 1
  • 1
  • 5
  • Possible duplicate of [Web-scraping JavaScript page with Python](https://stackoverflow.com/questions/8049520/web-scraping-javascript-page-with-python) – Turtvaiz Nov 24 '18 at 23:04

1 Answers1

5

You can try with this Requests-HTML library which should let you scrape the content from that page. This library supports xpath and has the ability to take care of dynamic content.

import requests_html

session = requests_html.HTMLSession()
r = session.get('https://www.meleenumerique.com/scientist_comite')
r.html.render(sleep=5, timeout=8)
for item in r.html.xpath("//*[contains(@class,'speaker-name')]"):
    print(item.text)
SIM
  • 21,997
  • 5
  • 37
  • 109