0

I wrote very basic parser. 

import requests

url = "https://www.tui.ru/offices"
r = requests.get(url)
t = html.fromstring(r.content)
print(r.content)    
name = t.xpath("//h2[@class='TuiUI']/text()")

print(name)

Don't know why, but html that i recieve after request doesn't have div block with main content of this page. HTML in browser goes like this:

<body>
 <div id="app">
  <div id = "AppContainer">
    ...
  </div>
 </div>
</body>

but in r.content i have this:

<body>
  <div id="app"></div>
</body>

How can i fix it?

UPD:

I tried Request-HTML instead of requests and i recieved header, footer and other stuff, but i am still not able to see main content of the page in html from request.

url = "https://www.tui.ru/offices"
session = HTMLSession()
r = session.get(url)
r.html.render()
print(r.html.html)

Main part of HTML code that i recieve looks like this:

<div class="offices-list__container">
 <div class="offices-list__loaderWrap">
  <div data-loader="circle" class="offices-list__loader">
  </div>
  </div>
</div>

But exacly the same part in HTML in browser looks like this:

<div class="offices-list__container">
     <div class="card-list offices-list>
      #here is the main info, that i need to parse
      </div>
 </div>

Really don't know how to handle this out

PTaHHHa
  • 67
  • 1
  • 11
  • Another: [Python 3: using requests does not get the full content of a web page](https://stackoverflow.com/questions/47730671/python-3-using-requests-does-not-get-the-full-content-of-a-web-page) – wwii Mar 14 '20 at 14:23
  • @wwii Thanks for giving me links about my question. Currently i am trying to solve my problem using Requests-HTML. However, i am still not able to recieve main content of my page. HTML from Requests-HTML and HTML from browser has differences in div blocks,that have the content i need. – PTaHHHa Mar 14 '20 at 16:44

0 Answers0