-2

While parsing the Bitmex site, I noticed that the results of parsing using bs4 were slightly different from what I actually saw in my web browser. When I used the Devtools and continually refresh, I realized it took some time that the window we were looking at was opened, and the BS4 immediately took html and seemed to differ between the two results.

The site URL is 'https://www.bitmex.com/app/trade/XBTUSD'

You can see the difference just checking the tag <title>...</title>.

When I use bs4, they give me the result

<title>BitMEX | Bitcoin Mercantile Exchange</title>

But when I see them in Chrome, they give me the result 

<title>▼ 6486.0 (XBTUSD) 거래 - BitMEX</title>

like this.

How can I bring to Python the same code that I saw at the Chorme?

Of course, Using selenium, I can bring them to Python....but I wonder there is no way to bring them using bs4.

My programming knowledge and its ability is not good, Question can seem to be odd. If you can't understand my question I'd appreciate it if you could ask me again.

Thank you for reading it.

Kun C
  • 3
  • 2

1 Answers1

0

Again, there is some javascript code that changes the title.

}, function(e) {
    var t = e || "";
    t !== document.title && (document.title = t)
})(h)

This is the part of javascript from that page that changes the title.

Since BeautifulSoup doesn't run javascript, the title won't change in BeautifulSoup - it will retain the starting title from the html source. The browser will execute javascript and dynamically change it later.

nosklo
  • 217,122
  • 57
  • 293
  • 297
  • Opps... I'm sorry. If I had read your answer more carefully yesterday, I would not have asked such a question today, but it was my fault that I could not read your answer carefully. – Kun C Aug 15 '18 at 20:00
  • In fact, I didn't know much about programming so I didn't know exactly what You was talking about yesterday.(Even I don't know what dynamic site is..) I should have asked you a question again, but I didn't do it and made it bother you. I'm sorry. – Kun C Aug 15 '18 at 20:04
  • @KunC no problem, hope that it is clear now - pages are dynamic these days, there is programming code running and creating/changing/removing elements all the time. – nosklo Aug 15 '18 at 20:05
  • I see. Can I ask you to answer my questions if I have trouble while I make program? Thank you for your kind reply, and Have a nice day! – Kun C Aug 15 '18 at 20:11