0

I get the response by scrapy shell. WHen I use view(reponse), and see the result on browser, there is tbody tag. But I check the tbody tag in the shell, it's lost. I want to know the reason.

It's scrapy version 1.7 and python3.6. I use Chrome Inspect tool to check the element, I find the tag <tbody>, like that: page But I check in the shell:

[in] "tbody" in response.text
[out] False

[in] "tr" in response.text
[out] True

So, I think it's weird. why does it happen?

RayZen
  • 141
  • 1
  • 4

2 Answers2

1

The <tbody> tag is always added by the browser DOM, even if the table did not have one in the source, it is implicit and the browser corrects the "oversight" of the page author.

You can either check the real page source for it, or write your XPath syntax to allow for either case.

Also see Why do browsers insert tbody element into table elements?

nyov
  • 1,382
  • 7
  • 23
1

You should always go for the page source (Ctrl + u or Right-click -> View page source) to know the exact position of the Elements specially in case of scrapy or Xpath.

View Source will only show the original HTML used to create the page initially whereas Inspect Element shows the DOM in its current form.

Madhav Kumar
  • 161
  • 2
  • 7