Scrapy xpath major issues...

Asked Jan 27 '17 at 04:36

Active Jan 27 '17 at 04:36

Viewed 24 times

I really have no idea how to use xpath in scrapy... I thought I did but apparently I am lost.

Here is the page I am scraping:

http://chc.tbe.taleo.net/chc01/ats/careers/requisition.jsp?org=INTELLIGENESIS&cws=1&rid=413

Here is my XPATH for the job title:

  item['title'] = job.xpath('//*[@id="siteWrapper"]/div[4]/table/tbody/tr[2]/td/h1/text()').extract()

So... whats up with scrapy not scraping anything....

I have tried MANY different ways of writing the xpath to no avail....

asked Jan 27 '17 at 04:36

Jason Baker

Have you tried without the `tbody`? – alecxe Jan 27 '17 at 04:38
Why did that work? I have to know lol.. beating my head against this crap for an hour... the xpath works everywhere else ive tried except scrapy... – Jason Baker Jan 27 '17 at 04:40
Sure, `tbody` is generated by the browser - please see the question I've linked as a duplicate. – alecxe Jan 27 '17 at 04:41
Sorry for creating a duplicate... i searched for a while but wasn't looking for the right keywords. – Jason Baker Jan 27 '17 at 04:43

0 Answers0