0

I'm trying to scrape some information from a javascript heavy web site. Usually I can piece some way to read a page from the xhr header information using chrome's inspector or firebug, but nothing I've tried so far works.

I could use selenium or the like, but I'd rather use requests, if possible.

Here's a sample url: https://www.fedex.com/apps/fedextrack/?action=track&cntry_code=in&tracknumbers=770562143139 I'm trying to get the delivery status and related info.

I'm hoping some combination of gets or posts with appropriate parameters will work. I realize there are many similar questions here, such as Scrape ajax web page with python and/or scrapy or How to call ajax in a webpage from python script without browser emulation or headless brawser?

Community
  • 1
  • 1
foosion
  • 7,619
  • 25
  • 65
  • 102
  • If this is about tracking, you could just use FedEx API that can provide info on delivery info among other things. – Saikiran Yerram Mar 31 '15 at 12:42
  • @SaikiranYerram It's just for tracking deliveries to me. I thought I needed to be a business and get credentials to use the API. I'll take another look. In any event, I'm still interested in figuring how to scrape the page. – foosion Mar 31 '15 at 13:08
  • @SaikiranYerram I can't figure out how to use the API without a key and password and they want an extensive application for that. Also, their sample code requires some libraries and they don't offer a python library, although https://pythonhosted.org/fedex/ looks promising. – foosion Mar 31 '15 at 13:20
  • You can use `requests` library to make the api call. I would imagine they want to just get profile of the developer or company, which makes sense. Also, Python isn't a right tool for running JS scripts. Have you looked at PhantomJS.org – Saikiran Yerram Mar 31 '15 at 13:30
  • @SaikiranYerram I've used phantomJS with selenium. I'd like something much lighter than that, if possible. It usually is possible to find the right url and cookie combination to get a javascript generated page. It certainly makes sense for FedEx to know its developers. The problem for an individual is getting properly signed up. – foosion Mar 31 '15 at 13:39

0 Answers0