0

I am trying to fetch jobs from Worday job boards. One example is Proctor and Gamble here: https://pg.wd5.myworkdayjobs.com/en-US/1000

I can visit the site and see the jobs in the browser. When I look at the network tab I see there's a https://pg.wd5.myworkdayjobs.com/wday/cxs/pg/1000/jobs endpoint that returns all the job listings as JSON via a POST request:

enter image description here

I've written some code to mimic this POST request using PHP 8 and Guzzle 7 but every time I send the request I get a timeout. Is there a way to send a request to this endpoint to get the jobs back as JSON?

This is the code I wrote to hit the endpoint that keeps getting timeouts back:

            $headers = [
                'Accept' => 'application/json',
                'Content-Type' => 'application/json',
                'Accept-Encoding' => 'gzip, deflate, br',
                'Accept-Language' => 'en-US',
                'Connection' => 'keep-alive',
                'Content-Length' => 58,
                'Host' => 'pg.wd5.myworkdayjobs.com',
                'Origin' => 'https://pg.wd5.myworkdayjobs.com',
                'Referer' => 'https://pg.wd5.myworkdayjobs.com/en-US/1000',
            ];
            $client     = new \GuzzleHttp\Client([
                'headers' => $headers
            ]);
            $response = $client->post("https://pg.wd5.myworkdayjobs.com/wday/cxs/pg/1000/jobs", [
                'timeout' => 5,
                'form_params' => ["appliedFacets" => [], "limit" => 20, "offset" => 0, "searchText" => ""]
            ]);

Is there a way I can modify this code to get a 200 response back from the URL like my browser does?

Note: there is an old answer here but it is out of date.

Connor Leech
  • 18,052
  • 30
  • 105
  • 150
  • can you try after adding `'cookies' => true` in the client constructor array below `header` ? – Midhun Jan 24 '23 at 07:38
  • If you are getting a timeout and not an HTTP response, I’d increase the duration to hopefully see an error message. It is possible that your scraping has been noticed and you are being blocked, too. – Chris Haas Jan 24 '23 at 13:07
  • These changes have no affect :( – Connor Leech Jan 31 '23 at 17:27

0 Answers0