How can I curl deepcrawl?

Question

My Deepcrawl crawl only give null value.

$ch = curl_init();

curl_setopt($ch, CURLOPT_URL,"https://api.deepcrawl.com/accounts/00000/projects/195334/crawls/1306396/reports/thin_pages_basic");

curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch,CURLOPT_HTTPHEADER,array('X-Auth-Token:Private'));

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$server_output2 = curl_exec ($ch);
curl_close ($ch); 

$results = json_decode($server_output2);

To create a session you first need to manually generate an API Key. This is a key/value pair that can be generated in the API Access page. When you click Generate New API Key, a popup will be displayed with the API Key Value and in the table of Active Keys you can find the API Key ID. This is then sent via Basic Authentication in a POST call to the sessions route.

curl -X POST -u '123:abcdef' https://api.deepcrawl.com/sessions
{
"token":"abcdef123",
"_user_href":"/users/example-user",
...
}

The token returned from the call to sessions is then passed to all API calls as the X-Auth-Token header:

curl -X GET -H 'X-Auth-Token:' https://api.deepcrawl.com/accounts/1/projects

Can someone explain me further about the authentication of deepcrawl? How can I able to curl it using X-Auth-Token only. Sorry for my bad English. Thank you

@delboy1978uk curl_error(): supplied resource is not a valid cURL handle resource — Christian Gallarmin, Oct 03 '18 at 15:22
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/181232/discussion-between-christian-gallarmin-and-delboy1978uk). — Christian Gallarmin, Oct 03 '18 at 15:28
Have a read here https://secure.php.net/manual/en/function.curl-error.php — delboy1978uk, Oct 03 '18 at 15:29
Operation completed without any errors I got this but json_decode() still null then. — Christian Gallarmin, Oct 03 '18 at 15:35

score 2 · Answer 1 · answered Oct 04 '18 at 07:49

CURL may be tricky sometimes.

When I face problems with CURL I first try to do the same on command line, in this case it worked perfectly so we can reject there is any kind of problem on API or on CURL so we must assume the problem is on the php call.

When you get a NULL it's likely there is an error set on curl handler, you just need to look for it and bring it to screen:

    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL,"https://api.deepcrawl.com/accounts/00000/projects/195334/crawls/1306396/reports/thin_pages_basic");

    curl_setopt($ch, CURLOPT_POST, 0);
    curl_setopt($ch,CURLOPT_HTTPHEADER,array('X-Auth-Token:Private'));

    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $server_output2 = curl_exec ($ch);

    if (curl_errno($ch)) {
        print curl_error($ch);
        die();
        // or whatever you might want to do with this error
       } 
    curl_close ($ch); 

    $results = json_decode($server_output2);

In this case the error was:

SSL certificate problem: unable to get local issuer certificate

Here you can see how to fix this problem, just need to add a valid CA cert to your php.ini (in the example cacert is used) like that:

1) Download the latest cacert.pem from https://curl.haxx.se/ca/cacert.pem

2) Add the following line to php.ini (if this is shared hosting and you don't have access to php.ini then you could add this to .user.ini in public_html)

curl.cainfo="/path/to/downloaded/cacert.pem"

How can I curl deepcrawl?

1 Answers1

Linked