Trying to replicate web request (image upload) using php and curl

Question

I'm trying to replicate a image upload to a website yet that website don't give an api function for that. I managed to get the request information using Charles Proxy:

Charles Proxy screenshot

Here is my php code:

$post_data = array(
    'photo' => '@'.$filename,
    '_csrftoken' => '5ebcec201972ab6304a33d418129cd13',
);

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://example.com/api/v1/upload/photo/');
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    'Host: example.com'
));
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
curl_setopt($ch, CURLOPT_COOKIEFILE, 'C:/xampp/htdocs/example/cookies.txt');            


$response = curl_exec($ch);
$http = curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);

print_r($response);

echo $http;

This returns a response with http code of 500.

score 0 · Answer 1 · answered Mar 20 '15 at 13:15

You are not posting correctly.

You do not need Charles Proxy

Before you do the upload (chrome,firefox),

right click select Inspect Element
Select the Network tab
Refresh the page
Select Documents (chrome) or HTML (firefox)
Clear the list
Post your upload
Select the upload Request in the list of Requests
In fireFox Select "Edit and Resend" In Chrome Select "View Source"

On the right side it will display Request and Response Headers

You need to make your Request look exactly like that Request Header

You have to watch for Redirects (e.g. 302) and Cookies that are added during the Redirect.

You are going to want to see the Request and Response Headers in case it does not work to see what went wrong.

curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_FAILONERROR,true);

You may want to get your cookies. Create another curl request to get the upload page.

To capture cookies: do curl request for upload page get the Response header ($head)

$data = curl_exec($ch);

if (curl_errno($ch)){
    $data .= 'Retreive Base Page Error: ' . curl_error($ch);
}
else {
  $skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE)); 
  $head = substr($data,0,$skip);
  $e = 0;
  while(true){
    $s = strpos($head,'Set-Cookie: ',$e);
    if (!$s){break;}
    $s += 12;
    $e = strpos($head,';',$s);
    $cookie = substr($head,$s,$e-$s) ;
    $s = strpos($cookie,'=');
    $key = substr($cookie,0,$s);
    $value = substr($cookie,$s);
    $cookies[$key] = $value;

  }

Format the captured for the upload request:

 $cookie = '';
 $show = '';
 $head = '';
 $delim = '';
 foreach ($cookies as $k => $v){
   $cookie .= "$delim$k$v";
   $delim = '; ';
 }

You need to add some options to your curl

Create the POST data string

$post = 'key1=value1&key2=value2&key3=value3';

curl_setopt($ch, CURLOPT_POSTFIELDS, $post);

Create an array to put the Request Header Key Values
Fill in the Request array with exactly what is in the Request header of your upload.
EXAMPLE:

$request = array();
$request[] = "Host: www.example.com";
$request[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$request[] = "User-Agent: MOT-V9mm/00.62 UP.Browser/6.2.3.4.c.1.123 (GUI) MMP/2.0";
$request[] = "Accept-Language: en-US,en;q=0.5";
$request[] = "Connection: keep-alive";
$request[] = "Cache-Control: no-cache";
$request[] = "Pragma: no-cache";

Add to curl:

curl_setopt($ch, CURLOPT_HTTPHEADER, $request);

Set follow to false. If there is a Redirect you can see what is happening. then create another curl request to the redirected location.

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);

After the upload curl request Request get the Headers:

$ch = curl_init($url);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, false);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_FILETIME, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 10);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_TIMEOUT,10);
curl_setopt($ch, CURLOPT_FAILONERROR,true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $request);
$data = curl_exec($ch);
if (curl_errno($ch)){
    $data .= 'Retreive Base Page Error: ' . curl_error($ch);
}
else {
  $skip = intval(curl_getinfo($ch, CURLINFO_HEADER_SIZE)); 
  $head = substr($data,0,$skip);
  $data = substr($data,$skip);
  $info = curl_getinfo($ch);
  $info = var_export($info,true);
}
echo $head;
echo $info;

If it did not work correctly examine the differences in the Request Header in the $info.

Right, the problem is that I can't replicate exactly the request... I'm closer yet I don't know what the problem is... — Th3lmuu90, Mar 20 '15 at 18:33
By using the `$request` array you should be able to get very close. If you replicate the Browser Request as shown in the Browser Development tools (FireFox Inspector), the site cannot differentiate between curl and a Browser. The answer to why it's not working should be in `$info` — Misunderstood, Mar 20 '15 at 18:36
I managed to replicate it with the help of this link: http://stackoverflow.com/questions/18318052/php-curl-post-with-file-attachement-custom-content-type-header — Th3lmuu90, Mar 20 '15 at 22:30
Curious, were you able to see the Request Header in the Browser using Chrome Dev Tools or FireFox Inspector like I suggested? I do not understand why you needed additional info. Everything you needed to know should have been in the Request Headers. — Misunderstood, Mar 20 '15 at 22:46

Trying to replicate web request (image upload) using php and curl

1 Answers1