6

is there a way of doing

curl -X POST -H "Content-Type:application/xml" --data @myfile.xml http://example.com

but directly in PHP?

CURLOPT_PUT/CURLOPT_UPLOAD as well as file_get_contents as well as exec

are not a solutions as it must be POST and the file is huge so must be streamed.

Any ideas?

Marcin
  • 5,469
  • 15
  • 55
  • 69
  • 1
    To post a file to a server, try this: http://stackoverflow.com/questions/4223977/send-file-via-curl-from-form-post-in-php – silkfire Mar 19 '13 at 19:42

2 Answers2

14

I had a similar problem trying to feed huge ingestion files to elasticsearch's bulk API from PHP, until I realized that the bulk API endpoint accepted PUT requests. Anyway, this code performs POST requests with huge files:

$curl = curl_init();
curl_setopt( $curl, CURLOPT_PUT, 1 );
curl_setopt( $curl, CURLOPT_INFILESIZE, filesize($tmpFile) );
curl_setopt( $curl, CURLOPT_INFILE, ($in=fopen($tmpFile, 'r')) );
curl_setopt( $curl, CURLOPT_CUSTOMREQUEST, 'POST' );
curl_setopt( $curl, CURLOPT_HTTPHEADER, [ 'Content-Type: application/json' ] );
curl_setopt( $curl, CURLOPT_URL, $url );
curl_setopt( $curl, CURLOPT_RETURNTRANSFER, 1 );
$result = curl_exec($curl);
curl_close($curl);
fclose($in);

Here, $tmpFile is the full path of the file containing the request body.

NOTE: The important part is setting CURLOPT_CUSTOMREQUEST to 'POST' even though CURLOPT_PUT is set.

You'd have to adapt the Content-Type: header to whatever the server is expecting.

Using tcpdump, I could confirm the request method to be POST:

POST /my_index/_bulk HTTP/1.1
Host: 127.0.0.1:9200
Accept: */*
Content-Type: application/json
Content-Length: 167401
Expect: 100-continue

[...snip...]

I'm using packages libcurl3 (version 7.35.0-1ubuntu2.5) and php5-curl (version 5.5.9+dfsg-1ubuntu4.11) on Ubuntu 14.04.

Shadocko
  • 1,186
  • 9
  • 27
  • 1
    +1 for verifying with tcpdump. Setting options and trusting that it was all that I needed to do (i.e., options never conflict, option order is irrelevant, *et cetera*) has bitten me more than once. – LSerni Jul 30 '15 at 15:49
  • Well, I'm a bit uncomfortable with the conflicting options, that's why I listed the package versions. In my case`CURLOPT_CUSTOMREQUEST` overrides other options but I'm not sure it's working *as intended* and is a durable solution. – Shadocko Jul 31 '15 at 07:33
  • Many thanks! I am posting/putting raw file data straight to an API, and while the PUT was easy using CURLOPT_INFILE I could not figure out how to get POST to work since it always uses multipart form data. This was exactly the part I was missing - just set the custom request to POST. – phansen Nov 22 '16 at 19:00
  • Neer pallandu vaalga! :D :D :D Saved my day :) – Sibidharan Aug 16 '17 at 18:32
1

Curl supports post, so I believe you are looking for something like this: Posting or Uploading Files Using Curl With PHP

// URL on which we have to post data
$url = "http://localhost/tutorials/post_action.php";
// Any other field you might want to catch
$post_data['name'] = "khan";
// File you want to upload/post
$post_data['file'] = "@c:/logs.log";

// Initialize cURL
$ch = curl_init();
// Set URL on which you want to post the Form and/or data
curl_setopt($ch, CURLOPT_URL, $url);
// Data+Files to be posted
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_data);
// Pass TRUE or 1 if you want to wait for and catch the response against the request made
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// For Debug mode; shows up any error encountered during the operation
curl_setopt($ch, CURLOPT_VERBOSE, 1);
// Execute the request
$response = curl_exec($ch);

// Just for debug: to see response
echo $response;

(All credit for code to original linked author).

As to "huge", you'll want to be more specific - kb, mb, gb, tb? The additional problems will be related to how long a PHP script can stay alive without being auto-terminated, script memory usage (which may require things be handled in chunks instead of loading the whole file), etc.

EDIT: Ah, for RAW post I think you'll be needing this then: Raw POST using Curl in PHP

Community
  • 1
  • 1
BrianH
  • 2,140
  • 14
  • 20
  • that is not a RAW POST with Content-Type:application/xml, it is a Content-Type:multipart/form-data. By huge I mean you can't load in memory – Marcin Mar 19 '13 at 19:44
  • Ah, well if you can't load it in memory then I see what you mean, none of these will work as curl_exec() will wait until you are done to send - by which time the script will have crashed out. – BrianH Mar 19 '13 at 19:53
  • not if you can stream, same way as you do with generic post and @ – Marcin Mar 19 '13 at 19:56
  • 1
    This is going to be trickier than I had first thought as PHP is not particularly supportive of this sort of thing out of the box. I suppose at this point the best you can hope for is a non-trivial solution. The best I can figure is that you'll need a stream wrapper where you can manually send to stream, as discussed here: http://stackoverflow.com/questions/1342583/php-manipulate-a-string-that-that-is-30-mil-chars-long/1342760#1342760 It's a mess, but it might be the only solution using PHP alone. – BrianH Mar 19 '13 at 20:07