Zend_Http_Client and cURL are removing newlines

Question

I'm trying to fetch a CSV file from a remote server and download it Using Zend_Http_Client

The fetched version has all of the newlines removed.

require_once('Zend/Http/Client.php');
$client = new Zend_Http_Client($url);
//also tried the curl adapter but no change

$client->setCookieJar();
$client->setAuth('user', 'pass', Zend_Http_Client :: AUTH_BASIC);

if(!empty($params)){
   $client->setParameterGet($params);
}


$client->request();
$request = $client->getLastRequest();
                                                                                                                                            

$response = $client->getLastResponse();
echo $response->getRawBody();

The response is all one line.

If I fetch the $url with curl it is on separate lines.

Also, I am looking at the source, not the HTML rendered version

UPDATE

So I rewrote that bit using cURL and it still does the same thing !?

 if(!empty($params)){
      $queryString = http_build_query($params);
      $url.='?'.$queryString;
 }

 $ch = curl_init($url);

 curl_setopt($ch,CURLOPT_USERPWD,"$username:$password");
 curl_exec($ch);

Any ideas

Looks like the server sends it without the newlines then! – BenMorel May 23 '11 at 11:46 — BenMorel, May 23 '11 at 11:46

score 1 · Answer 1 · answered May 23 '11 at 10:39

1

Can you try to setup Zend_Http_Client with the cURL adapter:

$client->setAdapter(new Zend_Http_Client_Adapter_Curl());

Also, are you sure you're not displaying $response->getRawBody() in your browser, which interprets it as HTML, therefore interpreting newlines as spaces? If you right click -> show source, do you have the newlines?

answered May 23 '11 at 10:39

BenMorel

34,448
50
182
322

Thanks but I tried `Zend_Http_Client_Adapter_Curl` , didn't work. – Yisrael Dov May 23 '11 at 10:53
Also I am looking at the source not the html rendered version. – Yisrael Dov May 23 '11 at 10:54

score 1 · Answer 2 · answered May 24 '11 at 05:47

1

Why are you using getRawBody() and not getBody()? rawBody() is usually not the one you want, and might be encoded in some form.

In any case can you post the response headers you get from the server? Also a link to the actual file or a few lines of it would help.

$response = $client->getLastResponse();
echo $response->getHeadersAsString();

answered May 24 '11 at 05:47

shevron

3,463
2
23
35

I used `getBody()` at the beginning, that is where the problem started. – Yisrael Dov May 24 '11 at 10:21
Also I can't post the file ( sensitive info ). Also if there is an error on the query string, I get back html and not csv. The html **does** have the line endings. – Yisrael Dov May 24 '11 at 10:22
Can you at least post the response headers? Even a couple of lines from the body, with sensitive material deleted, would help. In general it is highly unlikely that Z_H_C strips any data from the body. Sorry for asking stupid questions, but what are you using to view the source? Is is possible that it is some Windows program that makes "\n" line endings appear in one line because they are not "\r\n"? Can you check the CSV file in some hex viewer to *ensure* that there is nothing between the lines? – shevron Jun 15 '11 at 20:48

score 0 · Answer 3 · answered May 23 '11 at 20:09

Not really an answer, but a work around is to use the curl system call.

It looks like it is an issue with the line endings, they aren't getting detected even when I set the ini value.

 $urlArray = parse_url($url);

//put the params together
 if(!empty($params)){
      //split up any existing params
      $qsArray = parse_str($urlArray['query']);
      if(empty($qsArray)){
           $urlArray['query']=http_build_query($params);
      }
      else{
           $urlArray['query'] = http_build_query(array_merge($qsArray,$params));
      }
 }

 //set the username and password
 $urlArray['user']=$username;
 $urlArray['pass']=$password;


// http_build_url doesn't work so doing it by hand

 $urlString  = $urlArray['scheme'];
 $urlString .= "://";
 $urlString .= $urlArray['user'].':'.$urlArray['pass'] .'@';
 $urlString .= $urlArray['host'];
 $urlString .= $urlArray['path'];
 $urlString .= '?'.$urlArray['query'];

//     $urlString = http_build_url($urlArray);
//     echo($urlString);

//php is messing up the line endings, so using a system call
 return  `curl '$urlString'`;

If you can share the response headers from the Zend_Http_Client call and the response headers from command line curl (run curl -I ) this might give a clue as to why this is happening. — shevron, May 24 '11 at 05:50

Zend_Http_Client and cURL are removing newlines

UPDATE

3 Answers3