I am using php to scrape WWW. Example.com which requires session to login .I have logged successfully using php curl. Please check this link Keeping session alive with curl and php.Now In the link it shows how to make second request using same session cookie.
My problem is when i scrape
define("COOKIE_FILE", "cookie.txt");
// Login the user
$ch = curl_init('http://api.example.com/login/joe/smith');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
// Read the session saved in the cookie file
echo "<br/><br/>";
$file = fopen("cookie.txt", 'r');
echo fread($file, 100000000);
echo "<br/><br/>";
// Get the users details
$ch = curl_init('http://api.example.com/user');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
My code is
define("COOKIE_FILE", "cookie.txt");
// Login the user
$ch = curl_init('http://api.example.com/login/joe/smith');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
echo curl_exec ($ch);
// Read the session saved in the cookie file
echo "<br/><br/>";
$file = fopen("cookie.txt", 'r');
echo fread($file, 100000000);
echo "<br/><br/>";
// Get the users details
$ch = curl_init('http://api.example.com/user');
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, 'username=abc&password=xyz&domain=123');
echo curl_exec ($ch);
The output is
# Netscape HTTP Cookie File # https://curl.haxx.se/docs/http-cookies.html # This file was generated by libcurl! Edit at your own risk. www.example.com FALSE /unifyv3 FALSE 0 JSESSIONID 5366C3DB90DE7187A66E0BD19F595309
HTTP/1.1 200 OK Date: Fri, 05 Aug 2016 19:48:25 GMT Server: Apache/2.2.3 (CentOS) Set-Cookie: JSESSIONID=225D8E499DE23F0FDF454D73F0C05F25; Path=/unifyv3 Content-Length: 2850 Connection: close Content-Type: text/html;charset=UTF-8
Logout
Your session has expired
Click here to log in again
Redirecting to Login Page in secs.
Everything works Perfectly well when the second request is GET request. When it comes to POST request the website returns "Your Session is Expired Please Login Again".
How this problem can be solved?.I have tried several solutions and searched in google, Nothing works.Finally decided to post here. Thanks in advance.