file_get_contents and Curl not working on Heroko FREE

Question

Edit 4:

This is only an issue if you are using the free version of heroku (with one web dyno) and I made a workaround below. But it is still a valid question so I'm not sure what the reason(s) are for down voting...

This works locally and on another web hotel.

Below .htaccess works for index but when a "subdir"/mod_rewrite dir is accessed like category/Drama all I get is "Application Error"

Edit

So what I've done is, instead of all rewrites going via .htaccess I send every request like this: http://example.com/CatX/CatY/CatZ/ to http://example.com/redirs.php?data=CatXCatYCatZ. Then in redirs i fetch the page and output it, but this only gives application error on heroku and checking logs is even more confusing because it reports failing to get the request first then goes ahead and fetches it.

Problem url example: http://movie-nights.herokuapp.com/category/Crime/

I only have free addons and base package. I tried curl also but that doesn't output the data should have been fetched via curl - empty result but redirs.php works.

"Application Error"

"An error occurred in the application and your page could not be served. Please try again in a few moments.

If you are the application owner, check your logs for details."

RewriteEngine on
RewriteBase /

RewriteRule ^(.*)/css/(.+)$ css/$2 [L]
RewriteRule ^(.*)/bilder/(.+)$ %{SCRIPT_URI}bilder/$2 [L]
RewriteRule ^(.*)/grafik/(.+)$ grafik/$2 [L]
RewriteRule ^(.*)/js/(.+)$ %{SCRIPT_URI}js/$2 [R]
RewriteRule ^(.*)/lightbox/(.+)$ %{SCRIPT_URI}lightbox/$2 [L]
RewriteRule ^(.*)/bootstrap/(.+)$ bootstrap/$2 [R]

RewriteRule ^FAQ(/)*$ %{SCRIPT_URL}faq.php [L]
RewriteRule ^MovieDescription/$ %{SCRIPT_URL}xxx/xxx.php [L]
RewriteRule ^stream/url\=(.+)$ $1 [R]

RewriteRule ^id/([0-9]*)$ id/$0/ [L]
RewriteRule ^id/([0-9]*)/(.*)$ id/$0/ [L]

RewriteCond %{REQUEST_FILENAME} !-f            #<-- here
RewriteRule ^(.*)/$ redirs.php?data=$0 [QSA]   #<-- here

redirs.php

<?php
function makeUrl($attr, $url) {
    if (preg_match('#\?#', $url)) {
        $attr = '&'. $attr;
    } else {
        $attr = '?'. $attr;
    }
    return $attr;
}

function getWebCwd() {
    $dir = '';
    if (!empty($GLOBALS['BaseCat'])) {
        $dir = $GLOBALS['BaseCat'];
    }
    if (!empty($GLOBALS['subUrl'])) {

        // Can be for example:
        //$subUrl = 'Streamed-Movies/';
        $dir .= $GLOBALS['subUrl'];
    }
    return $dir;
}

$url = '/index.php';
$data = $_GET['data'] . getWebCwd();

if (preg_match('#category\/([a-zA-Z]+)#iu', $data, $matches)) {
    $url .= makeUrl('category='. $matches[1], $url);
}
$sid = session_id();
if (!empty($sid)) {
    $url .= makeUrl(session_name() .'='. $sid, $url);
}
$url = 'http://'. $_SERVER['SERVER_NAME'] . $url;
$opts = array('http' => array(
    'method' => "GET",
    'header' => "Accept-language: en\r\n" . "Cookie: ".session_name()."=".session_id()."\r\n" ) 
);
$context = stream_context_create($opts);
session_write_close();   // this is the key
$html = file_get_contents($url, false, $context);       // <-- line 92  
echo $html;

Error logs from Papertrail addon: (requested url is currect and works if you try to open it)

Jan 02 00:40:49 movie-nights app/web.1:  [Thu Jan 02 08:40:49 2014] [error] [client 10.34.132.2] PHP Warning:  file_get_contents(http://movie-nights.herokuapp.com/index.php?category=Crime&PHPSESSID=c54qgv6pgj2mme2is32qljs865): failed to open stream: HTTP request failed! HTTP/1.1 503 Service Unavailable\r\n in /app/www/redirs.php on line 92 
Jan 02 00:40:49 movie-nights app/web.1:  10.34.132.2 - - [02/Jan/2014:08:40:19 +0000] "GET /category/Crime/ HTTP/1.1" 200 - 
Jan 02 00:40:50 movie-nights app/web.1:  10.34.132.2 - - [02/Jan/2014:08:40:49 +0000] "GET /index.php?category=Crime&PHPSESSID=c54qgv6pgj2mme2is32qljs865 HTTP/1.1" 200 16864

So it tries to open http://movie-nights.herokuapp.com/index.php?category=Crime&PHPSESSID=c54qgv6pgj2mme2is32qljs865 => : failed to open stream: HTTP request failed! HTTP/1.1 503 Service Unavailable

=> but if you try to open that manually it works.

Edit2

I also get this error (below) but from what I've read about it, it shouldn't be a problem?

Jan 11 23:05:21 movie-nights app/web.1:  [Sun Jan 12 07:01:38 2014] [error] server reached MaxClients setting, consider raising the MaxClients setting

Edit3

See solution in my own answer below!

score 2 · Answer 1 · answered Jan 13 '14 at 15:40

2

It looks like your code is taking in a request, then opening a new HTTP request to your server. Unfortunately, you can only handle so many concurrent requests and you're blocking all of your workers (hence MaxClients error). This is causing your call back to your web application from within your web application to fail (with 503, unavailable).

If you actually want to redirect the user, you should return an 3XX status code with the new URL you want to redirect to. This will end the current request, return the redirect, and the user's browser will fetch the new URL.

answered Jan 13 '14 at 15:40

Winfield

18,985
3
52
65

The thing is I don't want to redirect there just do a more dynamic solution so I can match regex like /catx/view2/view3/ or /view2/catx/view3/ or /view3/catx/caty etc.. and then not have to update all possible combinations manually when I add a new feature.. something like http://stackoverflow.com/questions/16388959/url-rewriting-with-php was my intent.. but I also figured this is the problem.. – OZZIE Jan 14 '14 at 15:48
The way I want it to work can't be solved unless I change to use two or more dynos right? Which isn't free :( I'm looking into writing the .htaccess from php instead – OZZIE Jan 16 '14 at 17:57

score 0 · Accepted Answer · answered Feb 13 '14 at 18:59

I finally figured out a solution! In redirs.php I did this instead of curl or file_get_contents(url) where url is a relative url like /index.php?asf=123&b=3

$file = convUrlToGet($url);
$html = include_once($file);

function convUrlToGet($url) {
    $get = explode('?', $url);
    $file = str_replace('/', '', $get[0]);
    $get = $get[1];
    $gets = explode('&', $get);
    foreach ($gets as $g) {
        $g = explode('=', $g);
        $key = $g[0];
        $value = $g[1];
        $_GET[$key] = $value;
    }
    return $file;
}