Crawlable AJAX with _escaped_fragment_ in htaccess

Question

Hello fellow developers!

We are almost finished with developing first phase of our ajax web app. In our app we are using hash fragments like:

http://ourdomain.com/#!list=last_ads&order=date

I understand google will fetch this url and make a request to the server in this form:

http://ourdomain.com/?_escaped_fragment_=list=last_ads?order=date&direction=desc

everything is perfect, except...

I would like to route this kind of request to another script

like so:

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
RewriteRule ^$ /webroot/crawler.php$1 [L]

The problem is, that when I try to print_r($_REQUEST) in crawler.php I get only:

Array
(
    [_escaped_fragment_] => list=last_ads?order=date
    [direction] => desc
)

what I'd like to get is

Array
(
    [list] => last_ads
    [order] => date
    [directions] => des
)

I know I could use php to further break the first argument, but I don't want to ;)

please advise

==================================================== EDIT... some corrections in text and logic

Olivier Pons · Accepted Answer · 2013-12-13T14:19:44.407

3

Your forgot QSA directive (everyone missed the point =D )

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
RewriteRule ^$ /webroot/crawler.php%1 [QSA,L]

By the way your $1 is well err... useless because it refers to nothing. So this should be:

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
RewriteRule ^$ /webroot/crawler.php [QSA,L]

Tell me if this works.

edited Dec 13 '13 at 14:19

answered Nov 22 '11 at 20:29

Olivier Pons

15,363
26
117
213

I think it should be '%1' instead of '$1' http://stackoverflow.com/questions/6654834/difference-between-1-vs-1-in-htaccess – yunzen Dec 13 '13 at 13:49
1

Then you should drop that comment about the $1 as well. – yunzen Dec 13 '13 at 23:04
In his question there's a line of code : `RewriteRule ^$ /webroot/crawler.php$1 [L]` The `$1` is useless: either it's `%1` like you said, or it's `RewriteRule (.*) /webroot/crawler.php?someparam=$1 [L]` but it's not correct like it was written. – Olivier Pons Dec 15 '13 at 10:30

score 1 · Answer 2 · answered Nov 09 '11 at 15:26

1

If I'm not mistaken.

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
RewriteRule ^$ /webroot/crawler.php?%1 [L]

answered Nov 09 '11 at 15:26

Gerben

16,747
6
37
56

Its better if in crawler.php add print_r($_GET); output will be an array [ Array ( [order] => date .... ) ] not string like in other examples – user956584 Feb 09 '12 at 15:11

score 1 · Answer 3 · answered Nov 22 '11 at 14:54

1

Maybe is obvious for you, but in the documentation talk about escaped characters: Set up your server to handle requests for URLs that contain

The crawler escapes certain characters in the fragment during the transformation. To retrieve the original fragment, make sure to unescape all %XX characters in the fragment. More specifically, %26 should become &, %20 should become a space, %23 should become #, and %25 should become %, and so on.

answered Nov 22 '11 at 14:54

lluisi

56
1

I had some problem witch "&26" so i write simple fix: if ( strstr($_SERVER['QUERY_STRING'],'%26') ) : $_SERVER['QUERY_STRING'] = urldecode($_SERVER['QUERY_STRING']); parse_str($_SERVER['QUERY_STRING'], $_GET ); endif; – user956584 Jun 06 '12 at 23:17

Shiv Shankar · Answer 4 · 2012-05-20T05:09:33.980

Here is a solution that provides a routable URL and query parameters correctly set for processing in the server side script.

Example: If you want http://yoursite.com/#!/product/20 to become http://yoursite.com/crawler/product/20

First in .htaccess

RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
RewriteRule ^$ /crawler/index.php?_frag=%1  [L]

We need to get rid of the _escaped_fragment_ in the URL and replace it with something different, example: _frag so that the (Apache) web server does not get in to circular rewrites.

Second in crawler/index.php

<?php

if(array_key_exists('_frag', $_REQUEST)) {
    $_SERVER['REQUEST_URI']=$_REQUEST['_frag'];
    unset($_REQUEST['_frag']);
    parse_str($_SERVER['QUERY_STRING'], $frag); 
    parse_str(preg_replace('/^.*\?/', '', $frag['_frag']), $_REQUEST);
    $_SERVER['QUERY_STRING'] = http_build_query($_REQUEST);
}

// Continue with your usual script of routing
// $_REQUEST now contains the original query parameters

score 0 · Answer 5 · answered May 22 '12 at 02:19

0

In htacess work in virtual host not working, so i add in "directory"

<Directory "X:/DIR">
    RewriteEngine On
    RewriteCond %{QUERY_STRING} ^_escaped_fragment_=(.*)$
    RewriteRule ^$ /crawler/index.php?_frag=%1  [L]
</Directory>

answered May 22 '12 at 02:19

user956584

5,316
3
40
50

Crawlable AJAX with _escaped_fragment_ in htaccess

5 Answers5

Linked