how do some websites use any URL requested to create a page on the fly without 404's

Question

For example: http://www.ebuyer.com/flange

this url does not esist but when you visit the site it takes you to a search page where it has the search results for flange.

firstly i though that they had used .htaccess and ErrorDocumnet to point to a search script to return the search for what ever was in the url.

This is fairly easy to do, but what ihave noticed is that ebuyer responds with a 200 OK and my method responds with 404 Not found. The user will never see this but im guessing for SEO it would be better practice for this to return 200 OK.

how have they done this?

Thanks

Answer 1

You're right that it is NOT done via ErrorDocument directive since that returns 404 status to the browsers. It is actually done via RewriteRule. See following example:

RewriteCond %{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_FILENAME} !-d  [OR]
RewriteCond %{REQUEST_FILENAME} !-l
RewriteRule ^(.*)$ search.php?q=$1 [L,QSA]

Which simply means if requested file or directory or link doesn't exist on your web server then internally forward the request to a script called search.php and pass requested URI as search parameter q. In the end a HTTP 200 is returned to the browser instead of 404.

e.g. /flange would become /search.php?q=flange internally and return 200 to the browser.

Answer 2

What's happening is that there's some routing rules in play. Basically, a bit of code sees the URL (for example: www.ebuyer.com/flange ) and knows that the "flange" aspect is a key word it is supposed to try and find and display.

For a similar concept, like at the URL for this question. It's broken down into the following logical concepts:

[domain]/[function]/[identifier]/[extra information for human readability]

This tells the module on the server that it needs to look in the "questions" functionality, and deliver the question with the identifier of 14689056.

It's easy enough to have the system respond with a search results page as opposed to a 404 in this situation, because you're already re-directing the user behind the scenes based upon the URL.

In .NET (3.5+) and IIS, there's a nice set of classes associated with URL Routing that can be manipulated in the Global.Asax to do this (earlier versions of .NET can do this as well, it's just a bit more hassle involved since there are no native classes).

In apache, you'll need to deal with mod_rewrite and other odds an ends that I'm not overly familiar with as I'm a .NET programmer. This question Routing URLs in PHP will get you started heading in the direction you need to go.