2

I want to serve a web application and Compile a template in JavaScript with tow-way data binding. on the other hand , I want to produce just ready HTML content for web crawlers.

How to know if the request come from the popular web crawlers/bots such as Google,bing,facebook,twitter??

Ahed Eid
  • 395
  • 4
  • 17
  • 1
    they usually tell you with the user agent. It'll say "google bot" or "Yahoo" or "bing" or something like that. – Epodax Oct 06 '15 at 13:04
  • Why Didn't I Think of That – Ahed Eid Oct 06 '15 at 13:06
  • 4
    http://stackoverflow.com/questions/677419/how-to-detect-search-engine-bots-with-php - That answers your question and includes a list of all search engines etc. – Peter Oct 06 '15 at 13:11

1 Answers1

2

They usually send that kind of info through the user agent, something like

  • Google crawler
  • Yahoo
  • Bing

Or something similar, in php you can find the user agent with 

$_SERVER['HTTP_USER_AGENT'];

However, you should be aware that a user agent can be spoofed.

Epodax
  • 1,828
  • 4
  • 27
  • 32