1

If Nginx works as reverse proxy between Apache and the web, where should i add X-Robots-Tag (to add some additional headers) - in the htaccess of Apache or in the Nginx configuration file?

Evgeniy
  • 2,337
  • 2
  • 28
  • 68
  • You can do it both ways ) – Ivan Shatsky Feb 06 '20 at 12:41
  • @IvanShatsky Really? are you sure, that a rule from htaccess goes through Nginx? I ask namely because i realize some issues, where a cause could be that a rule from Apache http.conf or htaccess cant go through Nginx... Like X-Robots-Tag with noindex at Apache level and absence of it in the header i get from Nginx. – Evgeniy Feb 06 '20 at 12:54
  • I think nginx will preserve any header it gets from upstream (except you force it to do something else). – Ivan Shatsky Feb 06 '20 at 12:56
  • `nginx will preserve any header` - thats it, i miss some headers and can't debug their absence. – Evgeniy Feb 06 '20 at 12:58
  • @Evgeniy did you ever figure this out? One of our servers are setup to use a Reverse Proxy so doing what Google asks is not working correctly with X Robots Tag header in htaccess. – Carlitos Apr 20 '23 at 22:43

1 Answers1

1

You can do it in either place. Although, for a header like X-Robots-Tag, which is very much page specific then it would probably be easier to set this at the application server (in your application/CMS). Devs may not have access to the proxy server in order to apply specific headers.

Alternatively, you might choose to do this in the front-end proxy if you wanted to apply this to every request, or to a specific pattern.

But you should not apply the same header in both places, as that becomes hard to maintain (and debug).

MrWhite
  • 43,179
  • 8
  • 60
  • 84
  • You would also recommend to setup `X-Robot-Tag` on the Nginx level, if applyed to specific pattern? – Evgeniy Feb 06 '20 at 15:33
  • I wouldn't necessarily "recommend" doing it that way - but you certainly could. If this is related to `.pdf` files (as in your related ServerFault question) then yes, it would make sense to do this in the front-end proxy since this is unlikley something that would be controlled by the backend web-application. – MrWhite Feb 06 '20 at 18:10
  • Could you recommend, how to debug it? Developer said, X-Robots-Tag is implemented as in the documentation by Google, https://developers.google.com/search/reference/robots_meta_tag, on both levels, Apache and Nginx. But it isn't in the header at all. – Evgeniy Apr 24 '20 at 11:14