1

I have a site that's about to be taken down in month's time. What I need to do is place a robots.txt to prevent the search engine from indexing it any longer. However, after I placed the file on the root of the solution in the web server, and tried to check if I can access it by typing www.sitename.com/robots.txt, it just refreshes the screen or perhaps just returns to the home page. My application is running in MVC 3.

Things I've already tried:

  1. Added modules runAllManagedModulesForAllRequests="true" in web.config
  2. Used Dynamic Robots.Txt using IHttpHandler and Controller/Action approach (based on this link robots.txt file for different domains of same site)
  3. Played around with MIME Types (removed .txt and back)

I am expecting to see the contents of the robots.txt file same as when I access a css file or js file just by appending the file name in the URL.

nevrome
  • 1,471
  • 1
  • 13
  • 28
Warren
  • 11
  • 2

0 Answers0