mvc robots.txt route not working

Question

This is my RouteConfig file:

public static void RegisterRoutes(RouteCollection routes)
{
    routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

    routes.MapRoute("Robots.txt",
        "robots.txt",
        new { controller = "Home", action = "Robots" });

    routes.MapRoute(
        name: "Default",
        url: "{controller}/{action}/{id}",
        defaults: new { controller = "Home", action = "Index", id = UrlParameter.Optional }
    );
}

When going to www.mysite.com/Robots.txt I get 404.

In my HomeController I have a Robots() action.

What could I be doing wrong?

Edit: going to www.mysite.com/Robots gives me a server error 404. Going to www.mysite.com/Robots.txt gives me an IIS 404 error. Could that have something to do with it?

Going to www.mysite.com/Home/Robots works, but www.mysite.com/Home/Robots.txt gives IIS 404.

score 2 · Answer 1 · answered Nov 04 '14 at 11:36

2

Do you really need robots.txt available at other locations than www.mysite.com/robots.txt?

Otherwise just do:

public static void RegisterRoutes(RouteCollection routes)
{
    routes.IgnoreRoute("robots.txt");
}

And put the file in the root folder on your Visual Studio.

answered Nov 04 '14 at 11:36

David

2,551
3
34
62

1

_"In my HomeController I have a Robots() action"_ - guess OP wants a dynamically generated robots.txt file. – CodeCaster Nov 04 '14 at 11:37

score 0 · Answer 2 · answered Sep 11 '18 at 07:33

This is a bit of a late reply, but in case anyone else comes across it IIS is picking up the extension and attempting to manage it as a static file. It does not reach the routing module.

You can disable this functionality using runAllManagedModulesForAllRequests but there are performance implications. Andy Brown has listed a number of solutions and a detailed explanation here: https://stackoverflow.com/a/17037935/522859

mvc robots.txt route not working

2 Answers2