I use ASP.net to generate a sitemap.xml, get Google Webmaster says: "Your Sitemap appears to be an HTML page."

Question

I am trying to return an generated .xml file, but google picks it up as a html page. So I get: "Your Sitemap appears to be an HTML page. Please use a supported sitemap format instead."

Here is the ASP.net Controller that generate the sitemap.xml

[Route("sitemap.xml")]
public async Task<IActionResult> SitemapXmlAsync()
{

  using (var client = new HttpClient())
  {
    try
    {
      client.BaseAddress = new Uri("https://api.badgag.com/api/generateSitemap");
      var response = await client.GetAsync("");
      response.EnsureSuccessStatusCode();

      var stringResult = await response.Content.ReadAsStringAsync();
      var pages = JsonConvert.DeserializeObject<String[]>(stringResult);

      String xml = "<?xml version=\"1.0\" encoding=\"utf-8\"?>";
      xml += "<sitemapindex xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">";

      foreach (string s in pages)
      {

        xml += "<sitemap>";
        xml += "<loc>" + s + "</loc>";
        //xml += "<lastmod>" + DateTime.Now.ToString("yyyy-MM-dd") + "</lastmod>";
        xml += "</sitemap>";
      }

      xml += "</sitemapindex>";

      return Content(xml, "text/xml");

    }
    catch (HttpRequestException httpRequestException)
    {
      return BadRequest($"Error getting sitemap: {httpRequestException.Message}");
    }
  }

}

I assume I am missing something. Setting a different header?

You can see the result here:

https://badgag.com/sitemap.xml

Thanks in advance :)

Do you have to write the XML yourself? There are libraries that handle these things for you (and make sure they are valid XML schema). Side note, you never want to use a `using` statement for HttpClient. Use a shared static instance, see [here](https://aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/) — maccettura, Aug 22 '17 at 20:29
I don't see any problem, the headers say it's an XML document. Maybe it got cached in Google? — Juan Arias, Aug 22 '17 at 20:51
I cannot use a static file. The site has daily user generated content, so new pages need to be added to the sitemap.xml constantly. @maccettura — ganjan, Aug 22 '17 at 21:49
@JuanArias The error has been like this for a long time, so it seems it still is a problem. — ganjan, Aug 22 '17 at 21:50
@ganjan I never said to use a static file, I said not to _write_ the XML yourself and to let a dedicated library do it for you (since the XML you created is not working) — maccettura, Aug 22 '17 at 23:07

Sake de Vries · Answer 1 · 2017-12-04T15:33:48.967

I found this article about creating a XML sitemap with ASP.Net, this helped me to create a sitemap.ashx file with the correct sitemap layout Google and Bing require.

It basically is using XmlTextWriter to generate the required tags for a sitemap. This example is using HTTPContext to write an XML file. Here is the code from this site:

using System;
using System.Configuration;
using System.Data;
using System.Data.SqlClient;
using System.Text;
using System.Web;
using System.Xml;

namespace Mikesdotnetting
{

    public class Sitemap : IHttpHandler
    {

        public void ProcessRequest(HttpContext context) {
            context.Response.ContentType = "text/xml";
            using (XmlTextWriter writer = new XmlTextWriter(context.Response.OutputStream, Encoding.UTF8)) {
                writer.WriteStartDocument();
                writer.WriteStartElement("urlset");
                writer.WriteAttributeString("xmlns", "http://www.sitemaps.org/schemas/sitemap/0.9");
                writer.WriteStartElement("url");

                string connect = ConfigurationManager.ConnectionStrings["MyConnString"].ConnectionString;
                string url = "http://www.mikesdotnetting.com/";
                using (SqlConnection conn = new SqlConnection(connect)) {
                    using (SqlCommand cmd = new SqlCommand("GetSiteMapContent", conn)) {
                        cmd.CommandType = CommandType.StoredProcedure;
                        conn.Open();
                        using (SqlDataReader rdr = cmd.ExecuteReader()) {
                            // Get the date of the most recent article
                            rdr.Read();
                            writer.WriteElementString("loc", string.Format("{0}Default.aspx", url));
                            writer.WriteElementString("lastmod", string.Format("{0:yyyy-MM-dd}", rdr[0]));
                            writer.WriteElementString("changefreq", "weekly");
                            writer.WriteElementString("priority", "1.0");
                            writer.WriteEndElement();
                            // Move to the Article IDs
                            rdr.NextResult();
                            while (rdr.Read()) {
                                writer.WriteStartElement("url");
                                writer.WriteElementString("loc", string.Format("{0}Article.aspx?ArticleID={1}", url,  rdr[0]));

                                if (rdr[1] != DBNull.Value)
                                writer.WriteElementString("lastmod", string.Format("{0:yyyy-MM-dd}", rdr[1]));
                                writer.WriteElementString("changefreq", "monthly");
                                writer.WriteElementString("priority", "0.5");
                                writer.WriteEndElement();
                            }
                            // Move to the Article Type IDs
                            rdr.NextResult();
                            while (rdr.Read()) {
                            writer.WriteStartElement("url");
                            writer.WriteElementString("loc", string.Format("{0}ArticleTypes.aspx?Type={1}", url, rdr[0]));
                            writer.WriteElementString("priority", "0.5");
                            writer.WriteEndElement();
                            }
                            // Finally move to the Category IDs
                            rdr.NextResult();
                            while (rdr.Read()) {
                                writer.WriteStartElement("url");
                                writer.WriteElementString("loc", string.Format("{0}Category.aspx?Category={1}", url, rdr[0]));
                                writer.WriteElementString("priority", "0.5");
                                writer.WriteEndElement();
                            }
                            writer.WriteEndElement();
                            writer.WriteEndDocument();
                            writer.Flush();
                        }
                        context.Response.End();
                    }
                }
            }
        }

        public bool IsReusable {
            get {
                return false;
            }
        }
    }
}

While the link my anwser the question, a link doesnt make an answer complete. Please provide in your answer all the information and add the link just as a reference. Once the link expires your answer will be useless. — L. Guthardt, Dec 04 '17 at 15:10

score -1 · Answer 2 · answered Aug 23 '17 at 02:02

Try xml linq :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Xml;
using System.Xml.Linq;

namespace ConsoleApplication1
{
    class Program
    {
        const string URL = @"https://badgag.com/sitemap.xml";
        static void Main(string[] args)
        {
            XDocument doc = XDocument.Load(URL);
            XNamespace ns = doc.Root.GetDefaultNamespace();

            string[] locations = doc.Descendants(ns + "loc").Select(x => (string)x).ToArray();
        }
    }
}

I use ASP.net to generate a sitemap.xml, get Google Webmaster says: "Your Sitemap appears to be an HTML page."

2 Answers2

Linked