How to hide azure vm technical url from search engines?

Question

I have a project in Azure virtual machine with "myproject.cloudapp.net" techical address and "myproject.com" domain. Search engines have indexed technical address(myproject.cloudapp.net) and my project in search results with technical address now.

How to hide azure vm address(*.cloudapp.net) from search engines?

As suggested, request removal, but also.. add a robots.txt (http://www.robotstxt.org/robotstxt.html) to the root with the necessary markup to tell the bots not to index. Failing that put basic auth on the site to stop crawlers from being able to access it. — Simon W, Aug 19 '14 at 11:06

score 0 · Answer 1 · answered Aug 19 '14 at 10:05

0

You don't hide *.cloudapp.net because you don't own it. You should place a robots.txt file on your root folder and set it to nofollow. This is documented well online so I wouldn't repeat it but that is how to stop search engines crawling your site and indexing it.

To remove yourself from them you will need to request it. Each search engine has a well documented process for this.

answered Aug 19 '14 at 10:05

Steve Newton

1,046
1
11
28

robots.txt will also be available on `mydamain.com`, unless you do some domain detection/redirection on the file itself. – trailmax Aug 19 '14 at 22:24

score 0 · Accepted Answer · edited May 23 '17 at 10:09

I presume you have a web-application. You'll have to create a redirecting rule which checks what domain been requested and if it matches *.cloudapp.net, then do a permanent redirect to mydomain.com.

In web.config something like this answer:

<rewrite>
  <rules>
    <rule name="Redirect to www" stopProcessing="true">
      <match url="(.*)" />
      <conditions trackAllCaptures="false">
        <add input="{HTTP_HOST}" pattern="^.*.cloudapp.net$" />
      </conditions>
      <action type="Redirect" 
        url="{MapProtocol:{HTTPS}}://www.domain.com/{R:1}" />
    </rule>
  </rules>
  <rewriteMaps>
    <rewriteMap name="MapProtocol">
      <add key="on" value="https" />
      <add key="off" value="http" />
    </rewriteMap>
  </rewriteMaps>
</rewrite>

Disclaimer: I have not tested this redirect in web.config. My applications are running Asp.Net MVC and I do redirects by adding a global filter which does a bit more checks.

How to hide azure vm technical url from search engines?

2 Answers2