-1

This is my script code:

<script type="text/javascript" src="//example.com/js/infolinks_main.js"></script>

I want to make crawler not to follow or index example.com/js/infolinks_main.js.

How can I do this task? I have robots.txt in my root, but that URL is an external URL.

NB: I do not want to use iframe.

unor
  • 92,415
  • 26
  • 211
  • 360
Asif Iqbal
  • 1,132
  • 1
  • 10
  • 23

2 Answers2

1

The script element can’t have a rel attribute, so nofollow can’t be used. Even if it could be used, note that nofollow is not about disallowing bots to crawl/index the URL.

To disallow crawling the script, you have to use robots.txt:

User-agent: *
Disallow: /js/infolinks_main.js

Or if you want to disallow crawling of all your scripts:

User-agent: *
Disallow: /js/

You have to use the robots.txt file of the host where the scripts are hosted. It doesn’t necessarily have to be the host where your HTML documents are hosted.

(Note that this doesn’t disallow indexing the script. If you want to disallow indexing, you can use the X-Robots-Tag header with a noindex value, but then you have to allow crawling. As scripts are typically not indexed by general-purpose search engines, you probably want to prevent crawling, not indexing.)

unor
  • 92,415
  • 26
  • 211
  • 360
  • What about moving the script to another html file and adding nofollow to its header. Then embedding the file on an iframe? Would that make all links on the external html nofollow? – Michael Rogers Feb 27 '19 at 00:33
  • @MichaelRogers: This would affect the links in the `iframe` document. But it has nothing to do with the `script` in this `iframe` document. – unor Feb 27 '19 at 14:20
  • Yes i realized later and disallowed it on robots. it doesn't render to google, i hope they don't get mad for it. They are so strict with penalties. – Michael Rogers Feb 27 '19 at 18:31
0

rel=nofollow only applies to hyperlinks, there's no point in adding it to any other kind of element.

If your scripts create links, you could edit the script to ensure that they do so with a rel=nofollow attribute; but given that the google bot does not execute scripts when reading a page, there's no real point in doing so.

Gaurav kakkar
  • 1
  • 7
  • It is correct that the bot doesn't execute script but he reads them too. So if there a some text that the bot identify as a url he will try to index them. The only way of forbid him this is to use the robots.txt. – Alexander Behling Mar 06 '19 at 13:18