-1

I want to disallow the search link on my site in the robots.txt.

After I click the search submit button, my URL will look like: 

example.com/searching?k=something

How can I write this URL address into my robots.txt file?

My robots.txt file looks like this:

User-agent: *
Disallow: /admin_folder
Sitemap: https://www.domain-address.com/sitemap.xml
Host: www.domain-address.hu

function search_k()
{
    if($.trim($('#country_id').val()) != "" )
    {
        //document.location = http_host+'/kereses.php?k='+$('#country_id').val();
        var KeyWord = encodeURIComponent($('#country_id').val()).replace(/%20/g, '+');
        document.location = http_host+'/searching?k='+KeyWord;
    }
    return false;
}
unor
  • 92,415
  • 26
  • 211
  • 360
Parkolo11
  • 93
  • 3
  • 13
  • Why don't just use asterisk? `Disallow /searchig/*` – vatavale Aug 15 '17 at 11:46
  • You can also disable `query part`. See https://stackoverflow.com/questions/1495363/how-to-disallow-all-dynamic-urls-robots-txt?rq=1 – vatavale Aug 15 '17 at 11:57
  • What did you try? As you already have a robots.txt, you understand how it works, I assume? (Note that I removed the question asking whether you should disallow these URLs -- this would be a SEO question, which is off-topic here. It’s on-topic on [webmasters.se], but it’s likely already asked there). – unor Aug 15 '17 at 22:37

1 Answers1

0

If you want to disallow compliant robots from crawling all URLs on your site starting with /searching, then just add another exclusion to your robots.txt file after your other Disallow commands:

Disallow: /searching

For more details on the format, read more about the de-facto standard that most crawlers adhere to.