Does someone know a way to request Google to re-crawl a website? If possible, this shouldn't last months. My site is showing an old title in Google's search results. How can I show it with the correct title and description?
Asked
Active
Viewed 4.5e+01k times
241
-
1@Ωmega - Are you asking about a long list of separate domains, or a long list of URLs on the same domain? If it's the same domain, I'll edit my answer to provide a little more info. – kevinmicke May 30 '14 at 16:01
-
@kevinmicke - In my case, the list contains different subdomains of same domain, but Google Webmaster Tool sees them as different domain names, ...so de facto my question/bounty is regarding list of any URLs, even different domain names. Thanks! – Ωmega May 30 '14 at 16:42
-
2@Ωmega - Yeah, the subdomain thing with that tool is frustrating in some ways, but I can understand why they did it. In that case, I'm pretty sure there's no way that is both easy and gets the links recrawled quickly. That leaves you with: quick(ish) recrawl = use http://www.google.com/addurl/ and answer all the captchas, or easy = just wait until Google recrawls of it's own volition. Depending on how often the content on all your links are regularly updated, it might be that long if you just wait, though that's obviously not an ideal solution. Sorry I can't be more help. – kevinmicke May 30 '14 at 17:11
-
1https://support.google.com/webmasters/answer/6065812?hl=en&ref_topic=6066464 – SepehrM Jun 17 '15 at 11:09
-
Also a duplicate of http://stackoverflow.com/questions/4270294/how-to-force-google-to-re-index-a-page – Dan Dascalescu Dec 08 '15 at 06:09
-
1There is an API for that: https://developers.google.com/webmaster-tools/search-console-api-original/ – user1050755 Sep 16 '17 at 08:02
-
Stack Overflow is a site for programming and development questions. This question appears to be off-topic because it is not about programming or development. See [What topics can I ask about here](http://stackoverflow.com/help/on-topic) in the Help Center. Perhaps [Web Applications Stack Exchange](http://webapps.stackexchange.com/), [Webmaster Stack Exchange](http://webmasters.stackexchange.com/) or [Unix & Linux Stack Exchange](http://unix.stackexchange.com/) would be a better place to ask. – jww Jan 23 '18 at 18:27
5 Answers
456
There are two options. The first (and better) one is using the Fetch as Google option in Webmaster Tools that Mike Flynn commented about. Here are detailed instructions:
- Go to: https://www.google.com/webmasters/tools/ and log in
- If you haven't already, add and verify the site with the "Add a Site" button
- Click on the site name for the one you want to manage
- Click Crawl -> Fetch as Google
- Optional: if you want to do a specific page only, type in the URL
- Click Fetch
- Click Submit to Index
- Select either "URL" or "URL and its direct links"
- Click OK and you're done.
With the option above, as long as every page can be reached from some link on the initial page or a page that it links to, Google should recrawl the whole thing. If you want to explicitly tell it a list of pages to crawl on the domain, you can follow the directions to submit a sitemap.
Your second (and generally slower) option is, as seanbreeden pointed out, submitting here: http://www.google.com/addurl/
Update 2019:
- Login to - Google Search Console
- Add a site and verify it with the available methods.
- After verification from the console, click on URL Inspection.
- In the Search bar on top, enter your website URL or custom URLs for inspection and enter.
- After Inspection, it'll show an option to Request Indexing
- Click on it and GoogleBot will add your website in a Queue for crawling.
Tushar Katyal
- 412
- 5
- 12
kevinmicke
- 5,038
- 2
- 20
- 22
-
6Excellent! This will reindex your site in near-real-time! Damn it, Google, I'm impressed! – Sliq Jun 14 '13 at 20:40
-
4@kevinmicke: What's the turn around time for the re-crawl? I followed your advice 6 days ago, and nothing has been indexed. Thanks. – Steve Apr 12 '15 at 07:46
-
@Steve: It's been a while since I've forced a re-crawl, but I seem to remember it being much faster than that - on the order of minutes or hours instead of days. – kevinmicke Apr 15 '15 at 03:22
-
1FYI there is a quota. I noticed when using the "Crawl only this URL" option I had 500 in a month allowed. – Justin Skiles Oct 02 '15 at 17:11
-
@kevinmicke What if I am using Angular and some of the links are either `a) dynamically generated link: like ng-href={{link}}` or `b) I am binding a click handler to a button as to change state: $state.transitionTo('home'); due to the way my angular state router is configured`? – user3871 Oct 13 '16 at 05:04
-
1@Growler In my experience, dynamic links are found by Google's bots if there's an actual link to them somewhere on your site. An example would be a DB getting updated with new data by a user or back-end script, which then creates a new link on the site to the new data. I don't know for sure if the bots are going to click all of the buttons on your site, but my guess would be that they won't. – kevinmicke Oct 27 '16 at 21:51
-
Awesome. Thought this was mostly for checking how pages appear as GoogleBot – nights Aug 24 '17 at 02:51
-
5
-
@ThomasCheng The correct answer has changed. The answer above is incorrect. I was going to add the correct way to to do this in 2019, but the question is closed. – Jarom Oct 04 '19 at 15:32
30
The usual way is to either resubmit your site in your Google Webmaster Tools or submit it here: http://www.google.com/addurl/
seanbreeden
- 6,104
- 5
- 36
- 45
-
1There's nothing in webmaster tools to recrawl a site (checked July 2012). – mhenry1384 Jul 11 '12 at 14:23
-
2True, you can't "make" Google do anything, but you can go to `Optimization -> Sitemaps` in Google Webmaster Tools and resubmit your sitemap. – seanbreeden Jul 11 '12 at 18:42
-
8Google Webmaster has a new options to "Fetch as Google" to reindex site. – Mike Flynn Nov 17 '12 at 16:57
-
2@MikeFlynn: This new option is for a single page, but not for a whole site (except if you enter every single page one by one) – Martin Thoma Jan 15 '14 at 22:42
-
1@moose There is now an option to "crawl URL and directly linked pages", so it's better than nothing. – Justin Skiles Oct 02 '15 at 17:45
-
-
Yep probably. Back when I answered this (7 years ago, yikes!!) this is how it was done :) – seanbreeden Mar 27 '19 at 17:35
4
Google says that it is unable to control when your site is re-crawled. Regardless, you could also check this post on "forcing rewcrawls", I haven't tried it myself but it's worth a shot if you're desperate.
On another note, I might add that you make sure you have a sitemap.xml up as this will also help with SEO.
Drew Dello Stritto
- 915
- 8
- 10
-
Yes , I have `sitemap.xml` file there http://www.ruby-rails.in/sitemap.xml ,well thanks for the answer – Manish Shrivastava Feb 27 '12 at 14:30
2
As far I know, if you resubmit a sitemap it will trigger and crawler of your site.
Erik Simonic
- 457
- 3
- 13
-
-
If you manually submit a sitemap Google will re-index all the pages. I always do this after big changes on a webpage, and it works. (it worket about 6 mounths ago, the last time I did this.) – Erik Simonic Aug 20 '14 at 13:24
-
1
-
Yes indexing changes over time. +1 for new info regarding Google Crawler. – Erik Simonic May 07 '18 at 06:53
2
Nowadays, the revisiting of a website pretty much depends on its popularity, authority and how often its content changes. Having a sitemap.xml containing all URLs is always better. You can also set the lastmod tag of each URL entries. If you don't abuse it, crawlers will take it into account.
Matt
- 74,352
- 26
- 153
- 180
Jérôme Verstrynge
- 57,710
- 92
- 283
- 453