How to Make Google Crawl Your Website Faster
By Michael Benninger
If you manage a blog, e-commerce store, or other type of site that posts frequent updates, you’d likely love for your fresh content to be found by readers as quickly as possible. But since a significant segment of your site’s traffic probably comes from search engines—namely Google—your success, to some extent, depends upon how frequently your site is scanned for new content. At this point, you might be wondering, “How do I update Google Search results for my website?” And while that’s a totally reasonable question, the answer is a bit obscured.
Get to Know Googlebot
There are many known and unknown elements as far as Google’s Search algorithm is concerned. The tech giant itself doesn’t divulge the exact processes by which it scours and indexes the Web, but a few factors have been made public. For instance, it’s a fact that Google regularly administers a program called Googlebot to search the entirety of the open Web. Googlebot is a kind of spider that crawls across the Internet, absorbing content from billions of Web pages. But how often does Google update its results?
How Googlebot Works
This spider performs two separate kinds of searches: a monthly “deep crawl” that aims to dive into even the furthest reaches of the Web, and more-frequent “fresh crawls” that mainly focus on newer content appearing online. And after Googlebot grabs all of that information, algorithms analyze and index it in Google Search. Now you may be asking, “How often does Google crawl my site?”
Are you a Wix user? Find out why Wix websites were removed from Google’s index recently.
Controlling Google’s Crawl
When Google crawls your sites is not easy to predict. You can’t directly control how frequently Google scans your site, but you can do certain things to make your online presence more attractive in Googlebot’s eyes. Here’s a list of those tactics:
Establish accounts on Google Search Console (formerly Google Webmaster Tools) and Google Analytics. This is a great way to get Googlebot’s attention and the Search Console has tons of great features.
Create a sitemap and submit it to Google. Let them know about the new pages on your site. There are plenty of freely available sitemap generators online.
Keep adding content. Start a blog if you don’t have one, and add new articles at least three times a week.
Adjust your crawl rate. Look for this setting in Google Search Console. It controls how much time Googlebot spends searching your site, not how often it happens.
Make your site speedier. Check your page load time and remove or modify any elements—like oversized images or large PDFs—that are slowing your site down.
Remove any duplicate content. Make sure all of your pages have unique title tags, meta tags, and of course, content.
Share links on social media. Googlebot arrives on sites via links, so the more links to your content, the more likely you are to be found by Googlebot.
These quick tips are by no means the end-all and be-all as far as Googlebot is concerned, but they’re good guidelines to keep in mind. Abide by them, and your content should reach readers’ eyes in no time.
Now that you have an understanding as to what Googlebot is and how it works, put your knowledge to good use and get found more easily.