A sitemap is an important and integral part of a website and its overall search engine optimization strategy.
Although it is not mandatory to manual create sitemaps (we’ll talk about it more in this article), it is crucial to make sure that your website has a functioning sitemap.
If you wanted to learn more about sitemaps, this is the perfect post for you. In this article, we are going to discuss the following topics related to sitemaps:
- What is a sitemap?
- Why is a sitemap important, and what are the various benefits of a sitemap?
- What are some of the best practices to optimize a sitemap?
Let’s start at the top.
What is a sitemap?
A sitemap is a document that contains a list of all the pages on your website. This list of pages is maintained in the form of URLs.
A sitemap also contains the metadata for each URL and when it was last updated, in addition to the list of URLs that exist on a website.
Here is how a sitemap may look:
Why is a sitemap important?
You may be wondering, why is a sitemap even important?
Although there are various benefits of a sitemap — which we will discuss next — there is a basic concept you can understand to realize the importance of sitemaps.
One of the primary goals of any website is to get lots of free traffic. This free traffic mostly comes from search engines.
There are over 200 search engine ranking factors that determine the search engine ranking of a web page. However, arguably the biggest factor is the indexability of a page.
If Google can’t find a page on the internet, it sure can’t index it in the search engine results pages (SERPs).
Sitemaps make it easier for Google search crawlers to find a web page and later index it in the SERPs.
Various benefits of a sitemap
Apart from this huge benefit, there are several other reasons to actively maintain a sitemap. A few of the benefits of sitemaps are:
- Increased efficiency for search engine crawlers.
- Easier to get ranked for trending and time-sensitive content — especially for news-related websites that rely on timely content.
- A medium to inform Google of any changes in your website content or structure.
Best practices for optimizing a sitemap
So far, we have discussed the basic concept of a sitemap and the various benefits it offers to website owners, content marketers, and SEO professionals. Now, let’s take a look at some of the best, most recommended practices for optimizing sitemaps.
1. Generate sitemaps automatically
You can either generate a sitemap automatically (via plugins or online tools), or you can do it manually (by copy-pasting each URL on your website into a .txt or .xml file).
It is highly recommended to create sitemaps automatically. Manually creating a sitemap is not only time-consuming, but it is also more prone to errors.
Moreover, a sitemap may require regular updates. Doing it manually — every time you publish a new piece of content, change the structure of your website, or add new pages or categories — is a giant waste of time.
Recently, Google’s Webmaster Trend Analyst, John Mueller, also echoed the same thoughts on Reddit.
A user asked the following question:
“I have a client in a very technical, industrial niche. They sell industrial parts that are similar, but very specific. They have 2 Million+ SKUs in their inventory.
My challenge here is to create a sitemap. Because there are so many pages, I have to create it manually. I have a list of URLs broken up by products and categories that our dev team pulled through API. Since sitemaps can only contain 50,000 URLs, I have 37 excel sheets with 50,000 URLs on them each.
He further continued, “I'm using Screaming Frog to crawl those excel sheets and creating sitemaps from the crawl. The sitemaps are then pushed through FTP and I submit it to GSC. Is this a dumb approach?”
John Mueller answered: “Don't create a sitemap manually like that. You need to get it from the CMS or the underlying database directly.”
CMS plugins, such as the Yoast SEO plugin for WordPress, can be an excellent tool to automatically create sitemaps for free.
You can read the complete Reddit discussion here.
2. Manual submissions
If you want, you can manually submit your sitemap to Google via Google Search Console. In fact, it is recommended that you do that.
- Log in to your Google Search Console account
- Browse to Crawl > Sitemaps
- Click ‘Add Test Sitemap’
- Paste the URL of the sitemap and click Submit.
Tip: It is also recommended to test your sitemap and view the results before clicking the submit button. This may help identify potential errors and issues beforehand.
Just make sure to only manually submit a sitemap if you have made any major modifications. Trying to trick search engines into re-indexing pages won’t work.
3. Avoid ‘noindex’ URLs
There is no point in adding ‘noindex’ URLs to your sitemap.
A sitemap helps search engines identify pages (or URLs) that you want it to find and index in the SERPs. A ‘noindex’ URL is the complete opposite of that.
You don’t want the search engine crawler to find and index the ‘noindex’ URLs. So adding the ‘noindex’ URLs to the sitemap is counter-intuitive.
4. Ignore the ‘priority settings’
It is an outdated practice to have a “priority” column to help search engines identify which pages are the most important to you — so search engine crawlers could prioritize those URLs.
However, it does not work that way.
Googlebot simply ignores any “priority settings,” so it does not matter at all. No point in making any effort there.
5. Keep the file size to a minimum
Before 2016, a sitemap could have a maximum file size of 10 MB. The limit has been increased to 50 MB, but it is still considered good practice to not bloat the file size of your sitemap.
Keep it as low as possible to put less strain on your server.
6. Create multiple sitemaps if you have more than 50,000 URLs
A sitemap can contain a maximum of 50,000 URLs.
But what if your website has more than 50,000 URLs? It is not uncommon for large e-commerce stores with hundreds of product categories and thousands of products and multiple variants of each product.
In that case, you should just create multiple sitemaps.
You can also use multiple sitemaps for managing different sections of your website. With multiple sitemaps, it is also possible to create a sitemap index file — which is basically a sitemap file for sitemaps — and submit it to Google via Search Console.
Watch this two-minute video by John Mueller to learn what to do if your site has over 50,000 URLs.
Conclusion
A sitemap is an important piece of the overall SEO strategy.
It can help search engine crawlers find the right pages on your site and expedite the indexing process. With better crawlability and indexing, it becomes easier to rank higher in the SERPs and keep that organic traffic flowing to your website.
Check if you have an active sitemap or not with our free sitemap test tool
