Sitemap Files – Benefits, Generation and Use

An XML sitemap is a file that lists the pages on your website that you want search engines to know about. It is one of the simplest things you can put in place, and one of the few SEO tasks that is genuinely finished once it is done properly.

What are sitemap files and what are the benefits

A sitemap is an XML file, usually at a predictable address such as /sitemap.xml, listing the URLs on your site along with optional information about each: when it was last modified, and sometimes how often it changes.

It is worth being clear about what a sitemap does and does not do.

It does help search engines find your pages. Search engines primarily discover pages by following links. Any page that is not linked to from somewhere they already know about is difficult for them to find, and a sitemap gives them the list directly.

It does help with large or deep sites. On a shop with thousands of products, some pages are many clicks from the homepage. A sitemap puts them all on an equal footing.

It does help with a new site. A site with few inbound links has little for a crawler to follow, and a sitemap submitted to the search engines is the fastest way to be discovered.

It does not guarantee indexing. Listing a page in a sitemap is a suggestion, not an instruction. Search engines decide independently whether a page is worth including, and thin or duplicate pages will be skipped regardless.

It does not improve rankings by itself. A sitemap affects discovery, not quality. Anyone selling you a sitemap as a ranking service is selling you something else.

How to generate sitemap files

How you generate a sitemap depends on what your site runs on, and in almost every case you should not be creating it by hand.

WordPress generates a sitemap automatically since version 5.5, so a standard installation already has one. If you use an SEO plugin, that plugin will usually generate a more configurable version and disable the built in one. Either is fine, but you should not have two competing sitemaps.

CubeCart generates a sitemap from the admin, covering your categories, products and documents. Regenerate it after any significant catalogue change.

WooCommerce, OpenCart and Magento stores are covered either by the platform itself or by a widely used extension, and in each case the sitemap should include product and category pages.

A hand built or bespoke site can use one of the free online generators, which crawl the site and produce a file for you. This works, but it produces a static file that is out of date the moment you add a page, so it suits sites that rarely change.

Whatever generates it, a good sitemap follows some basic rules.

  • Include only canonical URLs. One entry per page, using the address you actually want indexed, with the correct protocol and with or without www consistently.
  • Exclude anything you do not want indexed. A page marked noindex should not appear in the sitemap, because that is a contradictory instruction. The same goes for redirects, error pages, admin and basket pages, and search result pages.
  • Keep it accurate. A sitemap listing pages that return 404 wastes crawl budget and reduces the weight given to the rest of it.
  • Split large sitemaps. A single sitemap file is limited to 50,000 URLs and 50MB uncompressed. Larger sites use a sitemap index file, which is simply a sitemap listing other sitemaps.
  • Use accurate last modified dates. A file that claims every page changed today, every day, is quickly ignored.

How to use sitemap files

Generating the file is only half of it. Two further steps make it useful.

Reference it in robots.txt. Adding a line reading Sitemap: https://www.example.com/sitemap.xml to your robots.txt tells every search engine where to find it, without you having to register with each one. This is the single most effective thing you can do with a sitemap and it takes one line.

Submit it to the search engines. Google Search Console and Bing Webmaster Tools both let you submit a sitemap directly. This is worth doing not for the submission itself but for the reporting: both will tell you how many of the submitted URLs were actually indexed, and why the rest were not. That report is one of the more useful free diagnostics available for a website.

Once it is set up, check it occasionally. The two things worth watching are whether the sitemap still generates after a platform upgrade, and whether the indexed count is drifting away from the submitted count. A large and growing gap usually means pages are being judged as thin or duplicate, which is a content problem the sitemap cannot fix.

Common mistakes

  • A sitemap left behind after a site move, still listing the old URL structure or the http:// version of every page.
  • Blocking the sitemap in robots.txt by accident, usually as collateral damage from blocking a directory.
  • Including every URL variation: filtered category pages, session parameters and sort orders, which produce large numbers of near duplicate entries.
  • Assuming a sitemap will fix an indexing problem. If pages are not being indexed despite being in the sitemap, the sitemap is not the issue.

If you host with us and are not sure whether your sitemap is correct, ask. It takes a couple of minutes to check and it is a common source of avoidable problems.

Copyright Havenswift Hosting 2007-2026. All rights reserved.