Get Free Audit

How to Create Sitemap in Magento

Oct 18, 2019

5978 Andrey Litvin

How to Create Sitemap in Magento

Sitemaps are structured lists of all URLs on your website with a list of their parameters. The main function of a sitemap is to make it easier for search engines to understand and index the content of your website. A website sitemap is also a useful tool in case you want to ensure that Googlebot is able to see all the content that you have on your website. Fortunately, the majority of CMS systems enable the site owners to generate one.

There are up to six different types of sitemaps, but in this article, we will focus on two sitemap formats that are seen most often and considered to be the most commonly used: HTML and XML.

Table of contents:

What is HTML sitemap?
What is an XML sitemap?
What is the difference between an HTML and XML sitemap?
How to create sitemap XML in Magento?
How to submit sitemap to Google?
How to generate a sitemap for custom model?

What is HTML sitemap?

In simple terms, an HTML sitemap is an outline text version of the site navigation on one page. This type of sitemap is essential for any kind of website, whether it is ecommerce, non-profit, blog or entertainment website. HTML sitemap performs an important function in making it easier for some users to navigate through the website since all the resources they might need are quickly accessible. This is why it is important to create and then keep your sitemap user-friendly.  If it’s easy to navigate, quick to load, and cleanly organized, the visitors of your website will be able to use it without any complications, in case they choose this type of navigation.

According to HTML sitemap best practices, it should have an anchor link connected to every page on your website. It will help the users to navigate through the website, and eventually, it will also make it search engine crawlable. Reading the sitemap, search engines become aware of every page on the site, including any URLs that are not easily discovered through the normal crawling process, constantly run by most of the search engines.

The importance of HTML sitemaps

The greatest HTML sitemap benefit is that it is a great way to have a clear overview of the website and all its pages. This makes sitemap a reliable navigation tool, available any time at the website footer. In most of the cases, providing a proper HTML sitemap will improve the user experience, and thus have an impact on your website’s ranking.

Furthermore, promoting your online presence, this type of sitemap is valuable, because it is the page that search engines will read in order to understand and index your website among the others. Search engine optimization is especially helpful for the websites with dynamic content, when the information on the website is new and does not have many links to it, or when some articles contain a lot of content while being not properly linked.

Igor Dragun

Partner With Us

Let's discuss how to grow your business. Get a Free Quote.
Talk to Igor

What is an XML sitemap?

An XML sitemap is a text file used to detail all URLs on a website. It can include extra data on each URL, including the information addressing its latest update, its priority level, the average frequency changes occur, the URLs relation to the rest of the site, and whether there are any other versions of the same URL in other languages. According to Google documentation, XML sitemaps are useful for websites with large archive or a large volume of media content, although, in our opinion, even if your webstore does not match these characteristics, it will still largely benefit from having a sitemap.

The sitemap protocol format contains XML tags and should:

  • Begin with <urlset> opening tag and end with </urlset> closing tag.
  • Include a <url> entry for URLs as a parent XML tag.
  • Specify the namespace (protocol standard) inside the <urlset> tag.
  • Include a <loc> child entry for <url> parent tags.

Other tags are optional and their support may vary from search engine to search engine. So, before including any of those optional tags, consult the documentation for each search engine.

Why XML sitemaps are important for SEO?

Creating sitemaps is especially useful for SEO. An XML sitemap is designed for search bots, and an updated XML file can be used to ping search engines, alerting them of changes and updates. An XML document lets search engines know that there is fresh new content. Having a well-built XML sitemap substantially increases your chances of having your web-pages crawled and indexed by the search engines, particularly if the navigation system of the website does not work properly or general internal linking strategy doesn’t provide links to all of your pages.

If your website is international or has the same page translated into different languages, the XML sitemap can be used with the purpose of sorting the content for the search engines. The sitemap will be able to signal the crawlers that the page has multiple versions, and they will use this information to make sure that the users will be provided with the correct version of the website, based on their language or location.

What is the difference between an HTML and XML sitemap?

Since the article is dedicated to two sitemaps, it is important to understand clearly the difference between both of them. So, this paragraph is dedicated to the HTML sitemap vs XML sitemap comparison.

HTML and XML sitemaps have different uses and values for search engine optimization. The main difference between HTML and XML sitemaps is that HTML sitemaps are written primarily for the users and needed most to help them to gain a comprehensive understanding of a website, and XML sitemaps are written first of all for search engines.

HTML sitemap is rather a general overview of the website, the pages, and information, which could be useful to the user. While being used mostly by the visitors of a website, it can also assist the search engine ranking.

In a nutshell, XML is primarily created to facilitate the functionality of the search engines, providing them with all the necessary information regarding the structure of the website.

Magento Audit

Magento Audit

Take your online store to the next level with BelVG Magento audit

Visit the page

How to create sitemap XML in Magento?

Below you will find clear step-by-step instructions on how to build a sitemap in Magento back office.

Step #1: navigate to Stores -> Settings -> Configuration.

settings configuration

Step #2: in the Catalog tab, select XML Sitemap.

catalog xml sitemap

Bear in mind that these settings can be performed for Default configuration, as well as for a website or a store view, depending on the task.

Step #3: configure the update frequency and priority for categories, products and CMS pages in the Categories Options, Products Options and CMS Pages Options sections.

categories option

In most cases, the default values in these sections are the preferable ones, but if you need to introduce some custom changes, here are the fields you have to modify.

There are the following values available for Frequency settings:

  • Always
  • Hourly
  • Daily
  • Weekly
  • Monthly
  • Yearly
  • Never

The Frequency values are aimed at search engines crawlers and set up how often the crawler can visit the page. It is recommended to set Always for the pages that change each time one visits them. The Never value should be applied to the archived pages; this way, search engine crawlers will visit them rarely to keep track of unforeseen changes, if any occur. However, the crawlers may ignore the values you set and visit the pages with “Weekly” value more often, then ones with “Hourly” value.

The Priority value is set in the range from 0 to 1, where 1 is the highest priority.

With priority setting, you signify which pages are the most important and which are not that much. Therefore, setting 1 Priority for all the pages is useless, for all the pages will be equally very important. For Priority, the default value is 0,5, and this is a recommended value in most cases.

Also, set up what images you want to the sitemap in Add Images to Sitemap field – None, Base Only or All.

When finished, press Save Config.

Step #4: set up sitemap generation settings.

Expand the Generation Settings section and configure the following:

generation settings

  • Enabled – enables the sitemap generation, so set at Yes.
  • Start Time – the server time at which sitemap generation will be launched. The best time is at night, when the load is minimal,
  • Frequency – how often the sitemap will update. The best way is to set the value equal to the frequency of product adding or changing at your store.
  • Error Email Recipient – the email address that the messages about errors with sitemap generation will be sent to.
  • Error Email Sender – email address that sends the message about sitemap generation errors. The available options are General Contact, Sales representative, Customer support, Custom email.

To set the email addresses for these fields, navigate to Stores -> Configuration -> General -> Store Email Addresses.

store emain addresses

Magento settings allow to define them for the whole website or for a separate store.

Error Email Template – the message template that will be used for sitemap generation notification.

Step #5: define sitemap file limits

Expand the Sitemap File Limits section and fill in the following fields:

sitemap file limits

  • Maximum No of URLs Per File – limit the number of URLs added to the website.
  • Maximum File Size – limit the file size.

Step #6: define the rules for adding the sitemap into robots.txt

Expand the Search Engine Submission Settings section and set Enable Submission field at Yes. This configuration adds the sitemap link into robots.txt file that is used by crawlers for getting the site indexation rules.

search engine submission settings

Step #7: press Save Config and flush configuration cache to enable the changes you made.

Step #8: to set up the sitemap generation, navigate to Marketing -> SEO & Search -> Sitemap.

marketing sitemap

Step #9: press Add Sitemap button in the upper right corner.

add sitemap button

Step #10: enter the Filename and Path values.

new site map

Commonly, sitemap.xml is used as a filename, while the path is set with a forward slash. Bear in mind that the directory, where the sitemap file will be created, must be writable.

Step #11: press Save or Save & Generate button.

If you press Save, the map will be automatically generated at the date and time you set it up. Otherwise, if you press Save & Generate, the sitemap will be generated immediately.

In case your Magento store has a large number of products, then the first option is more preferable, you can schedule the generation for the time when the load on the shop will be minimal.

Igor Dragun

Partner With Us

Let's discuss how to grow your business. Get a Free Quote.
Talk to Igor

How to submit sitemap to Google?

Submitting XML sitemap to Google can be carried out via Google Search Console and does not require special knowledge. Below is a simple step-by-step guide that will carry you through this process.

NB! Before adding a sitemap to Google, one must confirm their rights for the website in Google Search Console.

Step #1: go to Google Search Console tool and enter the website’s address.

Step #2: select Sitemap in the left sidebar.

Step #3: enter XML sitemap URL without the domain address in the corresponding entry line (marked 2 on the screenshot). Commonly, the address is site URL + sitemap.xml, however, to be certain, you may check in the Magento sitemap generation settings.

Step #4: press Submit button.

The sitemap will be added to the web crawler queue. One can track the progress of their sitemap submission in the Submitted sitemap section below.

In case the following warning – “Your sitemap appears to be an HTML page. Please use a supported sitemap format instead” – appears during the sitemap Google submission, then you need to check whether the sitemap URL you submitted at step 3 is correct.

google search console

How to generate a sitemap for custom model?

Adding a URL from the custom model is performed by adding a new ItemProvider into \Magento\Sitemap\Model\ItemProvider\Composite.

Below is the example of adding CMS pages into the sitemap. Although CMS pages are included in the XML sitemap by default, our example demonstrates how to add any other model:

app/code/VendorName/Sitemap/etc/module.xml


app/code/VendorName/Sitemap/etc/di.xml


app/code/VendorName/Sitemap/etc/adminhtml/system.xml


Igor Dragun

Partner With Us

Let's discuss how to grow your business. Get a Free Quote.
Talk to Igor

app/code/VendorName/Sitemap/Model/ItemProvider/ExamplePages.php


app/code/VendorName/Sitemap/Model/ItemProvider/ExamplePagesConfigReader.php


Magento store.belvg.com

Magento 2 Extensions

Take your online store to the next level with BelVG Magento extensions

Visit the store

Conclusion

As you have seen, there are two main sitemaps types – XML and HTML. They serve different functions – one is for the search engines, another is for user experience and search optimization. Following the detailed step-by-step instructions that can be found above, you will be able to create the necessary sitemaps on your own, but you find yourself in need of support, you can always find it here. Turn to BelVG team, and we will solve any problem you may have and provide you with quality Magento development solutions.

Igor Dragun
Partner With Us Looking for a partner to grow your business? We are the right company to bring your webstore to success. Talk to Igor

Post a new comment

BelVG Newsletter
Subscribe to our mailing list and get interesting stuff and updates to your email inbox.
Email *