Index Management in Magento 2

Dec 27, 2018966
Index Management in Magento 2

Indexes are rather important to the whole Magento ecommerce platform, so in this article I am going to explain to you the process of index management in Magento 2. In particular, I will elaborate on the following aspects:

What is index in Magento 2
Default indexes in Magento 2
Terminology of indexation
Updating indexes
Indexation via admin panel
Purpose of the Update Mode
Index commands for Magento 2
Magento 2 Components that Perform Indexing
Index Status Tables in Database
Mechanism of indexation by schedule, Mview
Partial and complete indexation
FAQ: Automatic indexing at a set time

What is index in Magento 2

Indexes are tables in the database that store data in a specific format for Magento to use, instead of formatting the information by itself.

The absence of indexes greatly slows down Magento performance. For example, before displaying a product, it is necessary to calculate its cost taking into account discounts, price of other products (if there is a share for 2 products) and so on. In addition, the Magento database has a complex structure, which means it is necessary to link records from a sufficiently large number of tables to obtain all the data about the product. So, as you could already understand, indexes provide the already formatted data, allowing to skip these operations.

Default indexes in Magento 2

Magento 2 offers a number of the following default indexes:

  • Catalog Product Rule
  • Catalog Rule Product
  • Catalog Search
  • Category Products
  • Customer Grid
  • Design Config Grid
  • Product Categories
  • Product EAV
  • Product Price
  • Stock

However, if you haven’t found an index that will suit your purposes, you can create your own.

Terminology of indexation

The concept of data indexation in Magento 2 contains the following entities:

  • Dictionary – original usual data entered to the system.
  • Index – representation of the original data for optimized reading and searching. It must be updated when the dictionary changes.
  • Indexer – object that creates and updates an index.

Updating indexes

Indexes need to be updated from time to time. This can be done in the following ways:

  • Update by schedule (performed by cron, added to crontab Magento);
  • Update on save (performed when data is saved, for example, after editing the product);
  • Manual update (via console).

The mechanism of updates is shown in this diagram:

Magento 2 indexation diagram

Thus, if the index does not need to be updated immediately (on save) when changing the initial data (dictionary), it means that it has the status Reindex required. And then either this status will change when you perform a scheduled task in cron (by schedule), or manually.

Indexation via admin panel

Information on indexing can be viewed in the admin panel in System> Index Management section (in the drop-down menu under the Tools heading).

Magento 2 indexation

In the Mode column you can see how this indexer is updated, in the Status column – whether this index is relevant or reindex is required and in the Updated column – the date of the last update.

It is also possible to rearrange indexers and manage their order. You can simply add this function to your Magento 2 backend with the help of free extension. Its functionality gives you an opportunity to optimize the indexing process and save time by moving outdates records to the top of the indexer table.

Indexer 240px

Indexer for Magento 2

Optimize the indexing process in your Magento 2 backend.

Free download

Purpose of the Update Mode

Here, in the admin area, we can change Update Mode of the indexers. You need to tick the necessary checkboxes, and then in the Actions dropdown select the required Mode. Click Submit.

Magento 2

What are the index commands for Magento 2?

Manual reindexing cannot be done from the admin panel – you’ll need the console access. Having entered php bin / magento, we can see the following commands and their descriptions in the indexer section:

indexer:info                  Shows allowed Indexers.
indexer:reindex            Reindexes Data
indexer:reset                 Resets indexer status to invalid
indexer:set-mode         Sets index mode type
indexer:show-mode    Shows Index Mode
indexer:status              Shows status of Indexer

These commands request methods from indexer objects and change their properties. With the help of these commands you can work with all indexers at once, or with each separately (with the specific indexer indicated).

indexer:info

Shows allowed indexers’ names and their codes to be used in other console commands (you’ll read further about it). For example, Design Config Grid indexer has the code design_config_grid.

Execute: php bin/magento indexer:info

Magento 2 index

indexer:show-mode

Shows the mode of all indexers.
Execute: php bin/magento indexer:show-mode

Magento 2

Using indexer’s codes to affect only them

On the example of indexer:show-mode let’s see how we can manage separate indexers. You can use the code of indexer (that can be shown by indexer:info command) to see the mode of only one indexer.

Execute: php bin/magento indexer:show-mode customer_grid
You can state here as many indexers as you wish and see information only about them:

Execute: php bin/magento indexer:show-mode customer_grid salesrule_rule
This is applicable to all the other indexer commands.

indexer:set-mode

Using this command you can set the mode.
Execute: php bin/magento indexer:set-mode schedule catalog_category_product catalog_product_category.
This command will change only the category products and product categories indexers to update on schedule.

Execute: php bin/magento indexer:set-mode realtime catalog_category_product
This command will change only the category products indexer to update on save.

Execute: php bin/magento indexer:set-mode realtime
This will change the mode of all your indexers.

indexer:status

Show the status of indexers.
Execute: php bin/magento indexer:status

magento 2 index management

Here you can see which indexers need to be updated (reindexed).

You can also see the status of separate indexers like we showed it above for indexer:show-mode command.

indexer:reindex

This command is that very instrument for manual reindexation that we promised you to talk about.
Execute: php bin/magento indexer:reindex
This command will reindex all the indexers.

You can also reindex separate indexers, stating one or more of them
Execute: php bin/magento indexer:reindex catalog_category_product
Execute: php bin/magento indexer:reindex catalog_category_product customer_grid

indexer:reset

Using indexer:reindex command you can face an unexpected problem, facing this message: “index is locked by another reindex process. Skipping.”
This is how it can look in the console:

Magento 2

In this situation you can use the command indexer:reset to stop any reindexing processes.
Execute: php bin/magento indexer:reset stock
This will unlock the indexer for further reindexing process.

Indexer 240px

Indexer for Magento 2

Optimize the indexing process in your Magento 2 backend.

Free download

Magento 2 Components that Perform Indexing

Magento_Indexer

Implements:

  • indexer declaration
  • indexer running
  • indexer running mode configuration
  • indexer status

Magento/Framework/Mview

Performs indexation by schedule via cron by default every minute and uses triggers.

Index Status Tables in Database

The database contains the following tables with information on the index status:

  • indexer_state
  • mview_state

Mechanism of indexation by schedule, Mview

When there are any changes in the product or another entity in the database, it sets triggers that rewrite data from tables with the ending _cl or if there’s no _cl ending, creates them before filling. (cl – change log).
Here is one example of such a trigger with filling in tables:

All the * _cl tables contain a version_id column. This is the number that indicates the update number. The mview_state table also contains a version_id column. This is the previous version of the update.
Indexation by schedule is carried out every minute. The Mview component checks if the numbers in the version_id column in the * _cl tables have changed relative to the same column in the mview_state table. If changed, the system runs reindexing.

Partial and complete indexation

In this article, we have already covered various aspects of indexation in Magento 2, and the only one left is partial and complete indexation.

Partial indexation

Partial indexation is only updating those entities in the index that have been modified. Partial indexing occurs both with an update by save and with an update by schedule. The products that were changed will be indexed again.

Complete indexation

Complete indexation is a complete update of the whole index. Some extensions or processes of Magento 2 may reset indices. In this case, the following message appears in the admin panel:

Magento 2 complete indexation

In this case, one of the indices may be invalid as well, for example, Customer Grid. For its updating, its full re-indexing is applied.

Full indexing is carried out automatically with the help of cron. If cron is disabled, you can do a full reindex of one or all indexes via the command line. We have already discussed in this article how to do this.

FAQs

How to make Magento 2 index automatically at a set time?

So, we need to change the indexing frequency through cron. By default, indexation is performed every minute. You can see it for yourself by opening the file vendor / magento / module-indexer / etc / crontab.xml.

Partner With Us Let's discuss how to grow your business. Get a Free Quote.
Talk to Igor

We are primarily interested in the <job name = “indexer_reindex_all_invalid”> tag. Five asterisks indicate that all invalid indexes are updated every minute. Asterisks  are given according to this table:

* * * * * command to be executed
| | | | |
| | | | +—– Day of week (0 – 7) (Sunday=0 or 7)
| | | +——- Month (1 – 12)
| | +——— Day of the month (1 – 31)
| +———– Hour (0 – 23)
+————- Minute (0 – 59)

You can also set the numerical values, with the asterisk simply meaning “every minute (hour, day, etc.)”. Therefore, if you want to update crown indexes only once a day at a certain time, you can write, for example, this: 0 4 * * *. This entry will indicate: 0 minutes – 4 hours – every day (4 am every day).

But we cannot make changes to the files that the vendor folder contain. For this you will need to create your own module.

In the folder / app / code create a folder vendor – Belvg, and inside her folder CronIndexer. This folder will be the root of our module. Here we will create a file registration.php with the following contents:

Next, create the etc folder and the module.xml file in it:

And now in the etc folder we will create a crontab.xml file with the following contents:

Now you can install this module by running the bin / magento module command in the console: enable Belvg_CronIndexer, and then bin / magento setup: upgrade.

So this is how we redefined the time; now the reindexing of invalid indexes will be performed not every minute, but once a day.

This is all to the topic of index management in Magento 2; I hope my article was helpful to you. If you have comments or questions, please leave them down below.


Partner With Us Looking for a partner that will help you to grow your business? We are the right company to develop your webstore. Feel free to get in touch with us. We will be happy to discuss your business opportunities and provide you with a Free Quote. Talk to Igor

Post a new comment

top
BelVG Newsletter
Subscribe to our mailing list and get interesting stuff and updates to your email inbox.
Email *