How to Create/Use Message Queue in Magento 2 • Dev Hooks • devhooks.in

In Magento 2, the message queue is a feature that enables asynchronous communication between different parts of the system. It helps decouple processes and improves performance by allowing tasks to be executed independently in the background.

Configuring custom message queues in Magento 2 involves several steps.

Let's get started!!

Step 1: Create a Module

Create a custom module or use an existing one to define your message queues. You'll need to set up the necessary files and configurations.

Step 2: Configuring the Message Queue

Configuring the message queue topology involves creating the following requires 4 XML files to configure the custom Message Queue in Magento 2.

communication.xml : Defines aspects of the message queue system that all communication types have in common.
queue_consumer.xml : Defines the relationship between an existing queue and its consumer.
queue_topology.xml : Defines the message routing rules and declares queues and exchanges.
queue_publisher.xml : Defines the exchange where a topic is published.

Let's create the required XML files under the app/code/vendor/module/etc/ folder.

1. Create communication.xml file at app/code/Devhooks/HelloWorld/etc/ and add the below code.

saveCopyzoom_out_map<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Communication/etc/communication.xsd">
    <topic name="devhooks.custom.queue.topic.name" request="Magento\Sales\Api\Data\OrderInterface"/>
</config>

In the above code, we've defined the devhooks.custom.queue.topic.name as a topic name. You can replace it with your name. Also, we've defined the data type of the topic in the request attribute Magento\Sales\Api\Data\OrderInterface. You can also pass the string or other interface as per your requirements.

2. Create queue_consumer.xml file at app/code/Devhooks/HelloWorld/etc/ and add the below code.

saveCopyzoom_out_map<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/consumer.xsd">
    <consumer name="devhooks.custom.queue.topic.name" queue="devhooks.custom.queue.topic.name" handler="Devhooks\HelloWorld\Model\Consumer\OrderConsumer::processMessage"/>
</config>

We defined the consumer name and specified the class Devhooks\HelloWorld\Model\Consumer\OrderConsumer and method processMessage that processes the message.

3. Create queue_topology.xml file at app/code/Devhooks/HelloWorld/etc/ and add the below code.

saveCopyzoom_out_map<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/topology.xsd">
    <exchange name="devhooks.custom.queue.topic.name" type="topic" connection="db">
        <binding id="devhookscustomqueuetopicname" topic="devhooks.custom.queue.topic.name" destinationType="queue" destination="devhooks.custom.queue.topic.name"/>
    </exchange>
</config>

We've definds the message routing rules and declares queues and exchanges.

4. Create queue_publisher.xml file at app/code/Devhooks/HelloWorld/etc/ and add the below code.

saveCopyzoom_out_map<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework-message-queue:etc/publisher.xsd">
    <publisher topic="devhooks.custom.queue.topic.name">
        <connection name="db" exchange="devhooks.custom.queue.topic.name" />
    </publisher>
</config>

This file defines the exchange where a topic is published.

Step 3: Create a Handler Class and Define the Method.

As you can see we've defined the handler class and the method on queue_consumer.xml file.

Now let's create OrderConsumer.php file at app/code/Devhooks/HelloWorld/Model/Consumer/ and add the below code.

saveCopyzoom_out_map<?php

namespace Devhooks\HelloWorld\Model\Consumer;

use Magento\Sales\Api\Data\OrderInterface;

class OrderConsumer
{
    
    /**
     * @param OrderInterface $order
     */
    public function processMessage(OrderInterface $order): void
    {
    	// Implement your logic to here
        // This method will be executed when a message is available in the queue
    }
}

Now create message queue consumer proccess is done. In our next step we'll create the publisher class which will process this queue.

Step 4: Create the Publisher

We can publish the queue in controller, model, etc as per your requirement. But in this article will create one model class that will publish our topic and use it on crontab.xml

1. Create OrderPublisher.php file at app/code/Devhooks/HelloWorld/Model/Consumer/ and add the below code.

saveCopyzoom_out_map<?php

namespace Devhooks\HelloWorld\Model\Consumer;

use Magento\Framework\MessageQueue\PublisherInterface ;
use Magento\Sales\Api\Data\OrderInterface;

class OrderPublisher
{
    
    const TOPIC_NAME = 'devhooks.custom.queue.topic.name';

    /**
     * @var \Magento\Framework\MessageQueue\PublisherInterface
     */
    private $publisher;

    /**
     * @param \Magento\Framework\MessageQueue\PublisherInterface $publisher
     */
    public function __construct(PublisherInterface $publisher)
    {
        $this->publisher = $publisher;
    }


    public function execute(OrderInterface $order)
    {
        $this->publisher->publish(self::TOPIC_NAME, $order);
    }
}

As you can see we've used our topic name devhooks.custom.queue.topic.name and used the PublisherInterface call to publish our topic. Now, let's create on cron and use call this class.

2. Create crontab.xml file at app/code/Devhooks/HelloWorld/etc/ and add the below code.

saveCopyzoom_out_map<?xml version="1.0" ?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:module:Magento_Cron:etc/crontab.xsd">
    <group id="default">
        <job instance="Devhooks\HelloWorld\Model\Consumer\OrderPublisher" method="execute" name="devhooks_publish_consumer">
            <config_path>* * * * *</config_path>
        </job>
    </group>
</config>

Step 5: Now, open the Command line in the folder root of Magento and run both commands

saveCopyzoom_out_mapphp bin/magento setup:upgrade
php bin/magento cache:clean

That's it!! You can confirm your topic name on queue table in your database.

Now you just need to run the cron and check the queue_message table in your database.

Also, you can check the queue status on the queue_message_status table in your database. You can see any of the below numbers. Each number states a different status which is mentioned below for your reference..

2 (new)
3 (in progress)
4 (complete)
5 (retry required)
6 (error)
7 (to be deleted)

You can also process Message Queue using the command. This will save you time and you can check the results instantly. All you have to do is run the below commands.

View a list of all consumers

saveCopyzoom_out_mapbin/magento queue:consumers:list

Run the specific queue

saveCopyzoom_out_mapbin/magento queue:consumers:start <queue_name>

Here you can pass the devhooks.custom.queue.topic.name as a queue_name.

Magento supports RabbitMQ. Please check the official doc from this link to know more about Message queues.

We have successfully created a custom Message Queue in Magento 2, we hope this article helped you to know How to Create a Custom Message Queue in Magento 2.

Bookmark it for your future reference. Do comment below if you have any other questions or doubts.

P.S. Do share this post with your team.

Review other articles maybe it'll help you too.