Azure Messaging Patterns with Azure AIS

This blog is the extraction of the final session of Integrate 2020 day -2 which is “Azure Messaging Patterns with Azure AIS” by our Azure MVP – Wagner Silveria

Wagner Silveira spoke about some common Azure Messaging Patterns that can be implemented on Azure Integration services.

First Things, first…

The session started with one of the widely asked questions “What is the difference between Messages and Events?”. In Azure AIS, Messages and Events has a different kind of properties and each can show different behaviours in different scenarios.

Messages

• Messages are transactional, flowing through a long-running transaction or workflow
• Since the message is flowing, it becomes necessary to handle some exceptions
• And the order of the messages also plays a vital role
• The order of the messages plays a vital role is because they can be related to others

Events

• Unlike messages, Events are informational
• And events are mostly used to trigger an action, and thus no exception is to be handled
• Order of the messages (nothing but events) is not a concern
• As an event triggers an action, each event is individually important

In this above illustration, we got a clear understanding for What is message and event but what is a stream? A stream is a series of events over time or collection of events that are mostly used to evaluate or analyse a specific event by itself.

In this above illustration, we got a clear understanding for What is message and event but what is stream? Stream is a series of events over a time or collection of events that are mostly used to evaluate or analyse a specific event by itself.

AIS Messaging Services

Do we understand what a message is? and what an event is? So, now what does AIS support in this. AIS supports both messages and events. Now let us understand this better with Azure Service Bus and Azure Event Grid.

Below is the illustration that shows a teardown comparison of Azure Service bus Azure Event Grid and how these services handle Messages and Events

The above illustration was more about Messages and Events handling services (Service bus and event grid). Now let’s see how other services handle messages.

And there he gave a comparison on how Azure Logic Apps and API Management processing the messages.

Implementing Azure Messaging Patterns in AIS

In this section, he explained about different messaging patterns available for a better message or event handling.

1. Publisher/ Subscriber Pattern
2. Claim check pattern
3. Sequential convoy

Publisher/Subscriber

Consider a scenario where a sender must send a message to the receiver. Normally this would be done with a Messaging engine, which sends the received message from the sender to the receiver.

This is a straightforward approach but what happens when a sender must send the same message to multiple users/receivers.

This can be done in two different ways

1. The sender can bear the effort in connecting to different hosts for sending the same message to different sources. But this method is not efficient enough.
2. So, for better efficiency comes the Publisher/Subscriber method

Note: Publisher/ Subscriber pattern will have a single publisher with Multiple Subscribers data being replicated for individual subscribers

In public/subscriber method the publisher sends a message to the Publisher/Subscriber Channel where the content of the message is duplicated for each subscriber (receiver) and the duplicated messages are now sent to a different subscription.

Now we got to know about Publisher/Subscriber pattern but let us see if it can be implemented into our AIS.

In both Service Bus and Event Grid, we have this by default – Topic Subscription (Service Bus), Event Grid Subscription (Azure Event Grid). The below images describe the same.

Content Based Routing

We can replicate the data with the help of publisher/subscriber pattern and send it to multiple subscribers but what if every subscriber needs their respective data. Can we implement routing in AIS?

Well, it is possible to implement Routing in both Service Bus and Event Grid.

Azure Service Bus Topic Subscription has a filter option where the user can set up the parameters for a message to pass through it.

Azure Event Grid subscription also provides a filter option so that only intended messages flow through each subscription.

Note: For both Azure Service Bus and Azure Event grid, the filters can be set at the subscription level.

INFO: Monitoring your Service Bus topics or Event grid in a subscription level is no more an issue. Use Turbo360 to better monitor your Azure Integration services.

Claim Check Pattern

Both the methods that we saw until now was dealing with the original capability provided by the Azure portal. But what if, the messaging engine couldn’t process the message as the message size is too big for both Service Bus and Event Grid to process.

There comes the Claim check pattern. In this approach, we would need a data repository to store the messages. In this method Instead of sending the actual message, a copy of the message will be sent through the messaging service whereas, the message will be stored in the common Storage by the send through other AIS like Logic Apps or API Management.

As mentioned in the above illustration, both the Azure Service Bus and Event grid can act as message handler to send the copy of a message and API Management & Logic Apps acts as a proxy to put the original message in the common Data repository of Sender and Receiver.

Sequential Convoy

Azure Integration Services are very powerful tools that can satisfy both Publisher/ Subscriber and Claim check pattern but there is another complex pattern which involves a stream of data – Sequential Convoy

In this pattern, there will be a stream of data which needs to be received in an order based on the receiver requirement. This can be achieved by Correlation or sequence Id with which the messages can be retrieved in order very easily

We saw the pattern but how it can be achieved in AIS?

This can be directly achieved by sessions in Azure Service bus and even Azure Logic App can be used to retrieve data in sequence

It’s a Demo Time

In the Demonstration, He took an Azure Logic App which will put the actual message into a Storage blob and send the copy of the message to the Message handling system Azure Service bus

In the other End, there will be receiver logic which will receive the message from the Azure service bus and validate that message with an original message in the Storage Blob. If it is a valid message it will be taken for the further process.

1. Below is the illustration of how the above Claim check pattern can be implemented through the Azure Integration Services

Conclusion

Wagner interestingly covered this session with a lot of illustrations and a great real-world demonstration. As the ending note, Wagner shared some key points and tips for AIS users.

• AIS supports all the integration patterns you are used to
• Understand what the messaging services provide, so you don’t reinvent the wheel
• Remember that the other integration services like Logic Apps and API management can help you compose reusable patterns

Then he concluded the session by answering a few questions from the audience.

Q&A

1. How can you guarantee at least once execution if Service Bus doesn’t have endpoints supporting distributed transactions? – John Brockmeyer

   We can use the peek-lock option that guarantees all the message we received but before you commit it back

2.  When is subscription filter access via the Azure Portal will become available? – John Anderson

   It is already available there. Filtering based on Message properties and SQL filtering is already available there in the Azure portal.

   INFO: Have a better hand on your messages using Turbo360.

3.  How do you handle retries with azure service bus topic-subscriptions? – Rodrigo Groener

   We can re-submit the messages from the client-side. All messages will be there on the Azure Service bus if you are receiving the message in peek-lock. You can any time retry the messages.

   INFO: With Turbo360 without even writing a single line of code, resubmit your messages from a topic subscription to the same or different Queue /Topic or even to an API endpoint. And achieve monitoring on the count of active and dead-lettered messages in the Topic subscription level.