Session-Enabled Message Processing in Azure Service Bus

This blog will brief on handling messages in session enabled Azure Service Bus Entities. To start with, let us have a brief introduction on Azure Service Bus. Azure Service Bus is a multi-tenant and fully managed cloud messaging service that is used to send information between applications and services. The asynchronous nature of operation offers flexible, brokered messaging, along with structured first-in, first-out messaging and publish/subscribe capabilities.

What is Azure Service Bus Session?

Azure Service Bus sessions offer combined and ordered handling of unbounded sequences of related messages i.e, it provides a way to group set of related messages. It also provides concurrent de-multiplexing of interleaved message streams while preserving and guaranteeing ordered delivery. A session in fact acts as a sub-queue storing messages from different sessions. Some of the common scenarios for using session are:

Grouping of huge message streams into related message streams
Identifying messages from a specific client

How the Session Works?

Any application or client sending messages to a Service Bus Queue or Topic can create a session by setting the ‘SessionId’ property to any value which uniquely identifies that session. On a session enabled Service Bus Queues and Topic Subscriptions, sessions come into play when there is at least one message with a specific ‘SessionId’. Once a session is created, there is no defined time for the session expiry or disappears. A message can be received for a session today and the next message a year later, if the SessionId matches with the session.

With this introduction, let us hop into creating session enabled entities, sending and receiving messages from it.

Creating Session Enabled Entities

Session Enabled entities can be created from the Azure portal by setting the ‘Enable sessions’ flag. This is applicable to both Queues and Topic Subscriptions.

The session enabled Azure Service Bus Queue can be created by setting the ‘RequireSession’ property to true, using WindowsAzure.ServiceBus library in .Net client.

QueueDescription _azureQueue = new QueueDescription("session-enabled-queue")
 {
        RequiresSession = true
  };

NamespaceManager namespaceManager = NamespaceManager.CreateFromConnectionString(“connectionString”);

var queue = await namespaceManager.CreateQueueAsync(_azureQueue);

Createing Azure Service Bus Queue

Service Bus Basic tier doesn’t support creating session enabled entities. Standard and Premium tiers support sessions.

Sending Messages to Session Enabled Entities

Sending messages to session enabled Service Bus entities is little different from entities in which session is not enabled. It is mandatory to set ‘SessionId’ property in every message that is being sent. Messages without ‘SessionId’ cannot be sent to Session enabled Queues. However, any message sent without ‘SessionId’ to a session enabled Topic will be forwarded only to non-session enabled subscriptions, session enabled subscriptions will not receive them.

MessagingFactory messagingFactory = MessagingFactory.CreateFromConnectionString(“connectionString”);
QueueClient client = messagingFactory.CreateQueueClient(“queueName”);
var brokeredMessage = new BrokeredMessage(new MemoryStream(Encoding.UTF8.GetBytes(“MessageContent”)));

brokeredMessage.MessageId = Guid.NewGuid().ToString();
brokeredMessage.SessionId = Guid.NewGuid().ToString();
await client.SendAsync(brokeredMessage);

Receiving Messages from Session Enabled Entities

Like sending messages, receiving the messages from session enabled Service Bus entities is also different.

MessagingFactory messagingFactory = MessagingFactory.CreateFromConnectionString(“connectionString”);
var queueClient = messagingFactory.CreateQueueClient(“queueName”);
var sessionClient = await queueClient.AcceptMessageSessionAsync(“sessionId”);
BrokeredMessage _message = await sessionClient.ReceiveAsync();
await sessionClient.CloseAsync();

A Receiver to receive session enabled messages is created by the client, accepting a ‘sessionId’ as a parameter. In the reactive call back model, it registers a session handler. When the Session Message Receiver object is accepted and while it is held by a client, that client holds an exclusive lock on all messages with the session’s ‘SessionId’ that exist in the Queue or Subscription, and on all messages that will still arrive with the same ‘SessionId’ while the session is held.

Receiving messages without setting ‘SessionId’ using AcceptMessageSessionAsync() will receive the first message in the Queue or Topic Subscription and will initialize the session receiver with the SessionId of that message.

The lock on the session is released when ‘Close()’ is called on the session client, or when the application is unable to perform the close operation. The lock on the session should be closed as soon as it no longer needs it or does not expect any further messages.

Session State

The session state of a message is an opaque binary object that can hold data of the size of one message, which is 256 KB in case of Service Bus Standard, and 1 MB in case of Service Bus Premium. The processing state corresponding to a specific session can be stored inside the session state. The recorded processing state corresponding to that session becomes instantly available when the session is acquired by a new processor in case of any unexpected failures in the previous processor processing that session.

All existing sessions in a Queue or Topic Subscription can be enumerated with GetMessageSessions() on the QueueClient and SubscriptionClient using WindowsAzure.ServiceBus library in .Net client. The session state remains if it is not cleared up, even if all the messages in a session are consumed.

The session state held in a Queue or in a Topic subscription counts towards that entity’s storage quota. It is recommended to clean up the retained state, after the application is finished with a session, to avoid external management cost.

Concurrent Sessions

When multiple concurrent receivers start receiving messages from the Queue, the messages belonging to a specific session are dispatched to the specific receiver that is currently holding a lock on that session.

With that operation, huge message stream residing in a Queue or Subscription is de-multiplexed to different receivers and those receivers can be on different machines, since the lock management happens service-side, inside Service Bus.

A receiver cannot have two messages concurrently in flight, but the messages must be processed in the order. A new message can only be obtained when the prior message has been completed or dead-lettered. Abandoning a message causes the same message to be served again with the next receive operation.

Sessions with Partitioned Entities

In a partitioned entity, if a message has the ‘SessionId’ property set, then the ‘SessionId’ is used as a partition key. By this way, all messages belonging to the same session are handled by the same message broker.

To send a transactional message to a session enabled Queue or Topic, the message must have the ‘SessionId’ property set. If the ‘PartitionKey’ property is specified as well, it must be identical to the ‘SessionId’ property. If both the ‘Partitionkey’ and ‘SessionId’ differ, then it turns out to be an invalid operation exception.

Unlike non-partitioned Queues or Topics, it is not possible to use a single transaction to send multiple messages to different sessions. If attempted, it will return an invalid operation exception. Read more about Azure Service Bus partitioned entities here.

Scenario where Session Enabled Entities can be Used

Consider a scenario, where 3 different clients are sending messages to Service Bus Queue concurrently (i.e. say 1 message from 1^st client, then 2 messages from 2^nd client, 1 message from 3^rd client and again 2 messages from 1^st client) with a unique Identifier corresponding to each client and at the receiving end, interleaved messages from each client should be received by different receivers.

Azure Service Bus Queue

In this scenario, the session enabled Queues are of great help. The client application, sending the message must set a unique identifier as a ‘SessionId’ in each message that is being sent. Now, the receiving application should create Session Receiver using the respective ‘SessionId’ and receive the set of messages related to the specific client. By this way, all the messages corresponding to a specific client can be received by a single receiver.

Handling Session Enabled Entities using Turbo360

Turbo360 is a 3^rd party application to manage and monitor Azure Serverless components. Turbo360 offers capabilities such as creating Session enabled entities, sending and receiving messages with ease.

Session enabled Azure Service Bus Queues

Summary

This blog has briefed on working and handling messages in Session enabled entities. To wrap up, the session provides a way to group set of related messages. It also concurrently de-multiplexes interleaved message streams while preserving and guaranteeing ordered delivery which can be used at various scenarios as discussed above.