Do you have a requirement to resubmit your Service Bus message to a Service Bus Queue or Topic with the property “RequiredDuplicateDetection” enabled? Do you need to perform resubmission within the duplication detection time frame window overcoming the message loss? Do it at ease with Turbo360!
Duplicate Detection
Enabling duplicate detection helps keep track of the application-controlled MessageId of all messages sent into a Queue or Topic during a specified time window. If any new message is sent carrying a MessageId that has already been logged during the time window, the message is reported as accepted (the send operation succeeds), but the newly sent message is instantly ignored and dropped. No other parts of the message other than the MessageId are considered.
Duplicate Detection History Time Window
The duplicate detection history time window will specify how long the message-id values for the received messages will be retained to check for a duplicate message when duplicate detection is enabled. By default, it has a value of 10 minutes and a maximum allowed value of 7 days.
The following code will create a Queue with the property duplicate detection enabled and the duplicate detection history time window set to one hour.
Queue Description orderQueueDescription =
new Queue Description("orderqueue")
{
RequiresDuplicateDetection = true,
DuplicateDetectionHistoryTimeWindow = new TimeSpan (1, 0, 0)
};
namespaceMgr.CreateQueue(orderQueueDescription);
Turbo360 can facilitate Queue property definition at the time of the creation
Configure Duplicate Message Detection
Duplicate message detection on a Queue or Topic is configured when the messaging entity is created by specifying properties in the appropriate description class. Once the messaging entity has been created these properties cannot be modified.
Duplicate Detection on Partitioned Entities
With this understanding of the property ‘Duplicate detection’ let us also understand how the Service Bus Queue or Topic behaves when the properties duplicate detection and partition are enabled.
Each partitioned Queue or Topic consists of multiple fragments. Each fragment is stored in a different messaging store and handled by a different message broker.
When a message is sent to a partitioned Queue or Topic, Service Bus assigns the message to one of the fragments. The selection is done randomly by Service Bus or by using a partition key that the sender can specify.
If the Queue or Topic has the QueueDescription.RequiresDuplicateDetection property set to true and the BrokeredMessage.SessionId or BrokeredMessage.PartitionKey properties are not set, then the BrokeredMessage.MessageId property serves as the partition key. (The Microsoft .NET and AMQP libraries automatically assign a message ID if the sending application does not.) In this case, all copies of the same message are handled by the same message broker. This enables Service Bus to detect and eliminate duplicate messages. If the QueueDescription.RequiresDuplicateDetection property is not set to true, Service Bus does not consider the MessageId property as a partition key.
What Happens When Duplicate Detection is Enabled?
When duplicate detection is enabled, Azure Service Bus will start identifying the duplicate messages. This period can be configured to range from only a few minutes to days. If a duplicate message is sent to the Service Bus, the service will automatically ignore the message.
We have a business scenario involving message resubmission to an Azure Service Bus entity with ‘Requires Duplicate Detection’ property enabled, and the requirement is to prevent message loss during resubmission. Azure Portal does not help to resubmit the message to Azure Service Bus when duplicate detection is enabled during a specified time window. But real-time business scenario would necessitate these messages to be resubmitted. Turbo360 understands the requirement and comes up with the capability to enable resubmitting the message to Azure Service Bus with duplicate detection enabled, without message loss.
When the message is sent to an Azure Service Bus Queue or Topic with the property duplicate detection being enabled, during a specified time window, then the received message gets lost if message-id is found to be duplicate. If every message coming in hold a unique message-id, the Azure Service Bus entity will accept the messages hence the message loss can be overcome.
Resubmit Your Message to Azure Service Bus Through Turbo360.
The messages can be resubmitted without any message loss during:
- Resubmission
- Repair and resubmission
- Automated active Message Processing
- Automated Dead-Letter Message Processing
Resubmission
When one or more messages are being resubmitted to a Service Bus Queue or Topic with the duplicate detection enabled, Turbo360 proactively informs the user about the potential message loss and provides a solution to overcome the same. In Turbo360 the message-id can either be regenerated to a unique GUID or edited to ensure it is unique before proceeding with the resubmission.
Active Message Processing
Turbo360 automated activities are also equipped to facilitate resubmission to duplicate detection enabled Service Bus Queues or Topics.
When active messages are resubmitted through automated activities to the Azure Service Bus Queue or Topic which has the property of duplicate detection enabled, the message-id can be replaced with the random GUID. Thus, message loss is prevented.
Dead-Letter Message Processing
Dead-letter Queue or Topic is to hold messages that cannot be delivered to any receiver, or messages that could not be processed.
When dead letter messages are resubmitted to the Azure Service Bus Queue or Topic which has the property of duplicate detection enabled the message-id can be replaced with the random GUID or edited to be unique. Thus, message loss is prevented.
Turbo360 active and dead letter message processing is equipped to deal with Service Bus entities with specific properties like duplicate detection, session, partition and express. Use Turbo360 to better manage and monitor your Azure Service Bus Queues and Topics at the application context.
Check out the blog on Azure Service Bus delete message to understand how to resubmit and delete messages in Azure Service Bus Messaging.
