A Real-World Scenario with Azure Serverless Application Challenges Solved by Turbo360

What is Serverless?

Serverless computing is the abstraction of servers, infrastructure, and operating systems. When you build serverless apps you do not need to provision and manage any servers, so you can take your mind off infrastructure concerns. Azure offers a lot of serverless services for organizations to build business orchestrations solving critical operations.

This blog takes up for discussion, one such Order Processing orchestration in an E-commerce domain. Further, it analyses the challenges in managing and monitoring that serverless application and how Turbo360 turns out to be a solution.

Order Processing Azure serverless application

Northwind, an eCommerce company needs to process the orders placed by its customers. Following are the business requirements to be achieved in the serverless orchestration which solves a specific part of the Order Processing application to manage orders for books:

1. Filter orders for ‘Book’ and execute
2. Segregate orders for hard copy and soft copy
3. Retrieve the book price from the Database
4. Generate Invoice
5. Notify Customer on order confirmation

Below is the Order processing system built using Azure Serverless services

1. Service Bus Queues and Topics to handle order details as messages
2. Function App to compute order filtering and invoice
3. Storage Account Table to store pricing information
4. Logic App to define the entire workflow

Below is the workflow

1. Customer places an order in the eCommerce Website
2. Website submits a new order message to the Service Bus Queue
3. Azure Function filters all the orders for the focused item (Books) to push them to a Topic
4. Rest of the other orders will be sent to a Service Bus Queue
5. The Book order will be further be segregated based on order copy type using Topic Subscription Rules
6. The Azure function will retrieve the price of the ordered Book from the storage table and compute the order price
7. An invoice will be created based on the computed price returned by the function
8. The notification will be sent to the customer with Product details and invoice through Microsoft Outlook

Implementation of business orchestration in the Azure portal

In this Orchestration, Azure serverless applications like Azure Logic App, Functions, Service Bus, Storage tables were used. Let’s see how to provision these entities in the Azure portal.

Create the Service Bus Queue – orderprocessingqueue

Prerequisite to create a Service Bus Queue or Topic is a Service Bus namespace. Create a new namespace with a name unique across Azure

To create the Namespace in the Azure portal

• • Click on the + Create a resource, select Integration and then Service Bus.

    

  • Enter a name for the namespace which should be unique across azure. Select Pricing tier, Subscription, Resource Group, and Location. Click Create

    

  • Now that the Service Bus Namespace is created, Inside the Namespace Click + Queue button to create a Queue that handles the order message.
  • Enter the details required to create the queue and click Create

    

  • Orderprocessingqueue is now created. Similarly, the queue with the name productqueue needs to be created to process the order details of products other than the book.

Create the Topic and Topic Subscription

This Topic will have two Subscriptions, to filter messages based on labels – Hard Copy or Soft Copy. These Subscriptions will contain the order messages of the books based on the respective copy.

To create the Topic and Topic Subscription

1. 1. Inside the Service Bus Namespace created, Click + Topic button to create the topic
   2. Here the name of the topic will be orderprocessingtopic, Inside the topic, Click + Subscription to create the Subscription. In this scenario, two topic subscriptions are needed.

Create the Functions – ordervalidator & priceretriever

This function will take care of segregating the messages from the orderprocessingqueue queue. if it is a book, the message is submitted to Topic orderprocessingtopic. In case of all other products, the message is pushed into productqueue. There is another function called priceretriever which retrieves the price of the selected book, computes the final price to send invoice and notification.

To create the Function

• Click on the + Create a resource, select Compute and then Function Apps.
• Give the details as required and click Create. Here in our scenario, the name of this function will be ordervalidator.
• Follow the same steps to create another function named priceretriever

  

• Copy paste the below code to the functions.
  

  Code for codevalidator

   
  using System.Net;
  using Microsoft.AspNetCore.Mvc;
  using Microsoft.Extensions.Primitives;
  using Newtonsoft.Json;
  
  
  public static async Task<bool> Run(HttpRequest req, ILogger log)
  {
  
  
  string requestBody = await new StreamReader(req.Body).ReadToEndAsync();
  dynamic data = JsonConvert.DeserializeObject(requestBody);
  string label = data.Label;
  
  
  return label.Equals("Book")?true:false;
  }
  
  

  Code for priceretriever

  using Microsoft.AspNetCore.Http;
  using Microsoft.AspNetCore.Mvc;
  using Microsoft.Azure.WebJobs;
  using Microsoft.Azure.WebJobs.Extensions.Http;
  using Microsoft.Azure.WebJobs.Host;
  using Microsoft.WindowsAzure.Storage;
  using Microsoft.WindowsAzure.Storage.Table;
  using Newtonsoft.Json;
  using System;
  using System.IO;
   
  namespace ItemPriceFunction
  {
      public static class Function1
      {
          [FunctionName("Function1")]
          public static IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)]HttpRequest req, TraceWriter log)
          {
              log.Info("C# HTTP trigger function processed a request.");
   
              
   
              string requestBody = new StreamReader(req.Body).ReadToEnd();
              dynamic data = JsonConvert.DeserializeObject(requestBody);
              string name = data.Properties.BookName;
   
              // Retrieve the storage account from the connection string.
              CloudStorageAccount storageAccount = CloudStorageAccount.Parse("Connection String");
   
              // Create the table client.
              CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
   
              // Retrieve a reference to the table.
              CloudTable table = tableClient.GetTableReference("orderprocessingtable");
   
              TableOperation insertOperation = TableOperation.Retrieve("HarryPotter","01" );
   
              var tableQueryResult = table.ExecuteAsync(insertOperation).GetAwaiter().GetResult();
   
              string price="";
              foreach (var variable in ((DynamicTableEntity)tableQueryResult.Result).Properties.Values)
              {
                  price=variable.StringValue.ToString();
              }
   
              return new OkObjectResult(price);
          }
      }
  }
  

Create the Storage Table – orderprocessingtable

This Storage Table stores the price of the products in the catalog. The function priceretriever picks up the price of the products in the order and computes the order value.

To create the Storage Table

1. An Azure Storage Account should be created. To create the storage account, Search Storage Account on search and Inside the storage account, Click Add.
2. Enter the details as required. For our scenario, the name the storage account will be orderprocessingsa.

   

3. To create a storage table, Click the storage account orderprocessingsa and select Tables -> + Table to create the table. For our scenario, the name of the storage table will be orderprocessingtable.
4. To Insert the value inside the table, Click Storage Explorer and select Tables -> orderprocessingtable and click +Add to the data.

   

Create the Logic app – OrderProcessiongLogicApp

This Logic app will orchestrate those above-created resources to process the order. This will process the order from the application and validate it and create the notification.

To Create the Logic app,

1. Search Logic Apps in the Azure portal
2. Inside the Logic Apps, Click Add.
3. Enter the details as required. For our scenario, the name the Logic app will be OrderProcessingLogicApp.

Orchestration with Logic App

Out-of-the-box,

1. Logic App is designed with a Service Bus trigger on receiving a new order message
2. The Azure Function segregates the order messages based on the label
3. The condition checks for the book product and puts the message into Service Bus Topic named orderprocessingtopic if it is a book.
4. Orders for all other products are pushed to the Service Bus Queue name productqueue.
5. Subsequently, the function priceretriever gets triggered and retrieves the price from the Storage Table.
6. The processed order information will be sent to invoice generation tool and the customer gets notified through Microsoft outlook on the order confirmation.

This whole process happens in 4 stages as below

1. 1. Order details processing:

Whenever a new order message is received in the orderprocessingqueue, the logic app will be automatically triggered to process the order message. When a new order message received in the orderprocessingqueue it will trigger the Function action. The Function ordervalidator segregates the orders based on the Label of the message. This will return the Boolean value true if the product is book or false if not. The Condition will segregate and send the order message to the respective entities based on the return value of the ordervalidator Function. Below will be the orchestration till the condition.

2. 2. Segregation of orders based on their type:

When the above condition fails, The order message will be sent to productqueue and If that condition passes, It will send the message to the orderprocessingtopic. Here is the orchestration when the condition is passed.

When the Condition fails, the order message will be sent to productqueue,

3. 3. Book Type Validation:

Here When a Segregation condition passes, It will send the order message to topic orderprocessingtopic with the subscriptions HardCopy and SoftCopy.

When the order message is received in any of the topic subscriptions, It will trigger the priceretriever function. This Function will retrieve the price from the Storage Table based on the Book name. Below will be the orchestration part that invokes and handles the priceretriever.

4. Invoice and Notification:
   • For Invoice and Notification, there are a lot of Logic App connectors like Stripe and FreshBooks can be used.
   • Notification channels like outlook, Gmail etc., are also available as Logic App Connectors to send invoice notification to the customer.

Challenges we might face with Azure entities

This orchestration has simplified the order processing of the products, but users might have some challenges while managing and monitoring these resources that participate in the above orchestration. Below will be the challenges that the user might face:

1. Processing Dead Lettered messages on orderprocessingtopic and productqueue
2. Defining topic subscription rules in orderprocessingtopic to filter out the order messages into Hard copy or Soft Copy subscription
3. Monitoring the health status of the entities to generate a consolidated report every hour
4. Monitoring the entities for any violation against configuration on their defined properties
5. Monitoring the entities for any error
6. Automating dead lettered message processing

There could be more challenges you may face in implementing cloud-native integration solutions

How Turbo360 solves the challenges

Let’s see how Turbo360 solves the challenges mentioned above.

Prerequisite to manage and monitor this Serverless Application in Turbo360 is to represent this as a Composite Application in Turbo360 and to associate all the entities that participate in the orchestration in this Composite Application.

1. 1. Message Processing:

A Service Bus message would get dead lettered due to several reasons. When the Time to Live of the order message exceeds the defined value on the service bus entities or when the header size exceeds the default size, the order message will automatically dead lettered. Message can be dead lettered for custom reasons that do not meet the business requisites.

There is an immense need to process these dead letter messages. However, it is not possible in Azure Portal. Turbo360 comes with the solution to view and process Dead Letter messages with its “Dead Letter Processing” capabilities.

Resubmit the dead letter messages to same or other queues or topics. Dead letter messages can either be retrieved in peek lock mode or in defer mode for processing. When the messages retrieved in the peek lock mode, user can view the properties, message details and Error details and can resubmit or repair & resubmit the copy of the messages.

When the messages retrieved in the defer mode, we can resubmit the copy of the messages to some other service bus entities or we can repair& resubmit the original messages too.

2. 2. Defining the topic subscription rule for filtering the messages:

In the Azure portal, it is not possible to define filter rules for the topic subscription but here is the need to filter out the messages on their label. Turbo360 comes with the solution to define “Rules”. We can define the rules based on the message properties.

In this scenario, There is need to filter out the messages based on the property BookType so the rule can be defined like this “BookType=’SoftCopy‘” and This rule will filter out the messages of the property BookType with the value SoftCopy into the Subscription ‘SoftCopy”. Similar rule to filter Hard Copy should be defined on the other subscription “HardCopy”

3. 3. Monitoring the health status of the entities to generate a consolidated report every hour:

When there is a need to monitor the state of the entities at specific hours, Turbo360 comes with the solution called Status monitor. Associating all the entities in the orchestration to a status monitor and defining the monitoring configuration will generate a consolidated report on status of all the associated entities at specified times in a day.

This monitor will send the report based on the properties defined. User can configure the notification channel on which the stake holders should be kept informed on.

4. 4. Monitoring the entities for any violation against defined properties:

To monitor the entities for violation against expected property configuration beyond certain duration, Turbo360 comes with the solution called “Threshold Monitor”. This threshold monitor will monitor your entities when configured properties violate the desired values for a specified period.

This monitor is based on the properties we define which are same as the properties on the status monitor and the violation persist time. Number of alerts that can be forwarded for the violation can be configured while creating the monitor.

5. 5. Monitoring the entities for any error:

There might be a need to monitor the server errors or user errors so Turbo360 comes with a solution called “Data Monitor”. With this monitor, the entities will be monitored based on the extensive set of properties. In this monitor, User can access the historical monitor reports of the data monitor.

This data monitor will give the consolidated report based on the properties configured and it monitors the associated entities at the frequency configured while creating the monitor.

6. 6. Automating dead lettered message processing:

When there are a hundred dead letter messages in orderprocessingqueue and a need to resubmit those messages to some other queue, processing them manually will be more tedious. So Turbo360 comes with a solution called “Activities”.

Activities can automate the active message processing or the dead letter message processing. Activities can be scheduled to auto run. Send messages or sent events activities can simulate test environment for the business orchestration.

Turbo360 clearly solves the challenges in managing and monitoring the Azure Serverless orchestrations.

Conclusion

In this blog, we discussed and arrived at a solution for the questions below:

• How to implement the order processing scenario using Azure serverless components?
• The challenges that the user will encounter in managing and monitoring this application in the Azure portal?
• How to overcome it with Turbo360?

Turbo360 is a one platform tool to operate, manage and monitor Azure Serverless components. It provides efficient tooling that is not and likely to be not available in Azure Portal. Try Turbo360 free for 15 days!