These days we often hear the term serverless, but what does it mean? According to Wikipedia, serverless is all about cloud computing, where the cloud provider dynamically manages the allocation of our machine resources. You no longer need to think about servers, compute power or availability, as this is all abstracted away. Therefore, you do not require to think about reserving resources upfront. Instead, they scale based on the need at a particular moment. Scaling happens on an event-driven basis, you add or remove compute power based on the load running on the solution. The number of resources used impacts the costs. This means we only pay for the execution time, often on a sub-second level. Consequently, if you don’t use any resource, you don’t pay anything at all.
Serverless options in Azure
Looking at Azure, we can currently identify three genuinely serverless services.
Logic Apps is the serverless workflow, which gives a visual designer and over 200 out of the box connectors. Triggers start a workflow, and actions perform the work, integrating with applications, data, and services.
Azure Functions is the serverless compute offering, triggered by events like timers, HTTP calls, or events from various Azure services. Once a Function triggers, code executes, which you can write in a diverse set of languages including C#, Node.JS, F# and more. Data can be pulled in during the execution of the code, and finally be sent out, using bindings for various services like Azure Storage, Service Bus, Cosmos DB and more.
Finally, there is Event Grid, which is a service for handling discrete events and can interact with services both within Azure and beyond.
Serverless in the integration space
Formerly when working with integration, BizTalk was one of the leading competitors, providing an enterprise-grade integration platform. Since the coming of Azure, hybrid scenarios have become more and more common. Where on-premises systems integrate using BizTalk, and cloud-native integrations using Azure. These days, many of these integrations have moved to the Azure, even when they communicate with on-premises systems. Thanks to increasingly better tools for setting up hybrid architectures with services like the on-premises data gateway, Azure Relay, and virtual network support.
Serverless is one of the big selling points in this shift, as it allows to implement integrations quickly and in an agile manner. Thanks to services like Logic Apps, creating these has never been easier. Planning and procurement of hardware are no longer necessary, meaning implementation can start much faster. Moreover, due to the many out of the box connectors and easy integration with other Azure services, including all the components in the Azure Integration Services, creating solutions can be done much more efficiently. Both development and operations are more straightforward, with out of the box tooling like Application Insights and OMS, and by third parties bringing even more value with products like Turbo360. Integration specialists can now focus on what is essential, adding value for the business, instead of dealing with everything surrounding this.
By bringing these serverless technologies into the integration space, and combining them with Azure’s PaaS services, we can build robust and scalable integration solutions. As a result of out of the box integration points, like connectors for Logic Apps, bindings for Azure Functions and event sources and handlers it’s effortless to integrate with many services, both within and beyond Azure. By smartly architecting solutions, you can harness the power of serverless resulting in less maintenance, high scalability, and lower costs.
Integration patterns using serverless
Those who have been working in integration for a while will know about enterprise integration patterns. They provide guidance on the various challenges encountered in the integration space, and how we can solve these. Although the tooling has changed, these still are beneficial, as they allow us to build our solutions in a consistent and proven way. Here we will have a look at some of the more common patterns. Also, provide architectures on how we can implement these using serverless technologies combined with PaaS in Azure. Of course, there are multiple ways to achieve this, so this is not a definitive guide. Instead, you can view this as guidelines around possible solutions.
File transfer
Even in this modern times, where we see an ever-expanding number of APIs, REST-based communication and new patterns like CQRS and event sourcing, many systems continue to use file-based integrations, especially in the on-premises enterprise integration space. This pattern looks at applications producing files containing information that other applications need to consume.
For this sample, on-premises application A sends out a message to a file system, and then the integration layer stores these into a SQL database. In Azure, there are different ways to work with files, but here the focus will be on Azure Files. This service gives the capability to work with SMB shares hosted in Azure, which any server can mount like it was a network share. Therefore, the enterprise systems working with file integrations don’t need any changes but can continue to work as they always have. Moreover, by leveraging the power of Logic Apps, these existing on-premises systems can effortlessly integrate with other systems, both on-premises and in the cloud.
Logic Apps has an out of the box connector for working with Azure Files. Consequently allowing to pick up files from the share, and incorporating these into its process. In this example sending it to an on-premises SQL database using the on-premises data gateway.
Message translator
The message translator pattern solves the problem of systems communicating in different data formats. A translation component in between these systems transforms messages from one format to another.
This sample gets messages from Dynamics CRM, translates them into another format. And then sends the translated message to Salesforce, implementing data sync between the systems. This example is implemented using a Logic App in combination with an enterprise integration account. Logic Apps has connectors for both Dynamics CRM and Salesforce. This means no development needs to be done to integrate these systems. The enterprise integration account is where you store maps used for transformation. Which can be in XSLT (1.0, 2.0 or 3.0) or Liquid format. In this case, Liquid maps implement the requested conversion, as the Logic Apps connectors both use JSON for communication.
Content-based router
This pattern is used to route messages to one or multiple different destinations based on the contents of the message. Various criteria can make up the routing logic, like existence or contents of a specific field.
The sample used here is that various clients send in messages over HTTP in a JSON format. Depending on the contents of the messages they route either to the European or the American subsidiary. Business rules use fields like which products the client ordered, the location of the client, and the total amount of products. An Azure Function together with a Service Bus topic will be used to implement this. The Function implements various checks by reading the contents of the message and applying business rules against the order lines. Depending on the outcome of these business rules, the Function adds a header to the message. This indicates if the subsidiary should be Europe or the USA, and then sends the message to the topic. The topic contains two subscriptions, one for Europe and one for the USA.
Consequently, one subscription receives the orders for the USA, while the other gets orders for Europe. The subsidiary systems of both locations will each retrieve their messages from their corresponding subscription in the topic.
Scatter-gather
The scatter-gather pattern provides a way to split a message in calls to multiple systems and to aggregate the results back into a single message.
The sample receives an order message in JSON format, get the prices for each item in the message. Finally, push out a quote in PDF format to the email specified in the incoming message. In this case, we will use a combination of services in Azure. The first is Durable Functions, which is an extension on Azure Functions. With Durable Functions, an orchestrator can be built, calling the different services to get the prices of the items. Once the Durable Function has gathered all the responses, it calls Logic App to create a PDF using the Muhimbi PDF connector, and send the result to the email address.
Event-driven consumer
Finally, the event-driven consumer pattern implements a way to consume messages as soon as they become available.
In this final sample, a system places invoices in a Blob Storage container. These invoices then need to be encrypted and afterward be placed in a SharePoint library for archival. For this case, Event Grid monitors the container where you place the invoices. Subsequently, when a file gets created, an Event Grid subscription invokes a Logic App. The Logic App retrieves file contents, and call a Function to use the PGP library to do the encryption. Finally, you can use the SharePoint connector in the Logic App to place the encrypted invoice in SharePoint.
Conclusion
Serverless is a game-changer in the integration scene. It allows for highly scalable, flexible and easy to build integrations, often for a reduced cost. Combine with the power and versatility of Azure PaaS you can create a complete solution. This can integrate into Azure, on-premises, and even to other cloud providers. Serverless is ideal for scenarios where we may have unpredictable workloads due to its scalability. As the costs depend on the usage, we can be sure that costs are under control. Enterprise integration patterns still apply and you can implement it with many options which Azure gives, including in a serverless manner. Correctly architecting these solutions is very important, to make sure you get all the benefits possible, with minimal costs. Get started with serverless in Azure for yourself today by diving into Logic Apps, Event Grid, and Azure Functions.
