Messaging Vs Eventing
In the messaging world, we can distinguish two types of message integrations. The first is messaging, which conveys an intent. This means there is an expectation on the receiving system to handle the event in a particular way and is used to transfer control or value. For example, let us pass an order from a webshop to an order system. We expect the receiving system to create handle the order, and make sure the order gets delivered. For these messages, we would use a service like Service Bus in Azure, which handles things like ordered and guaranteed delivery.
The other is eventing, which conveys a fact, indicating something has happened. Events can also be split into two different sections.
The first is series events, which represent a stream of events belonging together. Here we can think of telemetry readings from an IoT device, or logging events from our application. This type of events can be handled in Azure using Event Hubs, which is a streaming event store.
The other type of eventing is discrete events, which we will be discussing in this blog post. Discrete events are independent, and report some type of state change. For example, if we created an invoice for our order, an event could indicate that the invoice has been placed on a Azure Storage Account. Discrete events can be used to start an action, without indicating what type of action is expected. Each system which receives a discrete event will decide for itself what type of action it will take. In Azure, we use Event Grid to work with this type of events.
Azure Event Grid
Announced on August 16th of last year and GA since January 30th of this year, Event Grid is Azure’s fully managed event routing service, which greatly simplifies the development of event-driven solutions. Being a truly serverless service, Event Grid provides near real-time, scalable and reliable eventing, in which you pay only for the number of events being handled. Event Grid integrates with many services in Azure, with more coming each month. And not only does it integrate with Azure services, but you can also work with custom topics and webhooks, allowing you to integrate any application or service inside and outside of Azure.
By creating filters on our event subscriptions, we can route events to different endpoints, as well as deliver them at multiple receivers at once. Azure Event Grid is also robust, allowing events which could not be delivered to be retried with an increasing time interval between retries. Events which still can’t be delivered after 24 hours will be dead lettered, which is a topic in itself, meaning we can trigger events from our dead letter topic as well. Azure Event Grid is highly scalable, allowing for millions of events per second to be passed through, while still maintaining a sub-second end to end latency in the 99th percentile.
How it works
Looking at Azure Event Grid, we can identify several components which are used in conjunction to create an end to end eventing system. Of course, we have the event itself, which conveys the fact of something which has happened. These events are sent out by event sources, which are the systems where the fact took place. The events are sent into a topic, which is an Event Grid endpoint. Azure Event Grid then uses event subscriptions to determine where the event should be sent out to. These event subscriptions can include filters as well, allowing us to filter on the prefix or suffix of the subject which has been set in the event. Finally, the event is sent out to one or multiple event handlers, which are the systems which will pick up the event and take some actions.
Azure Event Grid Scenarios
Now that we have seen how Azure Event Grid work’s, let us have a look where we could leverage its capabilities. Events are a major component in many architectures. So introducing Azure Event Grid scenarios can really help us implement these scenarios in an easy and powerful way.
Serverless architectures
Being a truly serverless service itself, Event Grid easily fits in with many serverless architectures. It allows us to trigger from various event sources and can fan out the events to several event handlers. By using serverless event handlers like Azure Functions and Logic Apps, we can do highly scalable event processing, leveraging the true power of a serverless architecture.
For example, let’s say we have a system which places an invoice into Blob Storage. We can then use Event Grid to trigger off this and have a Function which is subscribed to this type of events. In the function, we encrypt the invoice using PGP (Pretty Good Privacy), which is an encryption method used for encrypting and decrypting data. The encrypted file is then placed onto another blob storage account. Another Event Grid topic gets triggered from this and has a subscription with a Logic App, which sends the encrypted invoice out via an email.
Enterprise integration
Event Grid can be a great asset in an Enterprise Integration scenario. Event Grid provides reliable messaging with retries and dead lettering, so we can be sure no messages are lost. Thanks to its highly scalable throughput we are assured that we can have systems creating events at an extremely high pace, and our integration solution will still keep up. For this scenario, let’s take a Dynamics instance which creates messages and places them on a Service Bus queue. We can then use Event Grid to trigger of this queue and have a subscription which starts a Logic App. The Logic App will then retrieve all messages from the queue, and update the corresponding entities in SAP.
DevOps
We can also integrate many details about our Azure services through Event Grid. For example when resources are being created or modified. This allows our DevOps team to automate their work, for example by calling a workbook whenever a virtual machine is created, automatically adding it to a virtual network and set up security.
IoT
Event Grid also integrates with Azure IoT Hub, allowing us to be notified whenever devices have been created or deleted. This allows us to, for example, dispatch an engineer when a new device has come online. The engineer can then finish setting up the device and make sure all configuration has been completed.
Cloud Events
As mentioned, with Event Grid we can also integrate with resources outside of Azure. We can use custom topics as event sources, which allow us to send events in the Event Grid schema to Event Grid. We can also send out events with subscriptions to webhooks, allowing us to send events to any system supporting these. However, it does not stop there, as Event Grid is one of the first services to support the CloudEvents standard. CloudEvents is a standard which allows working with events across platforms and gives us a specification for describing event data in a common way. This way we can have any platform or application working with events, to implement a common format, allowing easy integration and interoperability. With names like Azure, AWS, Google, IBM and many more backing this standard. We can expect this to become the de facto standard for exchanging events across providers.
Sample
Now that we know what Event Grid all about is, and how we can use it in our own scenarios, let’s have a look at implementing a sample solution with it. It is easy to get started with Event Grid, as the Azure portal provides us with templates in various locations. In this sample, we use one of these templates to automatically set a time to live on a blob. Once it has been created so that it is automatically deleted after a certain amount of time.
Start by opening the Azure portal and create a storage account.
Once the storage account has been created, go to the Events blade. In this blade, we will see several templates to quickly get started with Event Grid. We will choose the Logic Apps tab and select the When a new blob is uploaded template.
This will create a new Logic App, now specify the connections to the Blob Storage and Event Grid. Click on the Create button for the Blob Storage connector, specify a name, and select the account we just created.
Next, click the Sign in button for Event Grid, and sign in to your account. Click Continue once done. The Logic App will now be populated with the flow, which will wait for 30 days to delete the file which triggered the run. You can, of course, adjust any action in the Logic App to your need if you want to.
Save the Logic App and navigate to the Event Grid Subscriptions service in the portal. Make sure the filter is set correctly, and you should see the subscription you just created. The Event Grid Topic is set to the storage account, and the endpoint is set to the Logic App.
To test the integration, navigate back to the Blob Storage we just created. You can add, open a new container, and upload a file.
Navigate to the Logic Apps service and open the Logic App. You will notice that a Logic App is now running. This Logic App will continue to run until the time to live has passed. Later, the file will be deleted from the Blob Storage.
Conclusion
These Azure Event Grid scenarios show getting started with Event Grid is really easy. This allows us to quickly start building a truly serverless event routing service in Azure. This is a highly scalable and reliable architecture, with near real-time communication. We use events in many of our integration and serverless architectures. Event Grid makes it very easy to integrate these into our solutions. Make sure to try out Event Grid by yourself to see how fast you can get started with events.
