Scenario
A Logistics solution provider needs telematics powered fleet management solution to empower their business and provide them with accurate telematics information on location, tracking, route, fuel level, engine idling, delivery status of goods.
Telematics fleet management solution would offer the following advantages
- Real-time vehicle tracking
- Asset tracking
- Route optimization
- Fuel consumption tracking
- Improved operational efficiency
- Improved fleet safety
- Compliance verification
- Automatic trip logging
- Risky driver identification
- Vehicle maintenance issues alerts
The Fleet Management System
Microsoft Azure is the selected platform provider to implement the above-mentioned fleet management system. Below is the orchestration of the fleet management solution built using Azure Serverless technologies.
Before proceeding further, let us understand every entity in the orchestration in detail,
The telematics device installed in the vehicles will stream telematics data. Thousands of devices across the world will stream telematics data, which is being ingested by Azure Event Hubs. In addition, Azure Event Hubs is designed with a focus on the data point event scenario. In other words, Event Hubs is an “event ingestor” that accepts and stores event data and makes that event data available for fast “pull” retrieval.
This fleet management system requires to handle multiple data streams from different publishers in parallel. Event Hubs support a partitioning model that allows keeping related events together while enabling fast and highly parallelized processing of the individual streams that are multiplexed through the Event Hubs.
The Event Hubs Capture feature automatically writes batches of captured events into the Azure Storage blob containers and enables timely batch-oriented processing of events.
The Event Hubs emits an event to the Event Grid when the capture file is created. These events are not strongly correlated and don’t require processing in batches. Hence, Event Grid is selected to provide a reliable event delivery at massive scale.
Event Grid delivers the event to the Azure Relay which securely exposes the service that runs in the corporate network to the public cloud. Therefore, the actual business logic to process the telematics data in the storage blob container for decision-making analytics resides in the on-premise service.
Turbo360 to Manage and Monitor the Fleet Management System
Fleet Management System can be represented as a Composite Application in Turbo360. The Azure entities participating in the solution can be logically grouped into the Composite Application. Once the entities are associated with the Composite Application, management and monitoring of them become easy.
Event Hubs plays a critical role in the above orchestration. It is important to ensure client devices interacting with Event Hubs are active. In other words, this can be achieved using Turbo360 as the monitoring tool. Analyzing deep into the orchestration, below are the requirements arrived at to ensure the telematic devices on the vehicles interacting with the Event Hubs are active;
- Monitor incoming requests to the Event Hubs to ensure if all vehicles across the world are active
- Monitor Event Hubs partitions to ensure vehicles in every region are active
- Detect any errors in the Event Hubs ensuring the Event Hubs is available for the client devices
- Simulate a real-time environment to check if Event Hubs receives the telematics data
Let us see how Turbo360 can meet all the above requirements.
Pre-requisite
The pre-requisite to manage and monitor Event Hubs in Turbo360 is to associate the Event Hubs in a Composite Application. Create a Composite application to represent the Fleet Management System with all its Azure Serverless entities associated.
Get instant representation to the health of entities that participate in the Composite Application representing the Fleet Management System. The best part here is these entities can be from different Azure Subscriptions, Azure Regions and Resource groups. Get to manage all the entities at one place which is a challenge in the Azure Portal.
Monitor Incoming Requests
Let us assume some numbers for better understanding. Consider there are 2000 vehicles across the world expected to send telematics data once in every 15 mins. If the requirement is to check if all the 2000 vehicles are active, i.e. all devices send the telematics data in the required frequency.
Turbo360 Data Monitor can be configured to track incoming messages every 15 mins. Further, alerts can be triggered if the number of incoming requests in 15 mins duration is less than the number of vehicles. When an alert is received it reveals not all telematic devices in the vehicles are active.
Detect Errors
When our intention is to ensure client telematics devices interact with Event Hubs, it is necessary that the Event Hubs is available. The Event Hubs would be unavailable when it encounters Server Errors, User Errors, and Quota Exceeded Error. Turbo360 Data Monitor can be configured to monitor the above-mentioned metrics and alert whenever an error is detected.
The above 2 cases can be clubbed together in a single Data Monitor to detect the availability of client devices and the Event Hubs. Below should be the configuration to achieve the same. Moreover, the Data Monitor can be configured to send alerts on the supported Notification Channels to keep the stakeholders informed.
The alert would be triggered even if one of the metrics fail to meet the expected configuration ensuring the client telematic devices are active and the Event Hub is available hence the orchestration works as expected.
Monitor Event Hub Partitions
Event Hubs provides message streaming through a partitioned consumer pattern in which each consumer only reads a specific subset, or partition, of the message stream. In other words, This pattern enables the Fleet Management System to handle telematics data from vehicles segregated based on their region. Each partition in this scenario represents a specific region.
Event Hub retains data for a configured retention time that applies across all partitions in the event Hub. Events expire on a time basis; you cannot explicitly delete them. Because partitions are independent and also contain their own sequence of data, they often grow at different rates.
This demands to monitor all the partitions of the Event Hubs on their Size. This can be achieved by using a Threshold monitor. Furthermore, violation persistence duration in the threshold monitor configuration can be defined based on the business scenario.
In this Fleet Management system, the number of partitions in Event Hubs are 32 representing 32 regions. Exclusively for such cases, bulk monitoring configuration for multiple Event Hubs partitions was brought in in Turbo360. With this capability, the user can quickly define monitoring configuration for several Event Hubs partitions at a go.
An important consideration while creating an Event Hub. The number of partitions is specified at creation and must be between 2 and 32. Similarly, the partition count is not changeable, so you should consider long-term scale when setting partition count. You can increase the number of partitions beyond 32 by contacting the Event Hubs team. Refer to this MSDN documentation for more details.
Simulate Telematic Devices
As the orchestration is being built, there could be a need to test it by sending sample requests. Send Event activity can help to simulate the real-time environment. This activity can be configured to send events to a specific partition of the Event Hubs in batches, with a specified think time between the batches. Moreover, it is possible to send events in parallel simulating a real-time environment. Also, configure the frequency and schedule this activity to stream real-time data to Event Hubs and test if the orchestration works as expected. Below is a sample activity configuration to send events.
