An organisation we worked with wanted to implement an invoice automation project which would allow them to use a 3rd party specialist invoicing vendor who would allow the self-service of partners being able to manage and process invoices. Internally the organisation would use SAP to manage invoicing as part of the finance modules with SAP. This was a great opportunity to leverage Azure Integration Services to implement integration between SAP and the 3rd party invoicing specialists. This would involve the creation of purchase orders in SAP being pushed to the invoicing partner and then updates on invoices being sent back to the organisation.
The solution would look a bit like the below diagram.
The business value from this solution comes from the cost savings in automation of the invoice processes and allowing suppliers to self service submitting invoices, paying them, etc via the Invoicing Partners portal.
If we consider some of the design aspects in this scenario they include:
- There is an existing IDOC published to the integration platform for our purchase orders which could be reused
- SAP had existing Hana OData Services which we can use to extract invoice information and Fiori SOAP services which could be used to upload an invoice update
- The partner had a friendly API we can use to integrate with their system
- There were some complex data transformations which would be needed for the data within the interfaces and we can use the Logic App Integration Account to utilise XSLT and Liquid to help us here.
Implementation Overview
Before we get into the details of how we can support these interfaces and how BAM helped us lets consider the implementation of each interface at a high level.
Purchase Orders
In this interface we could reuse the existing implementation of a Logic App which received IDOCS published by SAP which the Logic App then forwards to Azure Service Bus. SAP would publish an IDOC for each time a purchase order is updated.
We could then build a purchase order update processing logic app which would take the following actions:
- Get the message from Service Bus
- Query SAP via its OData interface to get additional data
- Transform the data to a format which the invoicing partner needed
- Call a few different API operations on the Invoicing Partner’s system to upload the purchase order data
- Call back to SAP to confirm the Purchase Order was processed
In the Logic App we were able to use the SAP connector to consume an existing BAPI to update the purchase order but for the SAP OData Query and the Invoice Partners API we used Azure API Management as a proxy for these services to help give us a centralised view of HTTP usage and to allow richer security scenarios and to make our Logic Apps a little simpler.
While the diagram above is a slightly simplified view of the Logic App, in total the purchase order Logic App uses around 20 actions to implement the logic needed for this process.
Invoice Updates
The 2nd interface we build for this scenario was the invoice updates. Each time a supplier submits invoices, there is a change to them or they get paid we needed to reflect these changes into SAP. The Invoicing Partner has an API where we can implement a polling pattern in Logic Apps to look for changes made recently. We can then retrieve the details for the invoice and its current status. We would then transform the data to the SAP format and upload the data. Also if there were any PDF’s included with the invoice we would copy them down and load them into SAP too.
This interface looks like below 2 diagrams.
Invoice Poller
The invoice poller logic app will run on a regular schedule and check via the Invoicing Partners API to see if there are any invoice updates which need processing.
It will then for each one create a message on Azure Service Bus so that we can build interfaces off the back of a publish/subscribe pattern as needed.
Invoice Processor
The Invoice Processor Logic App will trigger from Service Bus and it will use API Management to call out to the Invoicing Partner API to check for the details of the invoice that has been updated.
It will then transform the data to SAP format and make calls to SAP via some existing Fiori SOAP Services in the organisation which will allow us to Add or Update an invoice. The Logic App will then check on the Invoicing Partner API to see if the invoice has any PDF attachments and it will download them and upload them to SAP so that the finance user is able to see the latest copy of the attachments in SAP when processing invoices. The below diagram shows the Invoice Processor.
The invoice processing logic app may not look so complicated in the diagram above but there are 60 shapes in this Logic App to implement all of the functionality that is required.
Operational Support and Monitoring
At this point we have built a solution that implements the interfaces quite nicely and Logic Apps and API Management have definitely helped us to make a relatively complicated interface a lot simpler, especially by the abstractions for connectivity provided by API Management and the SAP connectors.
Next we need to consider how we will run and support an effective solution in the real world.
Our Finance users are very competent users and the SAP team are very good at supporting the business process here and we want to be able to ensure that they have good visibility into the performance of the solution but also that the solution is robust and trouble free, but when there is a problem its not difficult to find out what is going on.
There are a number of different facets to the support experience such as the platform service, ensuring that the Logic Apps are available and running and performing well but there is also the support of the business transaction and how to troubleshoot issues if 1 purchase order out of many has a problem.
Next we will take a look at the different support capabilities we have in the solution.
Integration Expert
If we start with the Integration Expert in the Integration team. This user has a lot of deep dive information available and it is a highly privileged role with a lot of access to Azure to support these parts of the solution.
When something goes seriously wrong there is a lot of information available to the integration expert. Some of this includes:
- Logic App Run History
- API Management & App Insights
- Logic App Diagnostics sent to Log Analytics
The first thing the integration expert is likely to use is Logic App Run History. This will give you the usual friendly Logic App view where you can step through a Logic App checking what happened. You can often find information about problems here and troubleshoot errors caught by the Logic App from connectors. Sometimes its not so easy to workout which Logic App run is the ones you need to check.
In API Management we have set it up with Application Insights. This means we will have telemetry captured from all of the Logic App calls to API Management and we can see additional information about the API performance and errors and usage. Logic Apps automatically sends some headers on its calls to Azure API Management below:
- Request-x-ms-workflow-resourcegroup-name
- Request-x-ms-workflow-run-id
- Request-x-ms-workflow-name
You can also use these headers in your queries of App Insights to find the Logic App calls to APIM.
We also have the diagnostics capabilities of Logic Apps available too. If we need to we can turn on the sending of Logic App telemetry to a Log Analytics Workbook and it will allow us to do deep queries of whats happening in our Logic Apps for complex support troubleshooting.
There are some great out of the box capabilities for the Integration Expert to do deep dive troubleshooting in Azure Integration Services.
IT Support
Next we have the IT Support Operator user. This used may have a little bit of experience with Azure and some interfaces but its common in many organisations that you need to be careful about giving the support operator too much access to solutions they don’t have enough experience to support. This can often result in errors being caused by the support operator which are worse that the original issue they were looking at.
In Azure there is the ability with Role Based Access Control and Privileged Identity Management to provide fine grained access to resources in Azure but it’s a big challenge for the non-Azure / non-Integration Expert IT support user to know which Azure Resources are the ones which work together to implement a given interface and then support them.
This is where the core use case for Turbo360 comes into play.
Business Applications
Turbo360 Business Applications is designed to create a support view of your Azure Resources. You create a Business Application to represent the solution to a business problem. You then add the Azure resources which work together to solve that business problem to the business application.
If you imagine an Azure customer with an enterprise integration platform may have hundreds of Logic App implementing different interfaces, hundreds of queues in Service Bus, tens of API’s in API Management and many many Azure Functions in different Function Apps.
If you see below in the Turbo360 image you can see how you would add business applications to represent your key interfaces. This is where the support operator could then go when they know there is an invoice processing issue the invoice processing business app should have all of the related Azure resources in it.
The business application then has features to help the support operator perform every day support activities on some of the Azure resources you might need. You can manage Logic App errors and keep track on which ones are fixed and which are outstanding using the Action Required feature which is commonly used if we get a support issue for these interfaces. The most common issue is that one of the endpoints for the applications may have gone off line and some Logic App runs got an error. The operator can check for the details of the issue and then resubmit it for processing.
Business Applications provides value to the support of the solution because it lowers the total cost of ownership because the inexperienced IT support operator can deal with many support tickets for the service via Turbo360. They would normally have to escalate most of them to the Integration Expert.
We can also implement proactive monitoring to check for availability of services and ensure that data is being processed and raise issues to the support operator to be fixed rather than waiting for the business users to raise a ticket.
Business User Self Service
I mentioned earlier that the SAP team and Finance team are really good at managing their service and they are keen to be able to self service manage their interfaces. If they have an issue they would like to be able to check things themselves and then if the issue is one they are comfortable with they want to be able to try to resolve issues themselves too. They do not want to have to wait for IT support tickets to be passed between teams as this takes time to happen and this is where self service of integration can be a real cost reduction for the enterprise.
This problem is why Turbo360 Business Activity Monitoring (BAM) was created. With BAM we are able to give a simplified view of the integrated processes which can be understood by an every day user and does not require integration expertise. I mentioned earlier that one of the Logic Apps to implement the Invoice process was 60 shapes but we can potentially represent this with a much smaller number of BAM shapes as shown in the diagram below. Most of the time there would only be 4 of the shapes which got executes for the green path scenario but there are some other shapes to represent error conditions. Really the business user is interested in the following key milestones.
- Started Update
- Got Data from Invoice Partner
- Updated SAP
- Got error
These are the key milestones that the business user cares about. In BAM I would represent this with an activity diagram like below:
As the process executes then the user would be able to see which shapes were successful and where the error was. In the image below you can see how a successful transaction was processed and the route through the process which was ran.
In the BAM Tracking view the user can also search for transactions based from properties which have been promoted. You could promote data from a message body or a message context which you have sent to BAM. In our invoice and purchase order interfaces the key properties we promote include:
- SAP IDOC number
- Invoice number from the invoice partner
- Business Partner ID for the supplier who the invoice and purchase order relates to
This allows the BAM user to then search for transactions based on these values to help with common queries they might have such as:
- Did we process invoice 123 from Partner ABC
- How many invoices have we processed today
Below is what your tracking search might look like. You can customize the columns to suit the data that you want to capture.
BAM also has dashboards available so we can create a single view to get an overview of how the business transactions are performing. You can see in the below diagram I can get an idea of the number of transactions, when they are happening and the errors.
BAM can be used by all types of user, it’s a way to see the business view of your technical interfaces. The key benefits though are when you let application teams, business teams and front line IT support teams have access to BAM. These are the people who need these interfaces and want visibility into them and to be able to perform basic support operations to keep their part of the business operating well and so they can give their customers a great service.
Summary
In summary Azure provides deep dive information to help your integration experts troubleshoot and manage the integration platform and its components.
Turbo360 Business Applications allow you to democratize the support of your integration solutions by allowing the IT Support Operator to manage the resources which solve the business problem at an application level.
Turbo360 BAM helps you democratize integration even further by giving your business users visibility of their integration processes and allowing them to have a degree or self service.
The combines approach of Azure and Turbo360 allows us to provide an enhanced support experience for the various stakeholders but also lowers the total cost of ownership of the integration platform implemented with Azure Integration Services.
Try BAM
If you would like to explore how BAM can help you with the implementation of interfaces which integrate with SAP or other systems using the Microsoft Integration Services go to the Turbo360 Azure Distributed Tracing Tool.
