It is again the time of the year for the premier worldwide conference on Microsoft Integration technologies – INTEGRATE 2020. Due to COVID-19 situation, it was decided to run this event online and hence INTEGRATE 2020 REMOTE. It only got bigger this year with more than 900 registrations.
This year’s keynote was presented by Jon Fancey, Principal PM Manager at Microsoft. Jon used to be a Microsoft MVP and he moved to Microsoft a few years ago. He has been leading innovations in Microsoft Integration space. He is leading the Azure Integration Services team and was an architect at AIS a couple of years ago.
This keynote was very much awaited to know Microsoft’s Roadmap and new releases. Jon split his session under these categories –
- The journey we’re on
- Where we’re going
- One customer’s journey
Cloud Native
The concept of Cloud-native is not new anymore but has come of age for the last few years. He mentioned about four facets of Cloud-native paradigm.
Containerization and Componentization have been in practice for decades now but the maturity and the flexibility we have achieved in this space in the last few years is remarkable. It is driven by open source communities like Docker and Kubernetes. We may need a holistic way of managing these distributed components.
Open Source and Open Standards like Open API, Swagger and YAML lay the foundation to many standards for modern-day computing and integration. This extends the ability to define and consume APIs within applications easily. It has opened endless possibilities for integration and deployments.
The idea of ignoring infrastructure and scale on-demand as required by paying only for what and when needed is Serverless Computing. Many teams at Microsoft is working on Serverless and it is certainly the future of computing.
GitOps (a combination of GitHub as source repository and the tools like Azure DevOps) and CI/CD help in rapid change and deploy in different environments almost frictionless like never before.
All these enable us to build sophisticated and cloud-scale distributed applications not only on a single cloud but on many and in the local data centres, a combination of SaaS and other applications in the modern world. They also redefine the positioning of these three aspects like the sheer amount of data being managed in the cloud, Applications being built and distributed, people involved in managing these with various tools, business processes and communication between customers and trading partners.
The Modern Workplace Revolution
It is expected that half-a-billion new apps to be built in the next 5 years will be more than all those were built in the last 4 decades. Also, it is expected that nearly two-thirds of these applications will be built in the low-code category.
The availability of new tools like Power platform will enable this by dramatically reducing the amount of code necessary to build and get them into production. It doesn’t mean less sophistication in your application, but these tools do the heavy lifting for most of the code you would write otherwise, but now you can focus on your business side of building the applications.
People from different departments come together to build applications. This new shift in different teams collaborating is known as “Fusion Teams”. Citizen Developers from Line of Business would be able to take these low-code tools and build apps easily still having the governance and audit necessary from IT ops.
Integration is Everything
Of course, all these are possible because of Integration. In the broader sense, everything from traditional App-App, B2B, SaaS, IoT, Data & Analytics and People are part of the integration.
Applications around RPA, automation of manual and repetitive tasks is exploding in the last few years and are made possible because of integration.
Software Development Times magazine have predicted 2020 is the year of Integration. Integration has been there for last 20 years but the combination of ease of tools that help citizen developers to build apps, no to low-code tools, the possibility of using cloud platform to have higher degree of productivity like Azure Integration Services. The idea of utilization of these across different parts of the organisation and ecosystem whether it is On-Premise, or cloud makes year 2020 a promising one for Integration.
Azure Integration Services (AIS) – Microsoft’s integrated Cloud Platform
The set of capabilities and cohesive tool set like Logic Apps for work flow, EventGrid for event-driven indications, Data Factory for Extract,Transform and Load data, API Management enables to create, publish and consume APIs using uniform interface backed by industry standards and Service Bus that allows loosely coupled pub/sub typr messaging and all other things needed for Enterprise Application Integration.
Cloud Continuum – Azure and Power Platform
The concept of Integrations is on one side it is optimized for Citizen Developers and on the other is optimized for IT Pros Dev. It is crucial for teams in an organisation on how they work. There is a collaboration between these tools. There is a seamless collaboration between Logic Apps and Power Automate.
These tool set are developed to serve the purpose of different personas of your organisation but in the background they all serve the same purpose and collaborate well.
Logic Apps on Functions
Logic Apps could now be hosted on Azure Functions runtime. It is available in private preview
It is done to serve 3 purposes –
- Make Logic Apps run anywhere functions could run – Azure App Service, Stand Alone or in Kubernetes or in local dev environment using VS Code
- To provide a great development experience
- To provide performance improvements to drive new work loads, stateless workflows
Coming Soon – Automation Hub
It is the concept of being able to automate resource management. For e.g.: Use set of templates to Send email when VM is stopped.
Though all of these are possible today, you should be writing some scripts. But Automation Hub simplifies this through configuration, some of the tasks can be automated. Logic Apps is working behind the scenes.
Dapr Workflows – Distributed Application Runtime
It is cloud-native way of building distributed micro-services-based applications. Last week, Microsoft released the capability to integrate Dapr with Logic Apps
Now Logic Apps can now be triggered from bindings in Dapr, for eg: triggering Logic Apps from Kafka, which is not possible previously, is made possible through Dapr runtime. It helps to build microservices based applications in using any language and now integrate with Logic Apps. They can run inside any container, in local environment or in cloud.
Dapr Architecture
The logic App host is loaded inside a container which in turn is loaded into a Dapr Sidecar that enables communication from different microservices and the logic apps through various Dapr input bindings like RabbitMQ, Kafka, Event Hubs etc.
You can now run the Logic Apps through Dapr and return the data back through these bindings.
Introducing – Logic Apps SDK
There is already an SDK available for Logic Apps that provides CRUD operations at the resource level. A JSON definition template for Logic Apps can be modified with necessary configuration and deployed.
The new SDK has been extended. For e.g.: the code represented below, provides a very dynamic way of creating logic apps.
This SDK is soon to be released as Open Source in Git Hub Logic Apps repository. Using this, existing logic apps can be loaded and manipulated, the dependencies can be rewired through code.
But I’m still on-premises. How can I be cloud native too?!
Microsoft AIS team is now working on a BizTalk Migrator – a migration tool for those who are using BizTalk still. Through the tool, you can go through number of phases to migrate your BizTalk Applications. You can discover all the configuration in your BizTalk environment from the MSIs, parse them and analyse. The tool generates a report to provide what is possible to migrate and what is not. Based on the analysis you can configure conversions; your BizTalk Application can be converted into Azure Integration Services and post that you can run verifications as well. This helps to run tests to ensure the input and output values are same as they were in BizTalk environments.
Illustrated below is an example of one BizTalk Application migrated to AIS. Logic Apps are generated to reflect the control flow in BizTalk. Since Logic Apps don’t have Message box, Service Bus is used for messaging and Event Grids are used for event triggering.
Azure Arc
Introduced at Ignite 2019, Azure Arc, is an ability to create Arc agents in any Kubernetes cluster running anywhere – on-premise or cloud (GKE – Google Kubernetes Engine or AWS), deploy and manage from Azure.
This provides number of advantages to manage your Kubernetes Cluster management.
Central Management – provides cluster organization and inventory with a unified view in Azure portal across all locations.
At-Scale Control – Irrespective of the size and the location of the Kubernetes cluster, Azure Arc provides at-scale configuration and workload management to apply policies and governance.
GitOps – as part of deployment pipeline management.
Azure Services – Azure management capabilities brought to all clusters for consistent management.
There are lot of choices but simpler is better. Azure Integration Services have matured over the years that helps you to build anything with few things.
Workflows – addressed by Logic Apps.
Messaging – provides loose coupling between systems and platforms.
Events – allows to trigger events for eg: to create resources, scale out on demand as load increases.
APIs – to publish and consume pieces of functionality to build application.
Schedules – timing is important. to run something at desired times like run every five minutes or first Monday of the month.
Code – ability to build application and write in the desired language.
An application is an integration of certain components, whether pre-existing applications wiring up with APIs or SaaS applications wired using any micro-services. An application platform has an API layer, a data store, code to execute, being able to orchestrate calls between APIs using Workflows and be able to containerize using Kubernetes. Azure Arc fulfils the management layer by providing unified management and governance.
The operational capabilities like Azure DevOps and GitHub helps you to take your application to production with minimal friction. Improves the deployments whenever required like once a month or a week or as frequently as required. The modern IDEs like Visual Studio and Visual Studio code provide very good development experience. The power platform helps all personas in the organisation to develop over the Azure Arc.
Special Guest: Jon introduced James Blackburn, Principal Architect at Marks & Spencer (M&S)
Agile Integration at M&S
At M&S, they follow Agile Integration categorised under 3 major chunks.
Three Agile themes – Use cloud native architecture for integration, capability to build hybrid cloud integration, and set up a hybrid team to develop hybrid architecture.
Three Delivery Teams – hybrid architects and engineers to build core technology patterns like Logic Apps wrapped in patterns deployable and consumable by other teams, enterprise services build common APIs and services based on consumer demand, DevOps team to build and maintain interfaces.
Three Delivery Modes – on-demand request to DevOps team to deliver, commissioned model to request dedicated capacity, use the framework in self-service model where full stack developers take some of the frameworks and build their own integration.
Hybrid and Cloud Native Architecture
At M&S, they still have On-Premise implementation as first class citizen and cloud native architecture has got only fewer components.
Cloud APIs – has been using APIs and management for few years, moving towards converting that to Business-Business services.
Technology at M&S
Their focus then turned towards converting their implementation to Cloud Integration Technology. They built their cloud app in two areas focusing on
- Cloud Gateway Services that provides seamless connectivity to their on-premise.
- Cloud Transformation Capability that provides cloud native self-service workflow and have complex transformation use cases using serverless and containers.
Previously, their B2B traffic were routed through their Datacentres and cloud was an extension. But now, their datacentres are extension for the cloud.
People and Process at M&S
All these changes at M&S were achieved by making fundamental changes at the people resource level and processes.
Lesson learnt shared by M&S
It is all about setting up the right team with engineering, admin and architecture teams. Everything is automated with almost zero admin involvement and built on templates. Assess the failure rate and what is acceptable to balance the operations. Implement governance through patterns not to mean chaos.
Jon’s Q&A with M&S
1. How M&S is managing during this COVID-19 pandemic for the past 2-3 months
M&S being in retail industry is impacted like any other company in terms of operations. But, from technology perspective, any of their integration technologies or implementations has raised any challenges. Though they need to assess on how much does that cost for them to operate, they didn’t face any challenges in terms of component failures.
2. What have you learnt in last 3 months and what have you done differently?
Ability to define scalable templates so they could react to demand when to scale up and scale down. They are assessing their cost elements on non-production, what tools to use.
3. What are the changes in priorities from business point of view?
Very much focusing on clothing and home business moved to online that changed transactional dynamics. They are very much now focusing on what does their investment priorities are going to be.
The keynote has been very valuable in setting up the context for the next three days. There will be deep dive sessions in many core topics.
Jon appreciated the event INTGERATE and the Microsoft community and mentioned how important it was from their product group point of view. He wished everyone to stay safe and stay healthy.
Happy Integrate 2020 Remote!
