This blog is an abstract from the session “When to Use What Integration Offering: 2020 Edition” presented by Richard Seroter on Day 3 of INTEGRATE-2020. Richard is currently the Director of Product Management, Google Cloud and holds an accolade of 12-time Microsoft MVP for cloud.
Richard initiated the session by sharing his experience on presenting at INTEGRATE 2014 and suggested a book named, “The Paradox of Choices” which deals closely with the session.
Agenda
- Technology trends
- Decision criteria for choosing Integration technology
- Integration Technologies
Technology trends
During this part of the session we had a complete review on the technology trends that matter most to the Integrators. Richard gave a statistical overview on a hand full of technologies.
Multi- Cloud
Over 80% of integrators opt for Multi-cloud service provider, where they can have optional providers and choose between the primary or secondary choices to share the workload. The greatest scope of multiple providers is the end points will not remain homogeneous and integrators can consume a lot from different infrastructures.
Kubernetes
Kubernetes is a streaming technology and about 60% of enterprises are using Kubernetes today mostly in production. These enterprises started consuming different clusters. As integrators, you can talk to many clusters that change all the time as containers come and go.
Microservices
Recent data shows that 52% of companies are using microservice concepts, tools and methods. It contains granular services built in a chain and can face changes on few factors as services keep changing once in few years.
Managed Services
It is expected that by 2024 more than 80% of enterprises might shift from Iaas migration to managed platforms so that they can deal with higher level operations. An Integrator may face shift of integrations and endpoints.
Low Code
Low Code would occupy 65% of development activity in 2024, where the recent code patterns will remain unchanged but there will also be equal number of people opting for low code.
Edge Computing
As we move forward to edge architectures more than 50% of large enterprises will deploy single edge use case by the end of 2021. Using specified cloud provider will move out of fashion and workload will be spread widely.
Open Source
There will be decrease of open source usage from 2019 to 2022 which sounds shocking. Some enterprises are still comfortable with open source tech which might be deviated by some proprietary software used by the organisation.
Modernization
Modernization is an on-going effort, 59% of the companies are moving forward and taking efforts to modernise the integration, end points and infrastructure. This trend keeps growing by time.
Decision Criteria
Richard pointed out 8 key decision-making criteria on choosing the technology that better suits your integration.
- Payload Characteristics
- Endpoint type
- Workflow Needs
- Rate of Change
- Portability
- Operations Burden
- Available Skills
- Strategic Value
Integration Technologies
This is the final part of the session where Richard introduced few technologies and ranging of few metrics that speaks about the capabilities of the technology. The elaborated explanation will be of greater use to choose the right technology.
Queuing Engine
Queuing Engines acts as centralized broker that decouples senders and receivers of messages for publish and subscribe activities with transient data. The characteristics of Queuing Engine are,
- It supports bulk processing
- Offers good support on Real-time processing
- Excellent delivery guarantees
- Good connectivity to cloud-hosted end points and on-prem endpoints.
- It does not facilitate long or short running workflows
- Extends good support to CI/CD of components
- It is not Container friendly
- Offers good amount of automated operations.
Products like Azure Service Bus, Google Cloud and Amazon SQS are few Queuing Engines.
Event Stream Processor
Event Streaming Processor is completely based on a durable and persistent log and does not include any routing of data. The characteristics of Event Stream Processor are,
- Provides great support on bulk processing
- Supports Real-time processing
- Good Quality of Service
- Good connectivity to cloud-hosted end points and on-prem endpoints.
- It does not facilitate long or short running workflows
- Extends good support to configurations
- It is Container friendly
- Offers good amount of automated operations.
- It matches widely available skillset.
Workflow/ Data Pipeline Engine
This technology includes a series of procedure to automate a business process and often offers a connector library. The characteristics of Data Pipeline Engine are,
- Provides great support on bulk processing
- Supports Real-time processing
- Great Quality of Service
- Excellent connectivity to cloud-hosted end points and on-prem endpoints.
- Facilitates brilliant long or short running workflows
- Extends good support to configurations
- It is not Container friendly
- Offers good amount of automated operations.
- It matches widely available skillset.
Integration Broker
These are full-featured engine that offers message routing, connecting and B2B tooling. The characteristics of Integration Brokers are,
- Does not support bulk processing
- Good support on Real-time processing
- Great Quality of Service
- Good connectivity to cloud-hosted end points and on-prem endpoints.
- Facilitates brilliant long or short running workflows
- Extends good support to configurations
- It is not Container friendly
- Offers good amount of automated operations.
- It matches widely available skillset.
ETL/ELT Tools
ETL/ELT Tools are deals with transfer of large amount of data through customized pipelines. It might offer data sorting, joining, aggregation and many more.
- Supports bulk processing
- Does not support on Real-time processing
- Great Quality of Service
- Good connectivity to cloud-hosted end points and on-prem endpoints.
- Facilitates good long or short running workflows
- Extends good support to configurations
- It is not Container friendly
- Offers great amount of automated operations.
- It matches widely available skillset.
Communication Framework Protocols
It is an Advanced RPC communication between components that offer load balancing, bi-directional streaming etc.
- Does not provide good support for bulk processing
- Great support on Real-time processing
- Does not provide good Quality of Service
- Good connectivity to cloud-hosted end points and on-prem endpoints.
- Does not facilitate long or short running workflows
- Extends good support to configurations
- It is Container friendly
- Offers great amount of automated operations.
- It does not match widely available skillset.
Conclusion
Followed by Richard’s lively session there was an interactive Q&A session between the attendees and speaker. Questions were raised regarding Kubernetes, Data-driven trend, RPA integration and few more. Hope this blog helped you to analyse the landscape and choose the right integration technology from the wide range.
