The Role of Enterprise Integration in Digital Transformation

Kasun Indrasiri Director – Integration Technology

WSO2

Agenda

• Digital Transformation and Integration

• Centralized Integration

• Future Integration needs

• Microservices and Integration

• Decentralized Integration – Micro-Integration

• WSO2’s Next generation integration platform with Ballerina

Digital Transformation and Enterprise Integration

• DT in a nutshell – Evolving Business Models

– Focus on customer experience

– Optimize Operation

• Brown-field reality requires integration

– Hence Enterprise Integration is a key step towards Digital Transformation.

Enterprise Integration – In a nutshell

• “Technology for developing, maintaining, testing, deploying, and governing interfaces between applications, machines, or databases” – Forrester 2015

State of Integration Technologies Landscape

• Application and data integration technologies are moving to a dynamic space – Cloud, Mobile, APIs, IoT, Convergence of Data and Application Integration

Centralized Integration

• A central integration middleware that connects application, services and data

• Conventional SOA/ESB approach

• Legacy systems to cloud services connected with services, data, messages and processes.

Centralized Integration

• WSO2 Enterprise Integrator 6.0

Centralized Integration

• WSO2 Enterprise Integrator 6.0 – Runtime Structure – ESB and DSS as a single runtime

– Message Broker

– Business Process Server

– Application Server

– Analytics (primarily ESB analytics)

Can Conventional Integration Middleware survive?

• Drastic changes in modern Enterprise Integration needs.

• Not compatible with new architectural paradigms (e.g. Container Architecture, Microservices)

• Almost all integration middleware solution more or less offer the same set of functionalities for a decade or so.

Future Integration needs

• Growth and diversity of Integration needs – APIs and SaaS

– Internet of Things

– On-premise applications

• B2B, Proprietary, Legacy systems

Future Integration needs

• Agility and ease of Integration– Minimum integration skills and operational overhead.

– Customize existing integrations rapidly.

– Visual modeling, debugging, trouble shooting

– Analytics – Statistics, message tracing

– Error handling.

– Streamlined development life cycle.

Future Integration needs

• Orchestration – Implementing complex orchestration logics – Proliferation of services, APIs and applications to integrate.

– Complexity of the orchestration logic increases

– Need a simple and agile development of orchestration logic – Visual modeling tools.

Future Integration needs

• Orchestration – Implementing complex orchestration logics – Netflix API: Single API call to nested, conditional, parallel execution

of multiple backend network.

Future Integration needs

• Integrating applications, services, data, APIs and identity– There’s a broad integration challenge than the traditional ESB

related integration.

– Integration Server, Data Integration, Identity Bus, API GW/Composition

Future Integration needs

• Performance – No of transactions and latency

– Ever increasing growth of traffic.

• E.g: Growth of API calls in 1 year

Source : http://blog.mailchimp.com/10m-api-calls-per-day-more/

Future Integration needs

• Performance (in containers) – No of transactions and latency per container

– Startup Time

– Memory foot print

– Distribution size

– Average CPU consumption, Load Average

Future Integration needs

• Scalability – Container based scaling

– Scaling based on the integration solution

• E.g : Ability to scale a given integration solution without scaling others integration solutions.

Future Integration needs

• Microservices and ESB/Integration Middleware – Increasing adaptation of Microservices Architecture

– Microservices do not encourage the conventional centralized integration middleware.

– Integration middleware solutions are not container native.

Microservices architecture rejects ESB?

• Smart endpoints and Dumb pipes. – No ESB!

– All the business logic and routing logic should be moved to the endpoints.

Microservices architecture rejects ESB?

• Smart endpoints and Dumb pipes => Point to point Integration

Microservices and Enterprise Integration – In Practice

• Netflix API– Single API call to nested, conditional, parallel execution of multiple

backend networks.

– Netflix API layer contains the routing logic.

Microservices and Enterprise Integration – In Practice

• Uber API – Edge services contains the orchestration logic

Decentralized Integration

• No centralized integration bus/ESB.

• Build independent integration scenarios using integration framework -> Micro-Integrations/Integration Microservices

Micro-Integration/Integration Microservices

• Build a specific integration scenario and run that scenario independently with a light-weight integration framework.

• One integration scenario per each runtime.

• Runtime is extremely lightweight fully container native.

• Develop, deploy and scale each integration scenarios Independently.

Micro-integration

• Integrating Microservices

Micro-integration

• Integrating applications, services and data.

Micro-integration

• Integrating Microservices and other conventional applications

Micro-integration

• Scaling Micro-integrations

5 – Instances

2 – Instances

10 – Instances

Types of Micro-Integrations

APIs and Integration

• API expose the internal business functionalities.

• API GW is the central component that serves API traffic and the central location that security, throttling and caching

• API Gateway is the new ‘Monolith’

APIs and Integration

• API Gateway and Integration – Layered Architecture

Source : http://techblog.netflix.com/2016/08/engineering-trade-offs-and-netflix-api.html

APIs and Integration

• API Gateway and Integration – Layered Architecture

APIs and Integration

• APIs and Micro-integrations

How WSO2 address these modern Integration

Requirements

• Building a integration platform that addresses the future Integration needs.

• We don’t want to build just a ‘new ESB’.– But a reusable framework which is shared among similar

integration solutions – applications, services, APIs, data, identity etc.

• A visual and textual representation.

Next generation WSO2 Integration Platform

• Addressing the future Integration needs.

Ballerina

• A new programming language with text and graphical syntaxes

• Sequence diagram programming model

• Modern, network-aware, data-aware, security-aware programming system inspired by Java, Go, Maven, ...

Ballerina

• Graphical/textual parity with ability to switch back and forth• Common integration capabilities are baked into the language:

– Deep HTTP/REST/Swagger alignment– Connectors for Web APIs and non-HTTP APIs– Support for JSON, XML, (No)SQL data and mapping

• No magic – no weird syntax exceptions, everything is derived from a few key language concepts

• Maximize developer productivity and abstraction clarity• Support high performance implementations: low latency, low memory

and fast startup

Ballerina Concepts

Ballerina

• Complex Orchestrations made simple. – Sequence diagram metaphor is a simplest way to represent

orchestrations.

Ballerina

• Thinking in ‘Parallel’ – Multi-worker execution

Ballerina

• Connecting anything with anything. • Client and server connectors for HTTP 1.1/2, WebSockets, JMS,

(S)FTP(S) and more

• Client connectors for BasicAuth, OAuth, AmazonAuth

• Connectors for Web APIs: Twitter, GMail, LinkedIn, Facebook, Lambda Functions,

Ballerina

• Enterprise Integration Patterns– Most EIPs are integral part of the Ballerina Programming Language

– if-else, fork-join, for-each etc.

Ballerina

• Type Mapping – Visual Type mapping

– First class support for type mapping at the language level.

Ballerina

• Performance – Ballerina uses latest WSO2 Netty HTTP transport (over 5x of

performance improvement compared to traditional HTTP transport implementations)

– Super-fast startup and low memory foot print.

– Fully non-blocking execution.

Ballerina

• Debugging – Visual debugging

Ballerina

• Scaling – Container native

– Independent scaling of each micro-integration

Enterprise Integrator 7.0

• EI 7.0 : Ballerina replaces the ESB, Data Services runtime.

• EI will have two parallel versions for a long time.

• Migration tool will be able to handle about 80% of ESB and Data Services to Ballerina.

• We will continue to evolve and support Enterprise Integrator v6!– EI v6 will have continuing minor releases

– ESB users will not be left hanging

WSO2 Hybrid Integration Platform

• We focus on On-premise integration and iPaaS – WSO2 Integration Cloud – ESB 5 in the cloud with DSS

Source : Gartner

Summary

• Integration middleware is not disappearing…• Rather growing to cover broad integration scenarios.

• Micro-integration will rule the world.

• Next generation WSO2 Integration Platform is addressing those new paradigm shifts in Enterprise Integration with Ballerina.

Thank You!