Date post: | 17-Oct-2014 |
Category: |
Technology |
View: | 439 times |
Download: | 1 times |
Better Business Results with Open Source Integration
Kenneth PeeplesJBoss Technology Evangelist
2 RED HAT JBOSS MIDDLEWARE
Agenda for SOA Summit
● Overview of Open Source Integration with Data Virtualization
● Preview of JBoss Middleware Integration Offerings● Integration examples that will help you get started
4 RED HAT JBOSS MIDDLEWARE
Red Hat is…
“By running tests and executing numerous examples for specific teams, we were able to prove […] not only would the solution work, but it will perform better & at a fraction of the costs.”
MICHAEL BLAKE, Director, Systems & Architecture
“By running tests and executing numerous examples for specific teams, we were able to prove […] not only would the solution work, but it will perform better & at a fraction of the costs.”
MICHAEL BLAKE, Director, Systems & Architecture
Overview of Open Source Integrationwith Data Virtualization
Better Business Results through Open Source Integration
6 RED HAT JBOSS MIDDLEWARE
SOAs and abstraction
Decoupling is the key concept in SOAs achieved through abstraction based on service interfaces
Business processes and services make use of business information
Business information is exposed to business services as data services
7 RED HAT JBOSS MIDDLEWARE
Bridging the Gap
The need to bridge data gaps in a more straightforward, streamlined and scalable way is due to:
● More and more data
● Growing need for consolidated views of information
● Drive toward interoperability and standards
8 RED HAT JBOSS MIDDLEWARE
Data Services through Data Virtualization
Key requirements of data services
● Expose integrated information in one or more desired formats, even if the original data are in different formats
● Expose information through one or more consistent, standard interfaces, even if the original data are accessed through different interfaces
The requirements are achieved by
● Modeling to define the required format of data, integrated from the underlying sources
● A query engine for processing these abstract definitions efficiently, exposing the integrated information through one or more interfaces
9 RED HAT JBOSS MIDDLEWARE
Data Virtualization essential to a SOA
Data Virtualization provide the necessary interface to data for all business services. Data Virtualization:
● Expose all data through a single uniform interface● Provide a single point of access to all business services in the
system
● Expose data using the same paradigm as business services - as "data services"
● Expose legacy data sources as data services
● Provide a uniform means of exposing/accessing meta-data● Provide a searchable interface to data and meta-data● Expose data relationships and semantics● Provide uniform access controls to information
10 RED HAT JBOSS MIDDLEWARE
Data Virtualization and SOA Goals
● Create an enterprise data foundation for streamlined development, better integration and more efficient data governance
● Improve delivery of consistent, complete information from any data source to the application and business processes
● Externalize business knowledge buried in applications and databases
● Expand automation, moving beyond reports to active analysis and decision-making
● Build business processes in code that are actually followed by the organization
11 RED HAT JBOSS MIDDLEWARE
What is Data Virtualization?
Data Virtualization provides a service that:
● Encapsulates data-oriented logic● Enable loose coupling between the application and the data stores● Manages real-time connectivity to data stores● Offers a higher level of data abstraction compared to interacting
directly with data stores
● Provides appropriate views of data
● Allows views of data to be cached based on performance requirements
12 RED HAT JBOSS MIDDLEWARE
Translators and Resource Adaptors
A translator
● provides an abstraction layer between the run time engine and physical datasource
● Knows how to issue query commands into source specific commands
● Execute them using the resource adapter
A Translator also defines the capabilities of a particular source, like whether it can natively support query joins (inner joins, cross joins etc) or support criteria.
A Translator along with its Resource Adapter must be configured on a Source Model
13 RED HAT JBOSS MIDDLEWARE
Translators and Resource Adaptors (Continued)
A Resource Adapter provides the connectivity to the physical data source
This also provides way to natively issue commands to the source and gather results
Translator capabilities define what processing each translator/source combination can perform.
For example, most relational sources can process joins and unions, whereas when processing delimited text files these operations cannot be performed by the resource adapter or the "source" (in this case, the file system).
14 RED HAT JBOSS MIDDLEWARE
Modeling Environment for Data Virtualization
Virtual Databases contain two primary varieties of model types - source and view.
Source models represent the structure and characteristics of physical data sources, whereas view models represent the structure and characteristics of abstract structures you want to expose to your applications.
Models used for data integration are packaged into a virtual database (VDB).
The models must be in a complete and consistent state when used for data integration.
That is, the VDB must contain all the models and all resources they depend upon.
Source models must be configured with a Translator and a Resource Adapter with them before a VDB is tested in Designer or deployed for data access
15 RED HAT JBOSS MIDDLEWARE
Runtime Environment for Data Virtualization
● Connect to multiple, heterogeneous data sources
● Optimize performance● Manage data security● Take advantage of run-time system management features● Audit data service usage
Preview of JBoss MiddlewareIntegration Offerings
Better Business Results through Open Source Integration
17 RED HAT JBOSS MIDDLEWARE
Innovate faster, in a smarter way
A family of a lightweight, enterprise-grade products that are ideal for open hybrid cloud environments.
18 RED HAT JBOSS MIDDLEWARE
There is a better way – Red Hat JBoss Integration Product LineRed Hat JBoss A-MQ
● A small-footprint, high-performance messaging platform● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/amq/● Developers Download the Code - https://www.jboss.org/products/amq.html
Red Hat JBoss Fuse● A small-footprint, flexible enterprise service bus (ESB)● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/fuse/● Developers Download the Code - https://www.jboss.org/products/fuse.html
Red Hat JBoss Fuse Service Works● A service design, development and integration platform that enables
organizations to transition to an open hybrid cloud● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/fuse-service-works/● Developers Download the Code - https://www.jboss.org/products/fsw.html
Red Hat JBoss Data Virtualization (Only Open Source Data Virtualization product on the market)
● A lean, virtual data integration that unlocks trapped data and delivers it as easily consumable, unified, actionable information
● Product Page – http://www.redhat.com/products/jbossenterprisemiddleware/data-virtualization/● Developers Download the Code - https://www.jboss.org/products/datavirt.html
19 RED HAT JBOSS MIDDLEWARE
Integration Everywhere
20 RED HAT JBOSS MIDDLEWARE
JBoss Data Virtualization v6
21 RED HAT JBOSS MIDDLEWARE
What does Data Virtualization software do?Turn Fragmented Data into Actionable Information
Data Virtualization software virtually unifies data spread across various disparate sources; and makes it available to applications as a single consolidated data source.
The data virtualization software implements 3 steps process to bridge data sources and data consumers:
● Connect: Fast access to data from diverse data sources
● Compose: Easily create unified virtual data models and views by combining and transforming data from multiple sources.
● Consume: Expose consistent information to data consumers in the right form thru standard data access methods.
Virtual Consolidated Data SourceVirtual Consolidated Data Source
BI Reports
Data Virtualization Software• Consume• Compose• Connect
SAP Salesforce.comOracle DW Hadoop
Siloed & Complex
VirtualizeAbstractFederate
Easy,Real-time
InformationAccess
SOA Applications
DATA CONSUMERS
DATA SOURCES
22 RED HAT JBOSS MIDDLEWARE
Turn Fragmented Data into Actionable Information
ConnectConnect
ComposeCompose
ConsumeConsume
Unified Customer View
Unified Customer View
Unified Product View
Unified Product View
Unified Supplier View
Unified Supplier View
BI Reports & AnalyticsMobile Applications
SOA Applications & Portals
Unified Virtual Database / Common Data Model
ESB, ETL
Native Data ConnectivityNative Data Connectivity
Standard based Data ProvisioningJDBC, ODBC, SOAP, REST, OData
Standard based Data ProvisioningJDBC, ODBC, SOAP, REST, OData
JBOSSDV
DATA CONSUMERS
DATASOURCES
Design ToolsDesign Tools
DashboardDashboard
OptimizationOptimization
CachingCaching
SecuritySecurity
MetadataMetadata
Hadoop NoSQL Cloud AppsData Warehouse
& Databases MainframeXML, CSV
& Excel Files Enterprise Apps
Siloed & Complex
VirtualizeAbstractFederate
Easy,Real-time
InformationAccess
23 RED HAT JBOSS MIDDLEWARE
JBoss Data Virtualization:Supported Data SourcesEnterprise RDBMS:• Oracle • IBM DB2 • Microsoft SQL Server• Sybase ASE• MySQL• PostgreSQL• Ingres
Enterprise EDW:• Teradata • Netezza • Greenplum
Hadoop:• Apache• HortonWorks• Cloudera• More coming…
Office Productivity:• Microsoft Excel • Microsoft Access• Google Spreadsheets
Specialty Data Sources:• ModeShape
Repository• Mondrian• MetaMatrix• LDAP
NoSQL:• JBoss Data Grid• MongoDB • More coming…
Enterprise & Cloud Applications:• Salesforce.com• SAP
Technology Connectors:• Flat Files, XML Files,
XML over HTTP• SOAP Web Services• REST Web Services• OData Services
24 RED HAT JBOSS MIDDLEWARE
Key New Features and Capabilities for v6● Data connectivity enhancements
– Hadoop Integration (Hive – Big Data),
– NoSQL (MongoDB – Tech Preview) and JBoss Data Grid
– Odata support (SAP integration)
● Developer Productivity improvements – New VDB Designer 8 and integration with JBoss Developer Studio v7
– Enhanced column level security,
– VDB import/reuse, and native queries
● Simplify deployment and packaging – Requires JBoss EAP only; included with subscription
– Remove dependency with SOA Platform
● Business Dashboard– New rapid data reporting/visualization capability
25 RED HAT JBOSS MIDDLEWARE
Self-Service Business Intelligence
The virtual, reusable data model provides business-friendly representation of data, allowing the user to interact with their data without having to know the complexities of their database or where the data is stored and allowing multiple BI tools to acquire data from centralized data layer. Gain better insights from Big Data using JBoss Data Virtualization to integrate with existing information sources.
360◦ Unified View
Deliver a complete view of master & transactional data in real-time. The virtual data layer serves as a unified, enterprise-wide view of business information that improves users’ ability to understand and leverage enterprise data.
Agile SOA Data Services
A data virtualization layer deliver the missing data services layer to SOA applications. JBoss Data Virtualization increases agility and loose coupling with virtual data stores without the need to touch underlying sources and creation of data services that encapsulate the data access logic and allowing multiple business service to acquire data from centralized data layer.
Regulatory Compliance
Data Virtualization layer deliver the data firewall functionality. JBoss Data Virtualization improves data quality via centralized access control, robust security infrastructure and reduction in physical copies of data thus reducing risk. Furthermore, the metadata repository catalogs enterprise data locations and the relationships between the data in various data stores, enabling transparency and visibility.
● JBoss Data Virtualization – Use Cases
26 RED HAT JBOSS MIDDLEWARE
There is a better way – Red Hat JBoss Integration Product Line
JBoss A-MQ
JBoss Fuse(Includes JBoss A-
MQ)
JBoss Fuse Service Works
(Includes JBoss Fuse)
A small-footprint messaging platform, eliminating batch delivery
“JBoss A-MQ for real-time messaging”
A Integration/ESB platform for integrations that can extend outside the data center
“JBoss Fuse for integration/ESB requirements”
A Service design, development and integration platform that enables responsive and innovative enterprise
“JBoss Fuse Service Works for integration/ESB+SOA+Orchestration+Governance requirements”
Additional Capabilities
Additional Capabilities
27 RED HAT JBOSS MIDDLEWARE
JBoss A-MQ v6.1
New in 6.1:
● Full support of AMQP 1.0 – provides wire-level compatibility across connections● Improved high availability (HA) – embedded message store for shared-nothing
HA
28 RED HAT JBOSS MIDDLEWARE
JBoss Fuse v6.1
New in 6.1
• Full support of AMQP 1.0 – provides wire-level compatibility across connections• Vast library of connectors – over 200 out-of-the-box-connector via Apache Camel• Managed integration routes - start, stop, measure and trace Camel routes on-premise or in
the Cloud• Improved high availability (HA) – embedded message store for shared-nothing HA
29 RED HAT JBOSS MIDDLEWARE
JBoss Fuse Service Works v6
Enables organizations to Simplify integrations and focus on business innovation
30 RED HAT JBOSS MIDDLEWARE
SwitchYard Application Overview
Composite
ComponentComponent-Service
Service-Binding
Composite Service
Component-Reference
Component-Binding
Reference-Binding
31 RED HAT JBOSS MIDDLEWARE
Design Time and Run Time Governance
Integration examples to help you get started
Better Business Results through Open Source Integration
33 RED HAT JBOSS MIDDLEWARE
Sample 1: Big Data with Data Virtualization
● Objective:– Determine if sentiment data from the
first week of the Iron Man 3 movie is a predictor of sales
● Problem:– Cannot utilize social data and
sentiment analysis with sales management system
● Solution:– Leverage JBoss Data Virtualization to
mashup Sentiment analysis data with ticket and merchandise sales data on MySQL into a single view of the data.
ConsumeComposeConnect
Consumer 1: Excel Powerview to analyze the aggregated data
JBoss Data Virtualization
Hive
SOURCE 1: Hive/Hadoop contains twitter data including sentiment
SOURCE 2: MySQL data that includes ticket and
merchandise sales
Consumer 2 : DV Dashboard to analyze the aggregated data
https://github.com/DataVirtualizationByExample/Mashup-with-Hive-and-MySQL
34 RED HAT JBOSS MIDDLEWARE
35 RED HAT JBOSS MIDDLEWARE
36 RED HAT JBOSS MIDDLEWARE
37 RED HAT JBOSS MIDDLEWARE
38 RED HAT JBOSS MIDDLEWARE
39 RED HAT JBOSS MIDDLEWARE
40 RED HAT JBOSS MIDDLEWARE
Sample 2 : Travel Triage
http://www.ossmentor.com/2013/10/sap-nw-gateway-with-fuse-and-dv.html
41 RED HAT JBOSS MIDDLEWARE
Sample 3: Home Loan v1
https://github.com/FuseServiceWorksByExample/Home-Loan-Reference-Architecture
42 RED HAT JBOSS MIDDLEWARE
Sample 3: Home Loan v1 (Continued)
Closing
45 RED HAT JBOSS MIDDLEWARE
Question and Answer Session
46 RED HAT JBOSS MIDDLEWARE
References
Data Virtualization Datasheet - http://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-data-virtualization-datasheet
Fuse Service Works Datasheethttp://www.redhat.com/resourcelibrary/datasheets/red-hat-jboss-fuse-service-works-datasheet
Red Hat JBoss Data Virtualization for financial services Whitepaperhttp://www.redhat.com/resourcelibrary/whitepapers/data-virtualization-fsi
Red Hat JBoss Fuse Service Works technology overview Whitepaperhttp://www.redhat.com/resourcelibrary/whitepapers/red-hat-jboss-fuse-service-works-technology-overview
Turn data into actionable information with Red Hat JBoss Middleware Whitepaperhttp://www.redhat.com/resourcelibrary/whitepapers/turn-data-into-actionable-information-with-red-hat-jboss-middleware