Date post: | 22-Nov-2014 |
Category: |
Documents |
Upload: | amit-sharma |
View: | 477 times |
Download: | 37 times |
Session -I
Oracle Data Integrator
Introduction
Created By : Gourav Atalkar
Reviewed By: Amit Sharma
Contact Point :bispsolutions.wordpress.com
Agenda
What is Oracle Data Integrator?Why Oracle Data Integrator?Overview of ODI 11g ArchitectureOverview of ODI 11g ComponentsAbout Graphical ModulesTypes of ODI AgentsOverview of Oracle Data Integrator Repositories
Why Oracle Data Integrator ?
ELT Architecture provides high performance.Active integration enables real time data warehousing and operational data hubs.Declarative design improves developer productivity.Knowledge modules provides flexibility and extensibility.
ODI combines three style of data integration: data based, event based and service basedODI shortens implementation times with its declarative design approach.
Conventional Integration process: ETL
Source
Order
Correction files
Integration Process
Transform
Extract Transform Load
Target
Errors
Sales
Machine Machine
Machine
ELT
1) Extract: Extracting the data from various sources.2) Load: Loading the data into the destinations target.3) Transform: Transforming data according to a set of business rules.
ExtractExtractLoadLoad
Conventional ETL Process Next Generation ELT Architecture
ExtractExtractLoadLoad
TransformTransform
Oracle Data Integrator Architecture
Master Repository Work
Repository
Repositories Sources and Targets
Legacy Files DBMS
JVM
Run-time WS
Stand Alone Agent
Desktop
JVM ODI Studio
DesignerDesigner
OperatorOperator
TopologyTopology
SecuritySecurity
Java EE Applicatio
n
Java EE Applicatio
n
ODI SDKODI SDK
Web Logic 11g/Application Server
FMW Console
Data Sources Connection Pool
Servlet ContainerJava EE
Application
Java EE Applicatio
nODI SDKODI SDK
ODI Console
Run-time WS
Stand Alone Agent
Web Service Container
Public WS
Data Servi
ce
ODI Plug-inODI Plug-in
Oracle Data Integrator Components
ODI Studio Components:
Designer NavigatorOperator NavigatorTopology NavigatorSecurity Navigator
ODI Agent ODI ConsoleODI Repositories
Oracle Data Integrator Studio
Repository
Designer Navigator:Reverse EngineeringDevelop ProjectsRelease Scenario
Operator Navigator:Operate ProductionMonitor sessions
Operator Navigator:Define the infrastructure of the IS.
Security Navigator:Manage user privileges.
Designer Navigator
Designer Navigator
Designer Navigator is the GUI for defining metadata and rules for transformation and data quality. It uses this information to generate scenarios for production, and designer is where all the project development take place. Designer navigator is the core module for developers and metadata administrators.
Designer handles the following:
Models: Description of the data and application structure.Projects: The developments of various ODI objects.
Operator Navigator
ToolbarToolbar
In Operator navigator you can monitor execution
of ODI objects
In Operator navigator you can monitor execution
of ODI objects
This monitoring sessions steps enables you to
perform debugging
This monitoring sessions steps enables you to
perform debugging
Operator Navigator
Operator navigator is used to manage and monitor ODI in production. It is designed for production operators and displays the execution logs with error counts, the number of rows processed, execution statistics, and so on. At design time, developers use operator navigator for debugging purpose.
Through the Operator navigator, you can manage your interface execution in the sessions, as well as the scenario in the production.
The operator navigator stores this information in a work repository while using the topology define in the master repository.
Topology Navigator
ToolbarToolbar
Technology tree
Technology tree
In ODI Topology manager you set the architecture of your information system, technologies server information, schemas and repositories.
In ODI Topology manager you set the architecture of your information system, technologies server information, schemas and repositories.
Workspace to define Topology
objects
Workspace to define Topology
objects
Topology Navigator
Topology Navigator manages the physical and logical architecture of the infrastructure. Server, schemas, and agents are registered in the ODI Master Repository-a major ODI component that contains information about the topology of the company’s IT resources, security, and ODI resource.
Using the topology navigator, you can define the topology of your information system to ODI so that it can be accessed by other ODI modules. Topology navigator enables you to manage repositories. The topology navigator store this information in a master repository.
Security Navigator
Object TreeObject Tree
In security navigator , you manage objects, profiles, users and their
privileges and hosts.
In security navigator , you manage objects, profiles, users and their
privileges and hosts.
Workspace to define objects, profiles, users and hosts.
Workspace to define objects, profiles, users and hosts.
Security Navigator
Security Navigator manages users and their privileges in ODI. It is used to create profiles and provide rights to users to access ODI objects and features. This navigator is usually used by security administrators.
It is used to assign user rights for method on generic objects and to fine tune these rights on the objects instances.
What is an Agent ?
An agent is a run time component of ODI that orchestrates the integration process.
It is a lightweight java program that retrieves code from the repository at the run time.
At design time developer generate scenario from the business rule that they have designed. The code of these scenario is then retrieved from the from the repository by the agent at run time.
This agent then connects to the data servers and orchestrates the code execution on these servers.
Oracle data Integrator Agent
Agents are lightweight java processes that orchestrates the execution of objects at run time.Agent can do one of the following:-Execute objects on demand-Execute according to predefined schedules.
Oracle data Integrator Agent
Two Types of agents: Java EE and standalone
The agents comes in two different types:
The java Enterprise Edition(Java EE; formerly J2EE) agent can be deployed as a web application and benefit from the feature of an application server.
The standalone agent runs in a simple java machine and can be deployed where needed to perform the integration flow.
Both agents are multithreaded java programs that support load balancing and can be distributed across the information system.
Using the Two Types of agents
Deploying a Java EE agent in a Java application server(Oracle WebLogic Server):1)In ODI, define the Java EE agent in the Topology Navigator.2)In ODI, create the WLS template for the Java EE Agent.3)Deploy template directly using WLS configuration Wizard.Using a stand alone Agent:1)Launch an agent.2)Display scheduling information.3)Stop the agent.Advantages of Java EE agents over Standalone agents:1)High Availability2)Multiple agents, using Coherence.
Example of standalone agent
Designer Navigator:Reverse EngineeringDevelop ProjectsRelease Scenario
Operator Navigator:Operate ProductionMonitor sessions
Repository
Standalone Agent
Information System
Java Any Platform
Java Any Platform
RDBMS
ODI Console
ODI Console provides web access to Repositories. It enables user to navigate projects, models, logs and so on. Business user, developer, Operators, and administrators use their Web browsers to access ODI Console. The ODI console replaces Metadata navigator of previous version.
Oracle Data Integrator Repositories
Two Types of repositories are included into the ODI.-Master Repository-Work Repository * Development Repository * Execution Repository
Work Repositories are always attached to a single Master repository
Master and Work Repositories
Security
Topology
Versioning
Model
Project
Execution
Execution
Master Repository
Execution RepositoryWork Repository(Development) (Production)
Master and Work Repositories
Master Repository: It is the data structure containing the information about the Topology of the company’s IT resources, security, and version management of projects and data models. This repository stored on a relational database accessible in client/server mode from the different ODI modules.
Work Repository: It is the data structure containing the information about the data model, projects and their use. This repository stored on a relational database accessible in client/server mode from the different ODI modules.
Question