Leveraging and Integrating Existing Data in your CMDB ARSII-304
Alan AbramsXinify Technologies
The Ingredients To Success
› Data Structure Knowledge› External Data from Discovery/Inventory› Enterprise Integration Engine
– Remedy Link for Oracle– Remedy Link for MS-SQL– Flat File– Custom
› Atrium CMDB› Reconciliation Rules› Workflow Filters – if Needed
External Data Source
›Database data›Flat file data›Source Relationships can be stored as
– foreign key– relationship tables
Constantly changing/updating data may lead to inconsistent results.Plan your data transfers during a time when source data is in a steady state.
Enterprise Integration Engine
› EIE 7.0– Flat File– Custom– read/write AR System forms– CMDB Aware
• Configuration Items• Relationships
› Remedy Link for Oracle– Contains EIE 7.0 + service to read/write Oracle tables
› Remedy Link for MS-SQL– Contains EIE 7.0 + service to read/write MS-SQL tables
Remedy Link for MS-SQL Method
SQL Server
3rd Party Data
MS SQLServer
Remedy Link for MS SQL
Adapter
AR System Server
EIE
EIE Mapping Rules
Data ExchangeRule Helper
•MS-SQL Adapter•Pre-built adapter to SQL Databases
•Rule Helper•Uses the adapter to create vendor table and field name menus
•Data Exchange Application (DEA)•AR System based application at the heart of EIE
•EIE Service•DEA passes the service instructions on how to interact with the adaptor
Atrium CMDB
› Configuration Management Database– Store multiple datasets– Reconcile datasets– Store Relationships– Base data for OOTB Applications
› Data Sets– Data value in a field... but so much more– Data Sets you should use
• BMC.ASSET – The “Gold Standard”• BMC.SANDBOX – Temp location for GUI changes to Assets• VENDOR.PURPOSE.PRODUCT – custom datasets
– Examples: BMC.IMPORT.MARIMBA, MS.IMPORT.SMS, ACME.IMPORT.LEGACY
Using Sandbox
AR System Server
CMDB Master
Data Set
› Submit New/Update to Sandbox Data Set
› Run Reconciliation Job on Save– Data is Merged if allowed– Sandbox Entry is deleted
CMDB SandboxData Set
› Enable Use Sandbox Data Set
CMDB Reconciliation
Engine
CMDB Reconciliation
Rules
› Create Rule/Job to Reconcile Sandbox and BMC.ASSET
Remedy Asset Mgmt
› Query Data From Master Data Set
Optional Workflow
AR System Server
CMDB 3rd PartyDataset
Data Cleansing Workflow
› Do manual compare of data to determine need› Normalize \ Standardize
– KB vs MB vs GB– Tangerines vs Oranges – close but no prize
› Categorize› ...Get the square peg in the round hole› Fire on Merge and maybe Modify
Overview
AR System Server
SQL Server Example
3rd Party Data
MS SQLServer
Remedy Link for MS SQL
Adaptor
EIECMDB 3rd PartyDataset
CMDB SandboxDataset
Remedy Asset Mgmt
CMDB Reconciliation
Engine
CMDB Reconciliation
Rules
EIE Mapping Rules
Data Cleansing Workflow
CMDB MasterDataset
EIE Structure Simple
Data Exchange
Data Mapping
1-N
Relationship Exchange
Relationship Mapping
1-N
On Complete, Trigger Next
EIE Structure Complex
Data MappingComputer
Data Exchange
Relationship Exchange
Relationship MappingComputer – O/S
Data MappingComputer
Data MappingComputer
Data Exchange
Data MappingOperating System
Data Exchange
Data MappingDrives
Data Exchange
Data MappingMonitor
Data Exchange
Data MappingSoftware
Trigger Next
Relationship Exchange
Relationship MappingComputer - Drives
Relationship Exchange
Relationship MappingComputer - Monitor
Relationship Exchange
Relationship MappingComputer - Software
Trigger Next Trigger Next Trigger Next Trigger Next
How To – Rule Helper
› Populates the “vendor” table and column menus› Command Line driven
– <eie_install_dir>\rl_sql\bin\rulehelper.exe– example – C:\Program Files\Remedy\Apps\EIE\servername\rl_sql\bin\rulehelper.exe
› Run before you start any mapping work
Must Use Administrator
Login
How To – Rule Helper
1. Load 2. Add Desired Tables 3.Create Menus
How To – Data Mapping
VEN-CompSys to BMC:Comp
› Data Mapping Name = name for new mapping› Namespace = BMC.CORE› Class = Destination for data, pick from the list.› Vendor Application = SqlServer, Flatfile or Oracle› Vendor Table/View = Source of data (pick from list)
– if not in the list, run rulehelper.exe to create the menu items.› Direction = Vendor Data into CMDB.› Click Create Data Mapping.
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
If the vendor table/view does not show in the drop down list, it is because you have not used rulehelper.exe to populate the tables that the menus query.
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
› Keys must be unique on both sides– If multiple keys, stored as AND to define Unique
› Use your DatasetID as well
How To – Data Mapping
All field mappings must include the following CMDB Data Fields –› Field used to identify Key › DatasetId› Name = anything that identifies the entry› AssetLifecycleStatus (3 is the enumeration for “deployed”)› Short Description › Submitter › ReconciliationIdentity
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
› Only works if you can write to the Vendor / Source Database› Can be any value› Usually is transfer status
– 0 = transfer with no error– 1 = error
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
› If you “Update Row to Indicate Delete”– Set AssetLifecycleStatus to 6 (End of Life)
› If you choose to “Delete” Row– No field to set. EIE will do the deletion
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
Use to limit amount of data in a given query.Important if you are experiencing performanceor memory issues
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
› Only need to use if updating based on some data criteria› In most cases this is blank
– See update features on Options tab Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
› Usually the only response is status function– if using “function|status{}” nothing needed here
› Only works if EIE can write to the vendor tableData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
› Use if a value in the vendor field determines need to delete› Not used if Delete is determined by absence of data in vendor table
– See Delete tab for what to do in this caseData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Data Mapping
1. If set to Yes, no records with source data errors are transferred.2. Is it OK to transfer blank spaces.. not the same as NULL3. Say “Yes” if importing to AR – Gets rid of trailing blank spaces4. Stores change history in a field you designate5. If Yes, updates are only made when a change is detected in the vendor data
› You must select a field that can store a negative integer to store the checksum6. Allows you to retrieve new or change data since last transfer
› You must select a date/time field to store last transfer timeData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
123456
How To - Data Exchange
› Schedule Only– Use Schedule tab
› Event Driven Only– run eiexfer.exe from command line – use in a filter or active link
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Data Exchange
› Select the mapping(s) to include in the Exchange container.› Multiple Mappings
– Cuts down on queries for source or target› Main Mapping
– Need to designate if multiple mappings included– Key values only come from Main Mapping
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Data Exchange
› IntervalData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
› Time› To Be Triggered
How To - Data Exchange
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
Alternate Method
How To - Data Exchange
› While Testing– Limit records so you have less to undo
› In Production – Set Enable Debug to No
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Data Exchange
› Use if the AR Server involved in the exchange is NOT the server that EIE DEA is installed on.
› Advantage – 1 license can service Dev, Test & Production Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To – Relationship Mapping
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
Data Mapping
Data Mapping
Vendor Relationship Mapping
V_Computer
V_DiskDrive
Vendor Relationship Table
BMC_ComputerSystem
BMC_DiskDrive
BMC_HostedSystemComponent
•Map using Vendor Relationship Mapping.•Trigger on Complete of Data Exchange.
How To – Relationship Mapping
Data Mapping
Class Relationship Mapping
*In this case, the relationship dataexchange only runs after the data transfer for the two configuration item classes has completed.
•Map using Class Relationship Mapping.•Put in an Event Driven Only Exchange*.
V_Computer
V_DiskDrive
BMC_ComputerSystem
BMC_DiskDrive
BMC_HostedSystemComponent
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
Data Mapping
How To - Relationship Mapping
› Vendor Data into CMDB is the only valid Direction› Must Select Vendor or Class on Create Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› Define matching criteria that defines the relationship› Class 1 = Source› Class 2 = Destination
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› Always Two Keys– Source.InstanceId– Destination.InstanceID
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› These 6 attributes are always required› For DatasetID, select either class to create the
constantData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› Use if you are doing Class based mappingData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› Use if you are doing Class based mappingData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› This is a useless tab› Relationship is always the target of the
mapping› There is nothing to query
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
How To - Relationship Mapping
› Used only if a Vendor based mappingData Exchange
Data Mapping Relationship Exchange
Relationship Mapping
The Result of a Relationship Mapping
How To - Relationship Exchange
› Works the same as Data Exchange› Choose Relationship Exchange = Yes
Data Exchange
Data Mapping Relationship Exchange
Relationship Mapping
General EIE Hints
› When testing an exchange –– Test one exchange at a time (when done chain them to be triggered)– Set to Interval – 20 minutes– Stop and start the EIE service to cause it to run immediately– Set Limit = 1 (so that it only runs once)– Be sure you know your debug path
› When writing a Vendor Query –– Test it with Direct SQL – If it doesn’t work there, it won’t work here– Use Indexed fields if possible
This won’t work – MacAdress != NULLThis will – MacAdress IS NOT NULL
General EIE Hints
› When setting an AR System field in a mapping –– Doesn’t work like a setfield or pushfield operation– Must Use Rules to convert functions, keywords, etc.
Example of an Integer FieldThis won’t work – 0This will – Constant|0
Example of ConcatenationThis won’t work – $Manufacturer0$+”-”+$MediaType0$This will – function|concat($Manufacturer0$,”-”,$MediaType0$)
› Change ALL EIE Query Menus to On Open, not On Connect
Atrium CMDB
› Reconciliation Rules– Identify
• Look for potential duplicates– Compare– Merge– Cleanup / Purge
• Can delete entire data set from all tables• Can delete selected items
– Configuration Item Set field MarkedForDelete = Yes– Create Reconciliation Rule to Purge if field is set
» Define how to handle related items– Run as a Reconciliation Job
CMDB 3rd PartyDataset
CMDB Reconciliation
Engine
CMDB Reconciliation
Rules
CMDB MasterDataset