1 HP BSM Connector for Apache Ambari 1.0
HP BSM Connector for Apache Ambari For the Windows ® and Linux operating systems
Software Version: 01.00
Integration Guide
Document Release Date: Nov 2014
Software Release Date: Nov 2014
2 HP BSM Connector for Apache Ambari 1.0
Legal Notices
Warranty The only warranties for HP products and services are set forth in the express warranty statements
accompanying such products and services. Nothing herein should be construed as constituting an
additional warranty. HP shall not be liable for technical or editorial errors or omissions contained
herein.
The information contained herein is subject to change without notice.
Restricted Rights Legend Confidential computer software. Valid license from HP required for possession, use or copying.
Consistent with FAR 12.211 and 12.212, Commercial Computer Software, Computer Software
Documentation, and Technical Data for Commercial Items are licensed to the U.S. Government
under vendor's standard commercial license.
Copyright Notice © Copyright 2014 Hewlett-Packard Development Company, L.P.
Trademark Notices Microsoft® and Windows® are U.S. registered trademarks of Microsoft group of companies.
Apache is a registered trademark of Apache Software Foundation and/or its affiliates.
3 HP BSM Connector for Apache Ambari 1.0
Documentation Updates
The title page of this document contains the following identifying information:
Software Version number, which indicates the software version.
Document Release Date, which changes each time the document is updated.
Software Release Date, which indicates the release date of this version of the software.
To check for recent updates or to verify that you are using the most recent edition of a document,
go to:
https://hpln.hp.com/contentoffering/bsm-connector-apache-ambari
This site requires that you register for an HP Passport and sign in. To register for an HP Passport
ID, go to:
http://h20229.www2.hp.com/passport-registration.html
Or click the New users - please register link on the HP Passport login page.
4 HP BSM Connector for Apache Ambari 1.0
Support
This document describes HP Live Network Community Content. Hewlett Packard Software does
not test, certify or support Community Content. Please use the Community Content discussion
forums on HP Live Network or contact the original submitter of the Community Content directly
if any support, configuration or usability questions arise. Please back up all appropriate files
before applying Community Content in your environment.
5 HP BSM Connector for Apache Ambari 1.0
Contents
Chapter 1 - Introduction ............................................................................................................................ 6
Chapter 2 - Installation and Initial Configuration................................................................................... 9
Installation Prerequisites ........................................................................................................................ 9
Installation ............................................................................................................................................. 10
Importing Hadoop Model into BSM ................................................................................................... 11
Topology Integration ............................................................................................................................ 11
Event Integration .................................................................................................................................. 13
Chapter 3 – Configuration ....................................................................................................................... 15
Connection Settings using the configuration script ........................................................................... 15
Connection Settings using manual configuration .............................................................................. 15
Other configuration parameters .......................................................................................................... 16
Chapter 4 – Manage the Apache Ambari event and topology integration .......................................... 18
Chapter 5 – Uninstallation ....................................................................................................................... 19
Uninstall the collectors ......................................................................................................................... 19
Erase User Credentials ......................................................................................................................... 19
Delete Directories .................................................................................................................................. 19
Uninstall Generic Integrator ................................................................................................................ 20
Delete policies ........................................................................................................................................ 20
Chapter 6 – Troubleshooting ................................................................................................................... 21
Log files .................................................................................................................................................. 21
Increase the verbosity of the log output .............................................................................................. 21
Output Files ........................................................................................................................................... 21
6 HP BSM Connector for Apache Ambari 1.0
Chapter 1 - Introduction
HP Business Service Management Connector for Apache Ambari enables you to establish a link
between Apache Ambari environments and HP Business Service Management (BSM). The BSM
Connector for Apache Ambari is an add-on for the HP BSM Connector.
The BSM Connector for Apache Ambari comprises of the integration components and policies
that provide the following Apache Ambari specific functionality:
Topology forwarding to BSM
Event forwarding to BSM
This functionality enables you to consolidate management data from Apache Ambari
environments into an HP BSM Operations Management solution. The BSM Connector for
Apache Ambari provides integration for the following Apache Ambari versions:
Apache Ambari 1.5
Apache Ambari 1.6
7 HP BSM Connector for Apache Ambari 1.0
Apache Ambari Event Integration Overview
The BSM Connector for Apache Ambari requires an HP BSM Connector installation. HP
recommends that you install the Apache Ambari and HP BSM Connector on separate computers.
The following figure shows an overview the Apache Ambari integration scenario.
Apache Ambari facilitates the management and monitoring of the resources of an Apache
Hadoop cluster via REST based web service interfaces. These interfaces can be used to integrate
Apache Ambari with enterprise frameworks such as BSM. Using the REST APIs, we can
forward alerts from Apache Ambari to BSM Connector endpoint. BSM Connector for Apache
Ambari reports these new events and event updates into BSM.
8 HP BSM Connector for Apache Ambari 1.0
Apache Ambari Topology Integration Overview
The BSM Connector for Apache Ambari uses a topology-only custom policy to reflect the
Apache Ambari topology into BSM.
The BSM Connector runtime queries the REST web services exposed by Apache Ambari to
identify the Apache Hadoop Cluster topology. The discovered topology is then synced into
BSM.
9 HP BSM Connector for Apache Ambari 1.0
Chapter 2 - Installation and Initial Configuration
The BSM Connector for Apache Ambari can be downloaded from HP Live Network. The BSM
Connector for Apache Ambari installation package is installed on the BSM Connector host
system.
Installation and initial configuration of the BSM Connector for Apache Ambari involves the
following high-level steps, which are described in detail below:
1. Install HP BSM Connector 9.21 or higher.
The BSM Connector for Apache Ambari requires an HP BSM Connector 9.21 or higher
installation on a supported Windows or Linux operating system. Install the BSM
Connector Software before you install the BSM Connector for Apache Ambari. The BSM
Connector must be able to connect to the computer running HP Business Service
Management version 9.2x or higher and must be integrated into BSM.
2. Install the BSM Connector for Apache Ambari. For details, see "Installation" on page 10.
3. Import the Hadoop model into BSM. For details, see “Importing Hadoop Model into
BSM” on page 11.
4. Integrate topology by importing the custom topology policy into BSM Connector,
configure it for your Apache Ambari installation, and activate the policy. For details, see
"Topology Integration" on page 11.
5. Integrate events by importing XML Event policy into BSM Connector, configure it for
your Apache Ambari installation, and activate the policy. For details, see "Event
Integration" on page 13.
Installation Prerequisites
The following section lists the software prerequisites for installing BSM Connector for Apache
Ambari on Linux and Windows BSMC servers.
Software Version
HP BSM/OMi HP BSM/OMi Version 9.2x or higher
HP BSM Connector
Software
HP BSM Connector Version 9.21 or higher
Note about the BSMC Generic Integrator Component
The BSM Connector for Apache Ambari uses version 1.40.011 of
the HP Generic Integrator component, which is compatible with any
10 HP BSM Connector for Apache Ambari 1.0
other connectors you may have that also use the HP Generic
Integrator.
Note the following with regard to the case of a previous installation
of the Generic Integrator by another Generic Integrator-based
integration on the server hosting the BSM Connector:
If version 1.40 of the Generic Integrator was installed on the
BSM Connector server, the installation wizard skips the
installation of the component.
If an older version of the Generic Integrator was installed,
the component is updated to version 1.40. Version 1.40 is
backward compatible with older versions of the component,
so existing integrations are not affected by the update.
If the Generic Integrator was not installed on the BSM
Connector server, the installation wizard installs version 1.40
of the Generic Integrator.
HP BSM Connector OS
Windows Server 2008 64bit
Windows Server 2008 R2 64 bit
RedHat Enterprise Linux 5.x 64bit
Apache Ambari Apache Ambari 1.5
Apache Ambari 1.6
Installation Install the BSM Connector for Apache Ambari on the same computer as the BSM Connector as
follows:
On Windows:
1. Extract the BSM Connector for Apache Ambari package
“HPBsmIntAmbari_01.00.000_Win.zip” to a temporary location.
2. Open a command prompt and execute the installation script “install.bat” that comes with
the integration package.
install.bat
On Linux:
1. Extract the BSM Connector for Apache Ambari package
“HPBsmIntAmbari_01.00.000_Linux.zip” to a temporary location.
11 HP BSM Connector for Apache Ambari 1.0
2. Open a command prompt and execute the installation script “install.sh” that comes with
the integration package.
./install.sh
Importing Hadoop Model into BSM The BSM connector package includes the uCMDB model for Apache Hadoop Cluster that needs
to be imported into BSM. The steps to import the model into BSM is as follows:
1. As part of the connector installation the uCMDB model for Hadoop Cluster is copied into
the following locations:
On Windows:
<OvDataDir>/installation/HPBsmIntAmbari/ucmdb/HPBsmIntAmbari-
Hadoop.zip
On Linux:
/var/opt/OV/installation/HPBsmIntAmbari/ucmdb/HPBsmIntAmbari-Hadoop.zip
2. Copy the uCMDB package “HPBsmIntAmbari-Hadoop.zip” to a temporary location
<TEMP> on the BSM Server
3. To import the uCMDB package into BSM using “Package Manager” follow below steps:
a) In BSM console Click Admin > RTSM Administrator > Administration >
Package Manager.
b) Click . The Deploy Package for Server dialog box appears.
c) Click to upload the package. Browse to the following location and select the
<TEMP>/HPBsmIntAmbari-Hadoop.zip.
d) Click Open. The package will be loaded with a time delay.
e) Click Deploy
Topology Integration To integrate Apache Ambari topology into BSM, complete the following steps on your BSM
Connector host system:
1. Open the BSM Connector interface and verify if the custom topology policy “HP BSM
Connector for Apache Ambari – Topology Integration” is already imported. If the
policy is already imported then skip step 2.
2. If the policy “HP BSM Connector for Apache Ambari – Topology Integration” is not
already imported then import it using the BSM Connector interface from the below
location:
12 HP BSM Connector for Apache Ambari 1.0
On Windows:
<OvDataDir>/installation/HPBsmIntAmbari/policies/3ec805c6-ffda-42f4-ae20-
2d703413db6d_data
< OvDataDir>/installation/HPBsmIntAmbari/policies/3ec805c6-ffda-42f4-ae20-
2d703413db6d_header.xml
On Linux:
/var/opt/OV/installation/HPBsmIntAmbari/policies/3ec805c6-ffda-42f4-ae20-
2d703413db6d_data
/var/opt/OV/installation/HPBsmIntAmbari/policies/3ec805c6-ffda-42f4-ae20-
2d703413db6d_header.xml
Note: Skip steps 3 to 5 if the configuration details were already specified as part of the
installation
3. Verify the correct Apache Ambari hostname and port number is specified for the
variables “ambari-hostname” and “ambari-port” in the below configuration file:
On Windows:
<OvDataDir>/conf/HPBsmIntAmbari/genint/topology_integration.conf
On Linux:
/var/opt/OV/conf/HPBsmIntAmbari/genint/topology_integration.conf
Note: If the port number is not specified then BSM Connector for Apache Ambari uses
8080 as the default port.
4. Save your changes to the configuration file.
5. REST APIs exposed by Apache Ambari requires basic authentication. Hence to enable
the integration user needs to specify credentials of an Apache Ambari user.
The credentials can be configured with the following commands:
On Windows:
<OvInstallDir>/bin/win64/genintcrypt" –generate
(Note: This will re-initialize the credential store and all existing data gets lost)
<OvInstallDir>/bin/win64/genintcrypt --set ambari-user
<OvInstallDir>/bin/win64/genintcrypt --set ambari-password
On Linux:
/opt/OV/bin64/genintcrypt --generate
(Note: This will re-initialize the credential store and all existing data gets lost)
/opt/OV/bin64/genintcrypt --set ambari-user
13 HP BSM Connector for Apache Ambari 1.0
/opt/OV/bin64/genintcrypt --set ambari-password
Each of the commands will ask you to enter a value. The typed characters won't be
visible on the screen. Hit the ENTER key to store the entered value. The user credentials
will be stored in an encrypted file and won't be visible in plain text at any time.
6. Using the BSM Connector interface activate the policy “HP BSM Connector for Apache
Ambari – Topology Integration” to enable topology integration for Apache Ambari.
Event Integration To integrate Apache Ambari events into BSM, complete the following steps on your BSM
Connector host system:
1. Open the BSM Connector interface and verify if the custom topology policy “HP BSM
Connector for Apache Ambari – Event Integration” is already imported. If the policy
is already imported then skip step 2.
2. If the policy “HP BSM Connector for Apache Ambari – Event Integration” is not
already imported then import it using the BSM Connector interface from the below
location:
On Windows:
<OvDataDir>/installation/HPBsmIntAmbari/policies/233c23e1-bbc1-44b7-afad-
357d14a65ee5_data
<OvDataDir>/installation/HPBsmIntAmbari/policies/233c23e1-bbc1-44b7-afad-
357d14a65ee5_header.xml
On Linux:
/var/opt/OV/installation/HPBsmIntAmbari/policies/233c23e1-bbc1-44b7-afad-
357d14a65ee5_data
/var/opt/OV/installation/HPBsmIntAmbari/policies/233c23e1-bbc1-44b7-afad-
357d14a65ee5_header.xml
3. Verify the correct Apache Ambari hostname and port number is specified for the
variables “ambari-hostname” and “ambari-port” in the below configuration file:
On Windows:
<OvDataDir>/conf/HPBsmIntAmbari/genint/event_integration.conf
On Linux:
/var/opt/OV/conf/HPBsmIntAmbari/genint/event_integration.conf
Note: If the port number is not specified then BSM Connector for Apache Ambari uses
8080 as the default port.
14 HP BSM Connector for Apache Ambari 1.0
4. Save your changes to the configuration file.
Note: Skip step 5 if user credential was already specified as part of installation.
5. REST APIs exposed by Apache Ambari requires basic authentication. Hence to enable
the integration user needs to specify credentials of an Apache Ambari user.
The credentials can be configured with the following commands:
On Windows:
<OvInstallDir>/bin/win64/genintcrypt" –generate
(Note: This will re-initialize the credential store and all existing data gets lost)
<OvInstallDir>/bin/win64/genintcrypt --set ambari-user
<OvInstallDir>/bin/win64/genintcrypt --set ambari-password
On Linux:
/opt/OV/bin64/genintcrypt --generate
(Note: This will re-initialize the credential store and all existing data gets lost)
/opt/OV/bin64/genintcrypt --set ambari-user
/opt/OV/bin64/genintcrypt --set ambari-password
Each of the commands will ask you to enter a value. The typed characters won't be
visible on the screen. Hit the ENTER key to store the entered value. The user credentials
will be stored in an encrypted file and won't be visible in plain text at any time.
6. Using the BSM Connector interface activate the policy “HP BSM Connector for Apache
Ambari – Event Integration” to enable event integration for Apache Ambari
15 HP BSM Connector for Apache Ambari 1.0
Chapter 3 – Configuration
The following section describes various configuration settings of BSM Connector for Apache
Ambari.
Connection Settings using the configuration script
The integration comes with a basic configuration script which can be used to configure the
connection settings to the Apache Ambari server. The script will be executed automatically after
installing the integration assets.
It is possible to re-run the basic configuration script at any time using the following command:
On Windows:
<OvDataDir>/installation/HPBsmIntAmbari/configure.pl
On Linux:
/var/opt/OV/installation/HPBsmIntAmbari/configure.pl
The script will first ask for the following three inputs:
Apache Ambari hostname or IP address
Apache Ambari user name
Password for the specified user
Note: The user credentials will be stored in an encrypted file and won't be visible in plain text at
any time.
Connection Settings using manual configuration
Follow the below mentioned steps to manually configure the connection settings to the Apache
Ambari Server.
1. Configure the Apache Ambari hostname and port number for the variables “ambari-
hostname” and “ambari-port” in the below configuration file:
On Windows:
<OvDataDir>/conf/HPBsmIntAmbari/genint/event_integration.conf
<OvDataDir>/conf/HPBsmIntAmbari/genint/topology_integration.conf
On Linux:
/var/opt/OV/conf/HPBsmIntAmbari/genint/event_integration.conf
/var/opt/OV/conf/HPBsmIntAmbari/genint/topology_integration.conf
2. Save your changes to the configuration file.
16 HP BSM Connector for Apache Ambari 1.0
3. The credentials can be configured with the following commands:
On Windows:
<OvInstallDir>/bin/win64/genintcrypt" –generate
(Note: This will re-initialize the credential store and all existing data gets lost)
<OvInstallDir>/bin/win64/genintcrypt --set ambari-user
<OvInstallDir>/bin/win64/genintcrypt --set ambari-password
On Linux:
/opt/OV/bin64/genintcrypt --generate
(Note: This will re-initialize the credential store and all existing data gets lost)
/opt/OV/bin64/genintcrypt --set ambari-user
/opt/OV/bin64/genintcrypt --set ambari-password
Each of the commands will ask you to enter a value. The typed characters won't be
visible on the screen. Hit the ENTER key to store the entered value. The user credentials
will be stored in an encrypted file and won't be visible in plain text at any time
Other configuration parameters
Following are other optional configuration parameters of the BSM Connector for Apache
Ambari. These parameters are defined in the following configuration files:
On Windows:
<OvDataDir>/conf/HPBsmIntAmbari/genint/event_integration.conf
<OvDataDir>/conf/HPBsmIntAmbari/genint/topology_integration.conf
On Linux:
/var/opt/OV/conf/HPBsmIntAmbari/genint/event_integration.conf
/var/opt/OV/conf/HPBsmIntAmbari/genint/topology_integration.conf
1. Interval - Interval between the event (in seconds) and topology (in minutes) polling
operations. Default value is 45 seconds for events and 60 minutes for topology. The BSM
Connector for Apache Ambari polls for new or updated events every 45 seconds and for
topology every 60 minutes. This can be configured by updating the below setting.
"ambari-interval" : "45"
2. Port number - Port number, used for the connection to the Apache Ambari REST web-
service. Default is "8080”. Specify appropriate port number configured for the Apache
Ambari.
"ambari-port" : "443"
17 HP BSM Connector for Apache Ambari 1.0
3. Timestamp persistency file - Location of the timestamp persistency file. After each
successful run, the integration stores the timestamp into this file. On restart of the integration
daemon, the timestamp will be restored using the content of the file.
On Windows:
“ambari-timestamp-file": “<OvDataDir>/tmp/HPBsmIntAmbari/ambari-alerts.timestamp”
On Linux:
“ambari-timestamp-file": "/var/opt/OV/tmp/HPBsmIntAmbari/ambari-alerts.timestamp"
18 HP BSM Connector for Apache Ambari 1.0
Chapter 4 – Manage the Apache Ambari event
and topology integration
The integration daemon can be started and stopped using the OV control (ovc) service
management tool.
On Windows:
ovc -start AMB
ovc -restart AMB
ovc -stop AMB
ovc -status AMB
On Linux: /opt/OV/bin/ovc -start AMB
/opt/OV/bin/ovc -restart AMB
/opt/OV/bin/ovc -stop AMB
/opt/OV/bin/ovc -status AMB
To control individual collector use the above commands in a following example:
Event Collector
The collector responsible for event integration is named “ambarie”. Use below command to
control it.
On Windows:
ovc -start ambarie
ovc -stop ambarie
ovc -status ambarie
On Linux: /opt/OV/bin/ovc -start ambarie
/opt/OV/bin/ovc -stop ambarie
/opt/OV/bin/ovc -status ambarie
Topology Collector
The collector responsible for topology integration is named “ambarit”. Use below command to
control it.
On Windows:
ovc -start ambarit
ovc -stop ambarit
ovc -status ambarit
On Linux: /opt/OV/bin/ovc -start ambarit
/opt/OV/bin/ovc -stop ambarit
/opt/OV/bin/ovc -status ambarit
19 HP BSM Connector for Apache Ambari 1.0
Chapter 5 – Uninstallation
Uninstalling the BSM Connector for Apache Ambari requires you to complete the steps
appropriate for your host operating system
Uninstall the collectors
On Windows: ovc -stop AMB
ovcreg -del ambarie
ovcreg -del ambarit
On Linux: /opt/OV/bin/ovc -stop AMB
/opt/OV/bin/ovcreg -del ambarie
/opt/OV/bin/ovcreg -del ambarit
Erase User Credentials
Run the following commands and press ENTER when prompted for input without passing any
value.
On Windows:
<OvInstallDir>/bin/win64/genintcrypt --set ambari-user
<OvInstallDir>/bin/win64/genintcrypt --set ambari-password
On Linux: /opt/OV/bin64/genintcrypt --set ambari-user
/opt/OV/bin64/genintcrypt --set ambari-password
Delete Directories
Delete the directories using the below commands.
Note: Deleting the directories is optional.
On Windows: rd <HPBsmIntAmbari extraction directory> /s /q
rd "%OvDataDir%conf\HPBsmIntAmbari" /s /q
rd "%OvDataDir%datafiles\HPBsmIntAmbari" /s /q
rd "%BSMC_HOME%\conf\topology\xml\sync-packages\HPBsmIntAmbari" /s /q
rd "%OvDataDir%installation\HPBsmIntAmbari" /s /q
rd "%OvDataDir%tmp\HPBsmIntAmbari" /s /q
On Linux: rm -Rf < HPBsmIntAmbari extraction directory>
rm -Rf /var/opt/OV/conf/HPBsmIntAmbari
rm -Rf /var/opt/OV/datafiles/HPBsmIntAmbari
20 HP BSM Connector for Apache Ambari 1.0
rm -Rf /opt/HP/BSMConnector/conf/topology/xml/sync-packages/HPBsmIntAmbari
rm -Rf /var/opt/OV/installation/HPBsmIntAmbari
rm -Rf /var/opt/OV/tmp/HPBsmIntAmbari
Uninstall Generic Integrator
Uninstall the Generic Integrator by following the below steps.
Note: This is an optional step. Uninstall the "Generic Integrator" only if no other software
depends on it.
On Windows:
Open Control Panel and click on "Uninstall a program" under "Programs"
Find "HP BSM Integrations - Generic Integrator"
Right click it and select "Uninstall"
Confirm by selecting "Yes" to Uninstall
On Linux:
Run the below command:
rpm -e HPBsmIntGenint-1.40.011
Delete policies
Delete the policies installed by the BSM connector by following the below steps.
Note: This is an optional step. Skip this step if you wish to preserve possible changes made to
the policy.
Open the BSM Connector user interface and select the policy you wish to delete
Deactivate it by clicking on the "Deactivate" icon in the tool bar.
Click on the "Delete" icon in the tool bar.
Confirm by selecting "Yes" to delete the policy.
21 HP BSM Connector for Apache Ambari 1.0
Chapter 6 – Troubleshooting
Log files Check for errors in the integration daemon's log files:
On Windows:
For Events: %OvDataDir%log\HPBsmIntAmbari-Event.<date>.log
For Topology: %OvDataDir%log\HPBsmIntAmbari-Topology.<date>.log
On Linux:
For Events: /var/opt/OV/log/HPBsmIntAmbari-Event.<date>.log
For Topology: /var/opt/OV/log/HPBsmIntAmbari-Topology.<date>.log
Increase the verbosity of the log output Follow the below steps to increase the verbosity of the log output.
Open the integration configuration file
On Windows:
%OvDataDir%conf\HPBsmIntAmbari\genint\event_integration.conf
%OvDataDir%conf\HPBsmIntAmbari\genint\topology_integration.conf
On Linux:
/var/opt/OV/conf/HPBsmIntAmbari/genint/event_integration.conf
/var/opt/OV/conf/HPBsmIntAmbari/genint/topology_integration.conf
In the first section “genintd” remove the comment from the line “//,'log-level' : '3'”. This
sets the “log-level” parameter to 3. The "log-level" parameter is used for logging
verbosity customization. If the parameter is kept commented out or is deleted, a default
level 1 is used. Valid log levels are 0-3. Setting log-level to 0 (zero) disables log output.
Restart the integration daemon
Ensure to lower the log level again, when done with troubleshooting
Output Files Check for event and topology data being received and written into the xml output files:
On Windows:
%OvDataDir%datafiles\HPBsmIntAmbari\ambari-alerts.xml
%OvDataDir%datafiles\HPBsmIntAmbari\ambari-topology.xml
On Linux:
/var/opt/OV/datafiles/HPBsmIntAmbari/ambari-alerts.xml
/var/opt/OV/ datafiles/HPBsmIntAmbari/ambari-topology.xml