Ff4s Sap Install

User Guide

FlexFrame for SAP 4.1AInstallation Guide for SAP Solutions

English

FlexFrame for SAP Version 4.1A

Installation Guide for SAP Solutions

Edition April 2008 Document Version 1.0

Fujitsu Siemens Computers GmbH

Copyright Fujitsu Siemens Computers GmbH 2008 FlexFrame, PRIMECLUSTER. PRIMEPOWER and PRIMERGY are trademarks of Fujitsu Siemens Computers SPARC64 is a registered trademark of Fujitsu Ltd. SAP and NetWeaver are trademarks or registered trademarks of SAP AG in Germany and in several other countries Linux is a registered trademark of Linus Torvalds SUSE Linux is a registered trademark of Novell, Inc., in the United States and other countries Java and Solaris are trademarks of Sun Microsystems, Inc. in the United States and other countries Intel and PXE are registered trademarks of Intel Corporation in the United States and other countries MaxDB is a registered trademark of MySQL AB, Sweden MySQL is a registered trademark of MySQL AB, Sweden NetApp, Network Appliance, Open Network Technology for Appliance Products, Write Anywhere File Layout and WAFL are trademarks or registered trademarks of Network Appilance, Inc. in the United States and other countries Oracle is a registered trademark of ORACLE Corporation EMC, CLARiiON, Symmetrix, PowerPath, Celerra and SnapSure are trademarks or registered trademarks of EMC Corporation in the United States and other countries SPARC is a trademark of SPARC International, Inc. in the United States and other countries Ethernet is a registered trademark of XEROX, Inc., Digital Equipment Corporation and Intel Corporation Windows, Excel and Word are registered trademarks of Microsoft Corporation All other hardware and software names used are trademarks of their respective companies. All rights, including rights of translation, reproduction by printing, copying or similar methods, in part or in whole, are reserved. Offenders will be liable for damages. All rights, including rights created by patent grant or registration of a utility model or design, are reserved. Delivery subject to availability. Right of technical modification reserved.


Contents 1 Introduction ..................................................................................................... 1 1.1 Purpose of this Document ................................................................................. 1 1.2 General Usage of SAP Installation Guide During Installation............................ 1 1.3 Overview ........................................................................................................... 1 1.4 Notational Conventions ..................................................................................... 2 1.5 Document History.............................................................................................. 2 1.6 Related Documents........................................................................................... 2 1.7 IP Addresses Used in this Document ................................................................ 3 1.8 Hotlinks for FlexFrame ...................................................................................... 4

2 Overview .......................................................................................................... 7 2.1 FlexFrame ......................................................................................................... 7 2.2 Virtualization of SAP Services........................................................................... 7 2.3 Preparations ...................................................................................................... 8 2.4 Gathering SAP Services Information................................................................. 8 2.5 FlexFrame Installation Diagram Step by Step ................................................... 9 2.6 Application Nodes Released ........................................................................... 12 2.6.1 Linux................................................................................................................ 12 2.6.2 Solaris ............................................................................................................. 12

3 SAP Service Connection .............................................................................. 13 3.1 OSS Network Connection with Dedicated saprouter Server ........................... 13 3.2 OSS Network Connection without Dedicated saprouter Server....................... 14 3.3 Related SAP Notes ......................................................................................... 15

4 FA Agents ...................................................................................................... 17

5 SAP Service Scripts for Virtualizing Applications...................................... 19 5.1 Starting a Configuration................................................................................... 20 5.2 Configuring the Service Status Path................................................................ 22 5.3 Configuring the Interfaces ............................................................................... 23 5.4 Host Names..................................................................................................... 25

6 Pre-SAP Installation Steps ........................................................................... 27 6.1 General ........................................................................................................... 27 6.2 SAN Configuration........................................................................................... 29

7 Operating System in General ....................................................................... 31 7.1 Solaris 8/9 ....................................................................................................... 31 7.1.1 Integrating New Java SDK (JDK) in Solaris .................................................... 31 7.2 Linux................................................................................................................ 34 7.2.1 Integrating a new Java SDK (JDK) in Linux SLES8 / SLES9 32bit.................. 35

Contents


7.2.2 Integrating New Java SDK (JDK) in SLES9_x86............................................. 36

8 Database in General ...................................................................................... 37 8.1 Installation of Oracle in General ...................................................................... 37 8.1.1 TNS Listener for Oracle................................................................................... 37 8.1.2 listener.ora .................................................................................................... 38 8.1.3 Host Name Changes in "tnsnames.ora" .......................................................... 39 8.2 Installation of Oracle 9.2.................................................................................. 40 8.2.1 Post-Oracle 9.2 Installation Steps ................................................................... 41 8.2.2 Patch Installation ............................................................................................. 41 8.3 Installation of Oracle 10.1 (R1) ........................................................................ 41 8.3.1 Script root.sh ................................................................................................... 42 8.3.2 Patch Installation ............................................................................................. 42 8.4 Installation of Oracle 10.2 (R2) ........................................................................ 43 8.5 Installation of SAPDB / MaxDB........................................................................ 43 8.5.1 XUSER ............................................................................................................ 43 8.5.1.1 Same SID on Different Platforms..................................................................... 44 8.5.2 MaxDB 7.5/7.6 /etc/opt/sdb ............................................................................. 45 8.5.3 MaxDB 7.6....................................................................................................... 46 8.5.4 IO Balancing .................................................................................................... 47 8.5.5 Directories for sapdata / saplog ....................................................................... 47 8.6 SAP Installation with Database DB2................................................................ 47 8.6.1 Software Requirements ................................................................................... 47 8.6.2 Pre-Installation Steps ...................................................................................... 48 8.6.3 Installing DB2 .................................................................................................. 48 8.6.4 Installing DB2 License ..................................................................................... 49 8.6.5 LDAP-Plugin .................................................................................................... 50 8.6.6 Install SAP....................................................................................................... 50 8.6.7 Changes in System Files................................................................................. 51 8.6.8 Post Installation Steps ..................................................................................... 51

9 SAP in General............................................................................................... 53 9.1 ASCS ABAP Central Service Instance in General........................................... 53 9.2 SAP Java Cryptographic Toolkit ...................................................................... 53 9.2.1 Linux................................................................................................................ 53 9.2.2 Solaris ............................................................................................................. 54

10 Installing SAP 4.6C SR2................................................................................ 55 10.1 SAPDB / MaxDB / Oracle ................................................................................ 56

11 Installing Web AS 6.20 .................................................................................. 61 11.1 Installating Web AS 6.20 with sapinst 642....................................................... 61 11.1.1 Installing the Central Instance (CI) .................................................................. 62 11.1.2 Installing the Database Instance (DB) ............................................................. 64

Contents


12 Installing Web AS 6.40 with NetWeaver04 SR1........................................... 69 12.1 Description of Web AS 6.40 JAVA .................................................................. 70 12.2 Web AS 6.40 NetWeaver04SR1 JAVA MaxDB/Oracle ................................... 71 12.2.1 SCS................................................................................................................. 71 12.2.2 DB ................................................................................................................... 75 12.2.3 JC.................................................................................................................... 76 12.2.4 J ...................................................................................................................... 76 12.2.5 Web AS 6.40 Deployment SP16 ..................................................................... 77 12.2.5.1 JC.................................................................................................................... 77 12.2.5.2 J ...................................................................................................................... 77 12.3 Web AS 6.40 NetWeaver04SR1 ABAP Oracle ............................................... 77 12.3.1 CI..................................................................................................................... 77 12.3.2 DB ................................................................................................................... 81 12.3.3 APP ................................................................................................................. 82 12.4 Web AS 6.40 ABAP + JAVA-AddIn NetWeaver04SR1 ................................... 82 12.4.1 CI..................................................................................................................... 82 12.4.2 DB ................................................................................................................... 82 12.4.3 SCS................................................................................................................. 83 12.4.4 Web AS 6.40 Deployment SP16 ..................................................................... 87 12.4.4.1 DVEBMGS (JAVA-AddIN) ........................................................................ 87 12.5 Web AS 6.40 Application Server JAVA-AddIn................................................. 87

13 Installing NetWeaver 7.0SR1/SR2 ................................................................ 89 13.1 Installation of SAP Solutions ........................................................................... 90 13.1.1 Prerequisite Checker (Linux) ........................................................................... 93 13.2 Sapinit File (Linux)........................................................................................... 94 13.3 installsapinit.sh symlink (Linux) .................................................................... 95 13.4 Services File (Linux)........................................................................................ 96 13.5 NetWeaver 7.0SR1/SR2 and Oracle 10(R2) ................................................... 97 13.6 NetWeaver 7.0SR1/SR2 and MaxDB76.......................................................... 98 13.7 NetWeaver 7.0SR1/SR2 / MaxDB76 and SLES9_x86 .................................... 99 13.8 NetWeaver 7.0SR1/SR2 MaxDB76 and Solaris9 .......................................... 102 13.9 NetWeaver 7.0SR1/SR2 and EP................................................................... 102 13.10 NetWeaver 7.0SR1/SR2 ABAP+JAVA-AddIn ............................................... 104 13.11 NetWeaver 7.0SR1/SR2 JAVA only .............................................................. 109 13.12 LiveCache ..................................................................................................... 110 13.13 ERS Enque Replicated Server ................................................................... 110 13.14 Installation of SAP Solutions Post Phase ................................................... 110 13.14.1 Linux.............................................................................................................. 110 13.14.2 Solaris ........................................................................................................... 111 13.14.3 Linux and Solaris........................................................................................... 111 13.14.4 Solaris 10 Preparations ................................................................................. 112

14 Post-SAP Installation Steps ....................................................................... 113 14.1 Configuring the Virtual Host Name Manually................................................. 113

Contents


14.1.1 Central Instance (not valid for Web AS 6.40 J2EE) ....................................... 115 14.1.2 Database Instance (also valid for WebAS 6.40 J2EE)................................... 115 14.1.3 Application Instance (not valid for WebAS 6.40 J2EE) .................................. 116 14.1.4 SCS Instance................................................................................................. 117 14.1.5 ASCS Instance (WebAS 7.00)....................................................................... 119 14.1.6 NetWeaver 7.0 / NetWeaver 7.0SR1/SR2..................................................... 120 14.2 Configuring the Virtual Host Name for Web AS 6.40 ABAP and

JAVA-AddIn / WebAS 6.40 JAVA.................................................................. 121 14.3 Start Script startdb for Oracle and SAPDB/MaxDB ....................................... 138 14.4 Stop Script stopdb for Oracle and SAPDB/MaxDB........................................ 138 14.5 Scheduling of Batch Jobs (CCMS) ................................................................ 138 14.6 Profile Parameters for DEFAULT.PFL........................................................... 139 14.7 Profile Parameters for the Start Profile .......................................................... 140 14.8 Profile Parameters for the Instance Profile .................................................... 140 14.9 SAP Start and Stop Mechanism (Release 4.6CSR2) .................................... 141 14.10 SAP License .................................................................................................. 143 14.11 SAPLogon SAPGUI.................................................................................... 143 14.12 SAPROUTER on the Control Node ............................................................... 146

15 SAP System Administration ....................................................................... 147 15.1 Upgrading Application Software .................................................................... 147 15.2 Software Administration................................................................................. 147 15.2.1 Starting and Stopping SAP Services ............................................................. 147 15.2.1.1 SAP Service Scripts ...................................................................................... 147 15.2.1.2 SAP Service Script Actions............................................................................ 149 15.2.1.3 Return Codes of the SAP Service Scripts ..................................................... 150 15.2.1.4 Joined SAP Service Scripts ........................................................................... 150 15.2.2 Removing an Application from Monitoring by FA Agents............................... 151 15.2.3 Application Software...................................................................................... 151 15.3 SAP Release Upgrade .................................................................................. 152 15.3.1 Services Port ................................................................................................. 152 15.3.2 Group Sapinst................................................................................................ 152 15.3.3 FA-Agent ....................................................................................................... 153 15.4 Transport Directory........................................................................................ 153 15.5 Printing in FlexFrame .................................................................................... 153 15.5.1 Possible Printing Architectures and Methods ................................................ 153 15.5.2 Local Printing (Access Methods C and L)...................................................... 154 15.5.3 Remote Printing (Access Methods S and U) ................................................. 154 15.5.3.1 Architecture Prerequisites.............................................................................. 154 15.5.3.2 SAP System Prerequisites............................................................................. 155 15.5.3.3 Description..................................................................................................... 155 15.5.4 Front-end Printing on Local Printers .............................................................. 156

Contents


16 BRBACKUP and SAPDBA/BRCONNECT................................................... 157 16.1 Solution 1 for Oracle and SAPDB/MaxDB..................................................... 157 16.2 Solution 2 for Oracle...................................................................................... 158

17 Troubleshooting .......................................................................................... 159

18 Abbreviations .............................................................................................. 161

19 Glossary....................................................................................................... 165

20 Index............................................................................................................. 171

Installation Guide for SAP Solutions 1

1 Introduction As we continue to improve the installation process, this document is permanently subject to change.

The latest version can be found at

http://extranet.fujitsu-siemens.com/FlexFrame

Please make sure that you get the latest version before starting an installation, and always read the latest README file that contains further changes and fixes.

1.1 Purpose of this Document This document provides brief instructions for installing SAP with Oracle or SAPDB / MaxDB in a FlexFrame environment. It is absolutely essential to use the SAP Installation Guides and the latest SAP Notes for the SAP solution that is to be installed.

It is intended for certified FlexFrame consultants only; sound knowledge of FlexFrame technology and its background is required.

1.2 General Usage of SAP Installation Guide During Installation

Please note that the entire documentation on installing SAP on FlexFrame is not intended as a stand-alone documentation. It solely describes the differences to a standard SAP installation as described by SAP.

1.3 Overview The SAP installation is performed in 5 steps:

1. Information about the SAP Services is collected in an Excel spreadsheet; a configuration file is generated and stored on the LDAP server (Section 2.4).

2. SAP Service scripts for virtualizing the applications (Chapter 5).

3. Oracle 9.2/10.x and SAPDB (7.3 / 7.4) /MaxDB (7.5 / 7.6) installation, LDAP configuration (Chapter 6).

4. Installation of SAP software depending on the release (Chapters 9, 10, 11 ,12, and 13).

5. Post-SAP installation steps have to perform (Chapter 14).

Introduction

2 Installation Guide for SAP Solutions

1.4 Notational Conventions The following conventions are used in this manual:

Additional information that should be observed.

Warning that must be observed.

fixed font Names of paths, files, commands, and system output. Names of variables fixed font User input in command examples

(if applicable using with variables)

1.5 Document History Document Version Changes Date

1.0 First Edition 2008-04-01

1.6 Related Documents FlexFrame for SAP Planning Tool FlexFrame for SAP Installation of a FlexFrame Environment FlexFrame for SAP Administration and Operation FlexFrame for SAP Network Design and Configuration Guide FlexFrame for SAP FA Agents - Installation and Administration FlexFrame for SAP Installation ACC 1.0 SP13 FlexFrame for SAP myAMC.FA_LogAgent - Concept and UsageFlexFrame for SAP myAMC.MessengerFlexFrame for SAP Upgrading FlexFrame 3.2B or 4.0 to 4.1A FlexFrame for SAP White Paper PRIMECLUSTER Documentation ServerView Documentation SUSE Linux Enterprise Server Documentation Solaris Documentation

Introduction


1.7 IP Addresses Used in this Document All Application Nodes must belong to a dedicated FlexFrame pool; nevertheless it is possible to have only one pool in a FlexFrame environment. It is not permissible to use a node in more than one pool at the same time. For this reason Application Nodes belonging to the same pool use the same IP subnet.

All blades in the same cabinet can be members of different pools.

Control Nodes and the NetApp Filer have IP addresses for all pools.

The Control LAN is independent of the pool number and has its own IP subnet addresses. Pool No.

Client IP Server IP Storage IP Control IP (RSB) Heartbeat (PCL)

1 10.0.1.201 192.168.1.201 192.168.100.201 192.168.200.221 control1 192.168.200.201

2 10.0.2.201 192.168.2.201 192.168.101.201 192.168.200.221 control1

1 10.0.1.202 192.168.1.202 192.168.100.202 192.168.200.222 control2 192.168.200.202

2 10.0.2.202 192.168.2.202 192.168.101.202 192.168.200.222 control2

1 192.168.100.203 192.168.200.203 filer

2 192.168.101.203 192.168.200.203 filer

BX300-1 1 192.168.200.230 bx3001mg

mt

1 10.0.1.1 192.168.1.1 192.168.100.1 blade1

1 10.0.1.2 192.168.1.2 192.168.100.2 blade2

1 10.0.1.3 192.168.1.3 192.168.100.3 blade3

BX300-2 2 192.168.200.240 bx3002mg

mt

2 10.0.2.11 192.168.2.11 192.168.101.11 blade11

2 10.0.2.12 192.168.2.12 192.168.101.12 blade12

2 10.0.2.13 192.168.2.13 192.168.101.13 blade13

1 10.0.1.21 192.168.1.21 192.168.100.21 192.168.200.21 PW250_1

2 10.0.2.22 192.168.2.22 192.168.101.22 192.168.200.22 PW250_2

192.168.200.252 switch1

192.168.200.253 switch2

SAP instance

1 10.0.1.101 192.168.1.101 cic11

Introduction


Pool No.

Client IP Server IP Storage IP Control IP (RSB) Heartbeat (PCL)

1 192.168.1.102 dbc11

1 10.0.1.103 192.168.1.103 app01c11

1 10.0.1.104 192.168.1.104 app02c11

1 10.0.1.105 192.168.1.105 app03c11

2 10.0.2.111 192.168.2.111 cibw1

2 192.168.2.112 dbbw1

2 10.0.2.113 192.168.2.113 app01bw1

2 10.0.2.114 192.168.2.114 app02bw1

1.8 Hotlinks for FlexFrame PRIMECLUSTER4

PRIMECLUSTER Wizards (downloads & documents)

http://extranet.fujitsu-siemens.com/My.htm

Primecluster RMS Home (information on the Reliant Monitor Services)

http://athen.mch.fsc.net/~kon/reliant.htm

PRIMECLUSTER

http://www.primecluster.com/ SAP on Linux / FlexFrame

SAP on Linux (released kernels, hardware and distributions for SAP)

http://www.sap.com/linux

SAP NetWeaver on Linux

http://www.sap.com/solutions/netweaver/

Official FlexFrame home page

http://www.fujitsu-siemens.com/flexframe

Information on FlexFrame for SAP

http://extranet.fujitsu-siemens.com/FlexFrame

PRIMERGY manuals

http://manuals.fujitsu-siemens.com/primergyservers.html

Introduction


SAP on Solaris / FlexFrame

Global information and operating system manuals

Fujitsu Siemens Computers and SAP

http://manuals.fujitsu-siemens.com/unixservers.html

PRIMEPOWER manuals

http://manuals.fujitsu-siemens.com/unixservers.html Knowledge

SNMP protocol information

http://www2.rad.com/networks/1995/snmp/snmp.htm

Linux Knowledge Portal

http://www.linux-knowledge-portal.org/de/index.php

Official Linux documentation project

http://www.tldp.org/

Official PRIMERGY documentation

http://manuals.fujitsu-siemens.com/serverbooks/start.htm

Internal information on PRIMERGY servers

http://extranet.fujitsu-siemens.com/products/primergy Software

GNU Homepage

http://www.gnu.org/

Software by Fujitsu Siemens Computers

http://support.fujitsu-siemens.de/de/support/downloads.html

Open SSH website

http://www.openssh.org/

SSH download

http://www.ssh.com/


2 Overview

2.1 FlexFrame FlexFrame is a completely revolutionary solution for creating and configuring the infrastructure required to run a SAP environment.

An essential feature of FlexFrame is that the infrastructure is divided into the following elements:

Computing, represented by the Application Nodes Network, completely implemented on the basis of IP networks Control and management, represented by the Control Nodes Storage, represented by the NetApp Filer

A number of fundamental features of FlexFrame are implemented on the basis of this division into the areas of computing, network, control and storage:

Virtualized SAP Services Central, shared operating system Embedded high availability Network attached storage (NAS)

For further details, please refer to the Technical White Paper for FlexFrame 4.0.

2.2 Virtualization of SAP Services To enable high availability and scalability of SAP Services (database, SAP R/3, etc.), the fixed assignment of SAP Services to Application Nodes has been abolished. Here, each SAP Service can run not only on one host in the high-availability cluster, as has been the case with customary high availability solutions to date, but on virtually any required Application Node in FlexFrame, provided it has resources available for this.

Technically speaking, virtualization is achieved by the fact that an SAP Service is able to move, taking its IP address with it and retaining its virtual host name.

Virtualization is achieved here by SAP Service scripts that ensure the virtual host names and virtual IP addresses are set accordingly when SAP Services are started and stopped.

Overview


2.3 Preparations Make sure you have the following hardware and software available:

Your FlexFrame hardware A notebook with Microsoft Excel 2003 FlexFrame 3.2 Planning CD-ROM OS installation of Control Nodes and the necessary Application Nodes

(Linux/Solaris) must have been completed Installation DVDs/CDs from SAP for the required SAP Releases

2.4 Gathering SAP Services Information The FlexFrame Planning Tool is a program based on Microsoft Excel 2003. It was used during the setup of the hardware to collect all information for a FlexFrame landscape (original file name FFPLAN.XLS). You may use the Excel sheet that has already been completed or a copy of it. If all SAP Services were entered beforehand, you may skip the rest of this section.

After you have opened the Excel sheet, go to the SAP Services tab at the bottom of the document (scroll to the right in the tab list). If there are no SAP instances listed, click the Add SID button and insert the SAP Services that have to be installed. You will be asked for the following details:

SID

Select pool (dropdown list)

Select DB type (dropdown list)

Select SAP version (dropdown list)

Select one of the following (only Web AS 6.40)

CI (ABAP only) CI + SCS (ABAP/JAVA-AddIn) JC + SCS (JAVA only)

Overview


Select one of the following (only Web AS 7.00)

CI (ABAP only) CI + ASCS (ABAP/ASCS) CI + ASCS + SCS (ABAP/ASCS/JAVA-AddIn) CI + SCS (ABAP/JAVA-AddIn) JC + SCS (JAVA only) CI + ASCS + ERS (ABAP/ASCS/Enqueue Replicated Server) CI + ASCS + SCS + ERS

(ABAP/ASCS/JAVA-AddIn/Enqueue Replicated Server)

JC +ERS (Enqueue Replicated Server only)

Number of application instances (in addition to the CI/JC)

LiveCache (WebAS 7.0/MaxDB only)

If you want to install a SAP component like CRM or R/3 (ERP), you wont find it in the dropdown list of the SAP versions. You have to enter the underlying SAP Basis (Web AS) release which must be one of the four listed versions. If you need another Basis release or another DB type for your SAP component, please ask our SAP Competence Center for project-specific support.

Please make sure to have the latest version before starting an installation, and always read the latest README file that contains further changes and fixes.

After all information for the complete SAP Services landscape has been entered in the tool, the configuration files ff_*.conf can be created by pressing the button labeled Generate configuration files. A dialog box will ask you for a directory where the configuration files are stored. We recommend that you create a new directory for each project and version of the files, including the FFPLAN32.XLS spreadsheet.

The configuration file ff_services.conf now has all the information required to be stored on the LDAP server. This step must currently be performed manually until a tool is available to store this information in LDAP. For more information about LDAP please refer to the LDAP Installation Guide.

2.5 FlexFrame Installation Diagram Step by Step The following chart shows the installation steps of a complete FlexFrame installation. As the final step of the platform basis, SAP installation is shown at the bottom of the second chart (red frame).

The numbers in the grey bar on the left side of the chart refer to the sections of the chapter Quick Installation in the manual FlexFrame for SAP - Installation of a FlexFrame Environment.

Overview


FlexFrame Installation Steps

2.10.2

ExcelFlexFrame

Planning ToolFF Prep

CD

ff_wiring.plff_switch_conf.pl

CablingPlan

NetworkCabling

Hardware Installation

DVD

CN

Control Node 1

ExternalMedium:Configurationfiles; Patches

ff_wiring.pl Switches

2.10.1 2.6.3

2.2

2.3

2.4

2.52.6

2.9

OS Installation

ControlNode

SwitchConfi-

guration

Configuration Files:ff_cn.confff_an.confff_misc.conf

ff_services.conf

Manual Setup

Control Node

LDAPand SSH

Confi-guration

Preparations

HW Installation

Control Node 1

ff_filer_qtree_create.sh

ff_slapd_init.sh NAS system

Filer QTrees; LDAP

Initialization

Boot fromDVD

Serial cable

ff_setup_cn.sh

ff_apply_settings_cn.sh

Serial cable

EnterMAC adresses

ff_get_an_mac.pl ff_an_mac.conf

ff_ldap_conf.pl

ff_cn.conf; ff_an.conf; ff_services.conf; ff_an_mac.conf

2.8

MAC Addresses

Generates Scripts:

ff_slapd_init.sh

ff_filer_qtree_create.sh

NAS Confi-guration

ff_cn.conf; ff_an.conf; ff_services.conf; ff_an_mac.conf

The steps arenumbered accordingto the sections in the

quick installation chapter

2.10.4

PRIME-CLUSTER

Configuration

Control Node

Commands:cftool -nsdtool -shvdisp -a

DVD

CNControl Node 2

Boot from DVDOS Installation 2nd Control Node, see 2.4

NAS system

Service

CN

Overview


Proceed with SAP Installation

ff_cn.confff_an.confff_an_mac.conf

ff_create_an_cfg.pl

netboot.cfg

LDAP Configuration Files

in /FlexFrame/volFF/FlexFrame/ldap/etc/pools// ;/opt/FlexFrame/bin/ff_linux_ldap_client.sh/tftpboot/config/netboot__.cfg

ff_new_an.sh CustomizedBoot Image

ff_new_an.sh CustomizedBoot Image

/FlexFrame/volFF/FlexFrame/ldap/etc/pools//solaris_ldap_client.sh

2.11.4

2.11.5

2.11.6

Application Node Configuration

Setup LinuxApplication

Nodes

Setup Solaris Application

Nodes

2.11

Generation of netboot

config files

2.13

DVD AN

Linux

DVD AN

Solaris

Preparationfor Solaris Application

Nodes

2.11.3

2.11.2

Preparationfor Linux

ApplicationNodes

nb_unpack_bi

NAS system

ff_install_an_linux_images.sh

/tftpboot/config/netboot__.cfg

NAS system

NAS system

NAS system

NAS system

Overview


2.6 Application Nodes Released

2.6.1 Linux All current SAP-certified, Intel//AMD-based servers from Fujitsu-Siemens Computers have been released.

Please refer to: http://www.sap.com/solutions/netweaver/linux/platforms/fsc.asp

2.6.2 Solaris The PRIMEPOWER servers PW250/PW450 PW650/PW850 from Fujitsu-Siemens Computers have been released.

Please note: Relevant source for supported components including Application Nodes is the FlexFrame Support Matrix available at: http://extranet.fujitsu-siemens.com/flexframe


3 SAP Service Connection It is very important that the customers FlexFrame environment can be accessed via SAPs OSS (Online Software Service). This is a mandatory requirement when customer support is requested, particularly during the installation phase.

Please use the following OSS component for your customer calls to SAP about FlexFrame: CA-ADA-FSC

3.1 OSS Network Connection with Dedicated saprouter Server

The figure below shows an example of a possible network path from SAP to the FlexFrame environment. This figure includes a (possibly existing) server SAPROUTER, which runs the saprouter process:

CustomerSAPWalldorf

TokioFoster City

SAPROUTERTELNETSAPGUI

Control Node 1

Control Node 2

Application Node 1

Gateway

Application Node 2

Application Node ...

Application Node n

TELNETSSH

TELNETSSH

SAPGUISSH

SAPGUISSH

SAPGUISSH

SAPGUISSH

TELNETSAPGUI

CustomerSAPWalldorf

TokioFoster City

SAPROUTERTELNETSAPGUI

Control Node 1

Control Node 2

Application Node 1

Gateway

Application Node 2


Application Node n

TELNETSSH

TELNETSSH

SAPGUISSH

SAPGUISSH

SAPGUISSH

SAPGUISSH

TELNETSAPGUI

Please enter the server SAPROUTER as External Connectivity in the FF Planning tool to get a reservation of an IP address for the first pool.

SAP Service Connection


3.2 OSS Network Connection without Dedicated saprouter Server

The saprouter process can be configured on the Control Node cluster if there is no dedicated server for it (see section SAPROUTER on the Control Node on page 146).

In this case, a virtual IP address for the Client LAN has to be defined for the saprouter service and for every pool, and PRIMECLUSTER is configured accordingly (only for the first pool today).

The virtual IP addresses will be generated with the FlexFrame Planning Tool: go to the External Connectivity tab, select Add New, enter the pool name and Other as selected type. Afterwards enter Saprouter as comment and a valid host number. The host name for the first pool is predefined as saprouterINT and has to be used (check entry in /usr/opt/reliant/etc/hvipalias). To allow PRIMECLUSTER to control of the saprouter service, it is necessary to get the latest version of saprouter from SAP (Service Marketplace) and to install it in the directory /FlexFrame/volFF/FlexFrame/saprouter on the active Control Node. This directory must have been created, the saprouter program must be copied to it and the saprouttab file created in the same directory. The PRIMECLUSTER control of the service is already predefined and has to be activated!

To add the saprouter application to RMS, call:

cn1:~ # pcstool -s -i /etc/PRIMECLUSTER/scripts/addSaprouter FlexFrame To remove the saprouter application from RMS, call:

cn1:~ # pcstool -s -i /etc/PRIMECLUSTER/scripts/delSaprouter FlexFrame To activate the modified configuration, first shut down RMS on both nodes:

cn1:~ # hvshut -L cn2:~ # hvshut -L The capital letter -L keeps all services running. Then activate the modified configuration and distribute it to all Cluster Nodes:

cn1:~ # pcstool -a FlexFrame Finally, start RMS on both Control Nodes:

cn1:~ # hvcm cn2:~ # hvcm

Dont forget to check the correct permissions in the saprouttab file if there are multiple pools!

SAP Service Connection


CustomerSAPWalldorf

TokioFoster City

TELNETSAPGUI

Control Node 1(saprouter)


Application Node 1

Gateway

Application Node 2


Application Node n

SAPGUISSH

SAPGUISSH

SAPGUISSH

SAPGUISSH

TELNETSAPGUI

CustomerSAPWalldorf

TokioFoster City

TELNETSAPGUI



Application Node 1

Gateway

Application Node 2


Application Node n

SAPGUISSH

SAPGUISSH

SAPGUISSH

SAPGUISSH

TELNETSAPGUI

3.3 Related SAP Notes A table of related SAP Notes is provided below:

SAP OSS note number Title

35010 Service connections: Composite note (overview)

37001 Telnet link to customer systems


4 FA Agents A FlexFrame Autonomous Agent (FA Agent) is an efficient component for highly autonomous operation of systems with distributed instances. A FlexFrame solution consists of Network Attached Storage, Application Nodes, and redundant Control Nodes. The FA Agents product has been developed for this solution, consisting of storage, servers, and connectivity.

It permits fast and flexible installation of solutions, which simplify the operation of the applications and make them flexible. New nodes and application instances can be integrated without any changes to the configuration.

Components of a FlexFrame Autonomous Agent:

FlexFrame Autonomous Application Agent (FA_AppAgent) FlexFrame Autonomous CtrlAgent (FA_CtrlAgent) The FlexFrame Autonomous Agent components also permit the highly available, partly autonomous operation of distributed applications. The application instances can be distributed over any required number of nodes within a FlexFrame solution. Monitoring of several services is implemented via the FlexFrame Autonomous Agent.

The FA Application Agent components normally support SAP application instances (pure application instances and central instances), as well as SAPDB/MaxDB and Oracle databases.

For more information, please refer to the FA Agents - Installation and Administration manual.

Please make sure that the FA Application Agents are stopped on the hosts while you are installing SAP software:

/etc/init.d/myAMC.FA_AppAgent stop (Linux/Solaris9) svcadm disable -s smawfa-agents (Solars10) Check the status with:

/etc/init.d/myAMC.FA_AppAgent status (Linux/Solaris9) svcs smawfa-agents (Solars10)


5 SAP Service Scripts for Virtualizing Applications

Scripts for starting, stopping and restarting the applications and services are provided by SAP.

Make sure that the scripts are located at /FlexFrame/scripts.

The following scripts are installed:

Script name Application

sapapp Application server (ABAP) sapascs ABAP central services instance sapci Central instance (ABAP) sapdb Database instance (SAPDB, MaxDB, Oracle) sapers Enqueue Replicated Server sapj J2EE Application server sapjc J2EE Central Instance saplc LiveCache sapscs Java central services instance sapservice_config Global customer-specific parameters sapservice_functions Global parameters and functions start_all_sapservices Initial start of all configured applications stop_all_sapservices Stops all running applications stop_all_sapservices_local Stops all running applications on the local node stop_all_sapservices_SID Stops all running applications of one SID

SAP Service Scripts for Virtualizing Applications


5.1 Starting a Configuration The start_all_sapservices start script defines which applications are to be started in what sequence and on which virtual host. The script is executed on a Control Node.

The script below illustrates its syntax:

#!/bin/bash #----------------------------------------------------------------- # Rule is: # All sapservices in one 'eval' must start one after another, # each 'eval' may start simultaneous. # C11 eval " ssh blade1 sapdb C11 start ssh blade1 sapci C11 start ssh blade2 sapapp 01 C11 start ssh blade3 sapapp 02 C11 start " & # BW1 eval " ssh PW250-1 sapdb BW1 start ssh PW250-1 sapci BW1 start " & #echo "" #echo "All services should be up now." #-----------------------------------------------------------------

The start sequence within a system ID must always be observed:

ABAP ABAP + ASCS ABAP + JAVA-AddIn JAVA

ABAP + ASCS + JAVA-AddIn

1. DB 1. DB 1. DB 1. DB 1. DB2. CI 1. ASCS 1. SCS 1. SCS 1. ASCS 3. APP 2. CI 2. CI 2. JC 1. SCS 3. APP 2. APP 2. J 2. CI 3. APP

Equal numbers mean that the features can be started in parallel.



The figure below shows the dependencies of the SAP Services (x y means x is dependent on y).

The field containing JC and SCS is the minimal group of services for J2EE only. The field with CI and SCS is the minimum for ABAP and J2EE. Only ABAP is not mentioned here but it would be CI only. However it is possible to have a combination of CI and ASCS (ABAP central services instance). And still every service group needs its DB server.



5.2 Configuring the Service Status Path The SERVICE_STATUS_PATH parameter describes the global location for the service control files of the applications (services). If such a file exists, the application or service has been started and therefore should be running. The naming convention is defined as follows:

For the service types db and ci: __host For the service type app: app___host Explanation: id Consecutive two-digit number from 00-97

sid The lower-case three-digit system ID

Web AS 6.40 with JAVA/J2EE Engine and/or ABAP Engine

There are additional service types when you run a J2EE engine based on WebAS 6.40.

For the service types scs and jc: __host For the service type j: j___host Explanation: id Consecutive two-digit number from 00-97

sid The lower-case three-digit system ID The content of the service control file is the name of the node the application or service is running on.

Web AS 7.00 with JAVA/J2EE Engine and/or ABAP/ASCS Engine

There are additional service types when you run a J2EE engine based on WebAS 7.00.

For the service types scs, ascs and jc: __host For the service type j: j___host Explanation: id Consecutive two-digit number from 00-97


WebAS 7.0 with JAVA/J2EE Engine and/or ABAP/ASCS Engine and ERS

There are optional service types bound on WebAS 7.00.

For the service types scs and ascs: sapers___host Explanation: id Consecutive two-digit number from 00-97




WebAS 7.0 with ABAP and MaxDB (LiveCache)

There is a additional service type which supports LiveCache.

Service type: lc__host Explanation: sid The lower-case three-digit system ID

5.3 Configuring the Interfaces Depending on which interfaces the nodes have, the LAN type must be assigned to the interfaces.

The following section from the sapservice_config include script shows the connection between the LAN type, interface and netmask:

#----------------------------------------------------------------- # Netmask # #NETMASK_CLIENT=255.255.240.0 ls3195:255.255.255.0 NETMASK_CLIENT=255.255.255.0 NETMASK_SERVER=255.255.255.0 # # Interface # #----------------------------------------------------------------- # SunOS link aggregation # ~~~~~~~~~~~~~~~~~~~~~~~ # # real interfaces look like this: # fjgi2005000 # this is the first interface # fjgi2005001 # this is the second interface # fjgi2005000:1 # this indicates that the first interface # # is active # fjgi2005001:1 # this indicates that the second interface # # is active # # These interface names consist of three parts: # fjgi2005001:1 # ^^^----------- interface number # ^^^^-------------- vlan # ^^^^------------------ 1GB-model # # For our network to interface relation we use only the first two # parts of the interface names, the model and the vlan. In our # example:



# fjgi2005 # #----------------------------------------------------------------- # # Recommendations: # # for a clean linux environment without network redundancy, # FlexFrame = 3.1 # ETH_CLIENT=vlan2001 # ETH_SERVER=vlan2005 # # for a clean solaris environment with network redundancy, # FlexFrame >= 3.1 # ETH_CLIENT=fjgi2001 # ETH_SERVER=fjgi2005 # # for a mixed environment without network redundancy, # customer specific # ETH_CLIENT=eth0 belana:hme0 belana2:hme0 # ETH_SERVER=eth1 belana:hme1 belana2:hme1 # # for a mixed environment with network redundancy, # FlexFrame >= 3.1 # ETH_CLIENT=vlan2001 Baby_1:fjgi2001 Baby_2:fjgi2001 # klinge1:vlan2002 # ETH_SERVER=vlan2005 Baby_1:fjgi2005 Baby_2:fjgi2005 # klinge1:vlan2006 # # In a FlexFrame >= 3.1 environment with more than one pool, # we can have more than one configuration file, linked into an # pool-image. This avoid to define to much specials.

The LAN types Server and Client are used. The syntax will be illustrated with reference to the example of Client and Server LAN.

For example, the ETH_SERVER parameter describes a list of assignments between the node and the interface. The first element in the list is the standard interface for the LAN type server LAN; all of the subsequent elements are optional and describe a deviation from this type.



5.4 Host Names The following (virtual) host names for the SAP Services must be used and will be generated by the Excel planning tool:

Server LAN Client LAN

db-se ci-se ci ascs-se ascs app-se app scs-se scs jc-se jc j-se j

Explanation: id Consecutive two-digit number 00-97 sid The lower-case three-digit system ID

In FlexFrame 4.0, the following rules have to be regarded to define host names:

1. The host name in the Storage LAN is built using the host name and adding -st . 2. The host name in the Server LAN is built using the host name and adding -se . 3. The host name in the Control LAN is built using the host name and adding -co . 4. The host name in the Client LAN is built using the host name without an extension.

5. Host names are written in lower case exclusively.

6. Host names are supposed not to be longer than 13 characters.


6 Pre-SAP Installation Steps

6.1 General The customary instructions and guide lines apply to the installation of the software components. The deviations from these which are required are described here.

Dont change the host name of the Application Node where you are going to install the SAP and DB software.

After the installation is done, further steps are described in this document to virtualize the SAP instance (except SAPINST 640 SR1 when used with the SAPINST_USE_HOSTNAME parameter). Please install CI (J2EE: SCS) first, then DB, (J2EE: JC 2nd) and the Application Server at the end.

Setup the virtual interfaces before installing the SAP solution. It is absolutly essential to setup both virtual interfaces (client + server LAN) and make sure that they up an running.

Please make sure that the FA Application Agents are stopped on the hosts while you are installing SAP software:

/etc/init.d/myAMC.FA_AppAgent stop (Linux/Solaris9) svcadm disable -s smawfa-agents (Solars10) Check the status with:

/etc/init.d/myAMC.FA_AppAgent status (Linux/Solaris9) svcs smawfa-agents (Solars10)

The SAP Installation CDs/DVDs have to be copied to the NetApp Filer (from the Control Node into the directory /FlexFrame/volFF/FlexFrame/stage/SAP_SW/). The whole directory or each subdirectory has to be exported on the Filer with the necessary ro or rw options for the Application Node(s). On the Application Node, the mount will be done manually.

If an SAP Installation is done after the initial setup of the FlexFrame environment and without entering the SAP Service data in the FF Planning tool before, the necessary LDAP data has to be created with the script ff_sid_adm.pl (see chapter Adding / Removing SAP Systems and Application in the manual FlexFrame for SAP - Administration and Operation).

The directories that are necessary for SAP installation should have been created in a step during the LDAP server setup. A script will have been created to create the SID folder structure, if necessary.

Pre-SAP Installation Steps


control1# ff_setup_sid_folder.sh Check the folder structure as described in the SAP installation guide.

You may use ssh X server for the Sapinst GUI. $DISPLAY should also be adequate, like localhost:10.0. The X connection can be tested with xclock. RX600-01:~ # xclock An XCLOCK should pop up on your display. If not, check the DISPLAY variable or the.Xauthority file. Check the JAVA JDK:

server:~ # java -version java version "1.4.2_11" Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.2_11-b04) Java HotSpot(TM) Client VM (build 1.4.2_11-b04, mixed mode)

Check the JAVA environment on Linux:

server:~ # echo $JAVA_HOME /usr/java/

Set the JAVA environment on Solaris:

server:~ # echo $JAVA_HOME server:~ # JAVA_HOME=/usr/j2sdk/ Create the SAP installation directory:

server:/ # mkdir /usr/sap//install server:/ # chmod 777 /usr/sap//install server:/ # cd /usr/sap//install Setup the following variables, before starting the installation:

server:/ # export JAVA_HOME=/usr/java/ server:/ # export SAPINST_JRE_HOME=/usr/java/ server:/ # export TMP=

Pre-SAP Installation Steps


How to configure the interfaces manually:

Solaris (client LAN: 10.1.7.106 server LAN: 20.1.7.106):

server:/ # ifconfig fjgi2001:2 plumb 10.1.7.106 netmask 0xffffff00 broadcast 10.1.7.255 up server:/ # ifconfig fjgi2002:2 plumb 20.1.7.106 netmask 0xffffff00 broadcast 20.1.7.255 up Linux (client LAN: 10.1.7.101 server LAN: 20.1.7.106):

server:/ # /sbin/ifconfig vlan2001:0 10.1.7.101 broadcast 10.1.7.255 netmask 0xffffff00 server:/ # /sbin/ifconfig vlan2002:0 20.1.7.101 broadcast 20.1.7.255 netmask 0xffffff00

6.2 SAN Configuration Before installing a SAP system you have to prepare the SAPDATA and SAPLOG concerning the SAN requirements. Please refer to chapter Creating Volumes and Filesystems in the Administration and Operation Guide. Before the database instance is installed you have to execute ff_san_mount.sh pre sapdb SID start to make the mountpoints accessible.

Important:

Especially for MaxDB database installations there is a special reference needed to address the data and log volumes. If you are asked for datafile directories you have to enter /sapdb//sapdata for data volumes and /sapdb//saplog for log volumes.

The access rights have to be set to the database user and database group.


7 Operating System in General The following sections describe the installation of Java SDK (JDK). If you have installed an JDK in an old Application Node image and you change to a new image you have to install the previous JDK in the new image in the same way. You can find the installation path and version of JDK in the instance profile entry jstartup/jvm/home=.

7.1 Solaris 8/9

7.1.1 Integrating New Java SDK (JDK) in Solaris This section describes how a new SUN Java SDK (JDK) can be integrated in a Solaris Boot Image using the nb_get_j2sdk script. Starting a maintenance run is not required. It runs independently of the customary FlexFrame procedure and the commands used there on the Control Node.

The script works interactively. For legal reasons the EULA from the SUN Java SDK archive must be displayed and acknowledged.

The script does not belong to the basic functionality of "Netboot for Solaris" but is a project-specific script. It is therefore supplied in the SMAWnbpw-FF4mySAP package and normally packed to /opt/SMAW/SMAWnbpw/bin/nb_get_j2sdk. Because of legal problems it is not possible to distribute one or more versions of the SUN Java SDK unpacked within our Solaris Boot Images. Nevertheless this script is provided as a simple way of making one or more versions of the JDK available in parallel in a Solaris Boot Image.

With version 12.3 of nb_get_j2sdk it is possible to install the "Java Cryptography Extension" (JCE) too.

Now it is possible to make the same SUN Java SDK version available in parallel as a version without JCE and as a version which includes JCE in the same Solaris Boot Image. The versions will be distinguished by the directory name.

After the script has been called, the new SUN Java SDK version is available in the /usr/j2sdk/ directory (e.g. /usr/j2sdk/j2sdk1.4.2_10) or /usr/j2sdk/_jce (e.g. /usr/j2sdk/j2sdk1.4.2_10_jce) - if JCE is installed too on all Application Nodes which were booted from the Boot Image specified in the script call. Setting the environment variable JAVA_HOME to this path then enables this SDK version to be used by applications.

The script has to be called once for each SUN Java SDK version which is to be stored, and also for each Boot Image in which a version is to be stored.

Operating System in General


1. The current J2SE SDK is loaded from

http://java.sun.com/j2se/1.4.2/download.html or an older version of the SDK is loaded from

http://java.sun.com/products/archive/index.html The following files should always be fetched here:

Solaris SPARC Platform: 32-bit self-extracting file and

64-bit self-extracting file Example: j2sdk-1_4_2_11-solaris-sparc.sh j2sdk-1_4_2_11-solaris-sparcv9.sh If JCE should be installed too, the matching Java Cryptography Extension zip file must be loaded from http://java.sun.com/j2se/1.4.2/download.html Section Other Downloads and must be put beside the two self-extracting files mentioned above.

2. The script requires the SUN Java SDK version which is to be distributed to be locally available on the Control Node. The two files are located adjacently on the Control Node. In addition, an active Solaris Application Node which can be addressed via the Storage LAN is available temporarily for unpacking these files.

3. Call parameters of the script:

nb_get_j2sdk -s -a -i -p -s

Complete path name of one of the two SUN Java SDK Solaris SPARC Platform self-extracting files on the Control Node (e.g. /tmp/j2sdk-1_4_2_11-solaris-sparcv9.sh)

-a Name of the active Solaris Application Node in the Storage LAN which can be used temporarily for unpacking the self-extracting files (e.g. AN2-st)

-i Complete path name of the Boot Image in which the SUN Java SDK version is to be stored in the /usr part, (e.g. /FlexFrame/volFF/os/Solaris/ FSC_5.8_202_20050211/bi_FJSV,GPUZC-M_PW-P)

-p requests the unpacking of the JCE package. The installation directory is named /usr/j2sdk/_jce.



Files:

The script generates or modifies the /usr/j2sdk/ files in the Boot Image.

Actions:

The script performs the following actions:

Check of the call parameter -s and its argument (SDK archive) A check is made to see if an absolute path was specified and all the required

files exist.

The name of the directory in which the data is stored automatically when it is unpacked from the SDK archive is determined. The name must be ascertained using the javahome pattern from the archive and is designated as in this text.

Check of the call parameter -a and its argument (Application Node) ping is used to check whether the Application Node can be addressed. The root file system of this Application Node is determined via the entry in

/etc/bootparams. The SDK archive is later unpacked into this root file system under /JDK_PACK.

Check of the call parameter -i and its argument (Image) A check is made to see if an absolute path was specified, if the directory does

exist on the Control Node and if it has a correct structure.

If it does not yet exist, the j2sdk directory is created in the /usr part of this Boot Image.

Checks if the SUN Java SDK version does exist already . If a SUN Java SDK version exists, it is not overwritten as it may already be in use by one of the Application Nodes.

The two SUN Java SDK archive files and the JCE archive file, if available, are now copied to /JDK_PACK in the root file system of the Application Node which is to be used temporarily.

Afterwards, the unpacking of the two archive files on the Application Node is triggered by two ssh calls. Both times the EULA of the SUN Java SDK is displayed and must be acknowledged with yes. The unpacking of the JCE archive file - if available - on the Application Node is triggered by a ssh call. Afterwards, the unpacked files from the JCE archive are copied into the just before unpacked SUN Java SDK to jre/lib/security.



The original files in this directory will be saved by moving to _orig. The new files from the JCE archive will be copied to _jce and then linked to the "right" name. So it is possible to identify the orginal SUN Java SDK files and the new JCE files.

Subsequently the /JDK_PACK/ resp.

j2sdk/_jce (if JCE was installed too) directory which was generated during unpacking is moved to j2sdk/ in the /usr part of the Boot Image selected.

Finally, a cleanup takes place on the Application Node that, among other things, deletes the /JDK_PACK directory .

7.2 Linux

In case the startup of the lsnrctl should fail during the SAP installation on Oracle, please check section listener.ora on page 38.

Relevant SAP Note for installation on Linux

If you want to install an SAP R/3 system on Linux, refer to SAP Note 171356.

SAP advice to use the most recent kernel

Please be advised to use one of the most recent 2.6 or 2.4 kernels for your hardware platform and distribution. In case of problems with the operating system, please try to reproduce it on a tested kernel and contact the support organization of your Linux distributor if the problem persists. (www.sap.com/linux/platforms)

Errors while executing shell scripts

The execution of shell scripts starting with : (instead of #!/bin/sh) causes errors or core dumps due to the Linux kernel. Try to solve this problem by rebooting your host or installing a new Linux kernel (see SAP Note 359566).

Due to the fact that the root image of Linux is read-only, please check the existence of the following links and directory:

control1# cd /FlexFrame/volFF/os/Linux/FSC_3.2/root_img appnode1# mkdir /oracle/ora-config/ORCLfmap



Create the DB instance installation directory:

BLADE1:~ # mkdir /usr/sap/C11/install BLADE1:~ # cd /usr/sap/C11/install BLADE1:~ # mkdir C11DB BLADE1:~ # chmod 777 C11DB BLADE1:~ # cd C11DB

7.2.1 Integrating a new Java SDK (JDK) in Linux SLES8 / SLES9 32bit

This section describes how to integrate a new SUN Java SDK (JDK) into a Linux Boot Image on the FlexFrame environment without going through a maintenance cycle.

The current J2SE SDK can be downloaded from

http://java.sun.com/j2se/1.4.2/download.html An older version of the SDK can be found at

http://java.sun.com/products/archive/index.html

Example for an Application Node (IP address 172.16.2.19):

install .rpm Mount root fs as rw:

Export the root image as rw for only the maintenance Application Node: control1:/FlexFrame//vol0/etc # vi exports > /vol/volFF/os///root_img > -ro=172.16.1.0/24:172.16.2.0/24,anon=0,rw=172.16.2.19 control1:~ # rsh filer1_qa exportfs -a Do not forget to export the root image as ro after the maintenance! RX300-01:/ # remount rw /



Make sure that the RPM does not contain files for the var image!

RX300-01:/ # rpm -qilp .rpm | less > search for "/var" install the RPM: RX300-01:/ # rpm -i .rpm remount the root file system read only: RX300-01:/ # remount ro / control1:/FlexFrame//vol0/etc # vi exports modify the entry containing the line: AN 172.16.2.19 "rw": #/vol/volFF/os///root_img-ro=172.16.1.0/24:172.16.2.0/24,anon=0,rw=172.16.2.19 /vol/volFF/os///root_img -ro=172.16.1.0 /24:172.16.2.0/24,anon=0 control1:~ # rsh filer1_qa exportfs a

7.2.2 Integrating New Java SDK (JDK) in SLES9_x86 This section describes how to integrate a new IBM Java SDK (JDK) into a Linux Boot Image on the FlexFrame environment without going through a maintenance cycle.

The IBM SDK can be downloaded from:

https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=javasap To install the IBM Java, refer to the section Integrating a new Java SDK (JDK) in Linux SLES8 / SLES9 32bit on how to proceed the installation.


8 Database in General

8.1 Installation of Oracle in General

Installing more then one Oracle databases on a host: In a FlexFrame environment, Oracle binaries are installed to a shared filesystem. Thus you have to read the following SAP notes before installing: SAP note 98252 (Listener), 350251 (oraInventory) and 521230 (9i) or 819829 (10g) (Oracle client software)

Before you start installing Oracle, make sure that all the required directories have been created.

If the following error occurs during installation, check the mount option of all Oracle directories. nolock: ORA-01102: cannot mount the database in EXCLUSIVE mode On Solaris it has to be llock whereas on Linux it has to be nolock.

Oracle Parameter: After successful installation of the oracle software, please check the value of the following parameters (init.ora / spfile.ora): disk_asynch_io=FALSE filesystemio_options=directio

Before starting the Oracle RUNINSTALLER (runInstaller for patches or opatch for interim patches) please check the value of the environment variable ORACLE_BASE=/oracle/ o for the ora user (.dbenv*.*sh).

8.1.1 TNS Listener for Oracle Virtualization of SAP Services means that it is possible to run any Oracle instance on any Application Node. As a result, the listener task (start and stop) is performed automatically by the FlexFrame sapdb script . Each TNS listener can have its own listener name. In the /FlexFrame/scripts/ ora_listener_names configuration file, an individual listener name can be assigned to each . If the entry for an is omitted, the standard listener name LISTENER is valid. Example for /FlexFrame/scripts/ora_listener_names: #: C11:MY_LISTENER C12:YOUR_THING P10:LISTENER_P10 P12:P12

Database in General


Technical background

This configuration file for the SAP Service start and stop scripts is /FlexFrame/scripts/sapdb. sapdb starts the TNS listener in accordance with these entries.

To avoid TNS errors, please do not forget to configure these individual listener names in listener.ora, too.

Oracle 10 only LSNRCTL: Check if the correct host name is used in the following files: /oracle//10[1-2]_64/network/admin/lsnrctl.ora /oracle//10[1-2]_64/network/admin/tnsnames.ora /sapmnt//profile/oracle/tnsnames.ora

8.1.2 listener.ora

Every database instance must have its own listener.ora. The relevant parameter is HOST in the section LISTENER.

The example below is valid for Solaris only. On a Linux platform the IP address of db-se has to be entered. This also applies for tnsnames.ora. Otherwise the startup of the lsnrctl will fail with Segmentation error.

Example for a virtual DB host:

LISTENER = (ADDRESS_LIST = .. .. (ADDRESS = (COMMUNITY = SAP.WORLD) (PROTOCOL = TCP) (HOST = db-se) # Virtual host of the listener (PORT = 1527) ) )

Database in General


Example for the ADDRESS_LIST section: LISTENER = (ADDRESS_LIST = (ADDRESS= (PROTOCOL=IPC) (KEY= .WORLD) ) (ADDRESS= (PROTOCOL=IPC) (KEY= ) )

Example for the SID_LIST section: (SID_LIST = (SID_DESC = (SDU = 32768) (SID_NAME = ) (ORACLE_HOME = /oracle//_) # Must be available per automount )

8.1.3 Host Name Changes in "tnsnames.ora"

The physical node name must not be entered here as HOST. The virtual node name has to be entered here; the database is not tied to a physical node, but only to a virtual node.

The example below is valid only for Solaris. On a Linux platform the IP address of db-se has to be entered. Otherwise the startup of the lsnrctl will fail with Segmentation error.

Database in General


Example:

.WORLD= (DESCRIPTION = (SDU = 32768) (ADDRESS_LIST = (ADDRESS = (COMMUNITY = SAP.WORLD) (PROTOCOL = TCP) (HOST = db-se) # Virtual host of the database (PORT = 1527) ) ) (CONNECT_DATA = (sid = ) (GLOBAL_NAME = .WORLD) ) )

8.2 Installation of Oracle 9.2

During the Oracle DB installation, a password for ora has to be set. You may type in any password you want to, it has no effect. Only ff_scripts can change a user password.

Linux Installation of Oracle patch 9.2.0.7: Check the existence of the following directory: /oracle/ora-config/ORCLfmap

Solaris Before you start the Oracle installation, create the following link and directory: mkdir /oracle/ora-config/ORCLfmap

After the installation has been completed, make sure to import the latest patch set (read the SAP Notes 539921, 539922, 355776, 509314, and 540021).

The installation procedure will ask for /oracle//920_ (Linux) or /oracle//920_64 (Solaris) and /oracle/client. All directories will be configured automatically with LDAP. They are also part of the /etc/auto.master file. Start installation as described in the SAP installation manual.

Database in General


8.2.1 Post-Oracle 9.2 Installation Steps

After installation, execute root.sh and use the parameter /oracle/ instead of the default value (/usr/local/bin).

BLADE1:/oracle/C11/920_ # ./root.sh Running Oracle9 root.sh script... The following environment variables are set as: ORACLE_OWNER= oraC11 ORACLE_HOME= /oracle/C11/920_ Enter the full pathname of the local bin directory: [/usr/local/bin]: /oracle/C11 Copying dbhome to /oracle/C11 ... Copying oraenv to /oracle/C11 ... Copying coraenv to /oracle/C11 ... Adding entry to /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed.

8.2.2 Patch Installation Please have a look to corresponding SAP-Sites to get the newest Patches. Before continuing SAP-Installation install the patches as described in corresponding readme files (either patch contains own runinstaller or patch is installed with Oracle's opatch command interface).

8.3 Installation of Oracle 10.1 (R1)

Linux Installation: Check the existence of the following directory: /oracle/ora-config/oracle

LSNRCTL: Check if the correct host name is used in the following files: /oracle//101_/network/admin/lsnrctl.ora /oracle//101_/network/admin/tnsnames.ora /sapmnt//profile/oracle/tnsnames.ora

Database in General


8.3.1 Script root.sh


Example: SID=OLP

BLADE1:/oracle/OLP/101_64 # ./root.sh Running Oracle10 root.sh script... The following environment variables are set as: ORACLE_OWNER= oraolp ORACLE_HOME= /oracle/OLP/101_64 Enter the full pathname of the local bin directory: [/usr/bin]: /oracle/OLP Copying dbhome to /oracle/OLP ... Copying oraenv to /oracle/OLP ... Copying coraenv to /oracle/OLP ... Adding entry to /etc/oratab file... Entries will be added to the /etc/oratab file as needed by Database Configuration Assistant when a database is created Finished running generic part of root.sh script. Now product-specific root actions will be performed. Successfully accumulated necessary OCR keys. Creating OCR keys for user 'root', privgrp 'root'.. Operation successful. Oracle Cluster Registry for cluster has been initialized /bin/cp: cannot create regular file `/etc/init.d/init.cssd': Read-only file system /bin/ln: creating symbolic link `/etc/rc.d/rc2.d/S96init.cssd' to `/etc/init.d/init.cssd': Read-only file system

The deamon cssd is only necessary for ORACLE RAC. You can ignore the error message regarding Read-only file system.

Solaris: The full pathname of the local bin directory is also /oracle/. However you will not receive such an error message.

8.3.2 Patch Installation Please have a look to corresponding SAP-Sites to get the newest Patches. Before continuing SAP-Installation install the patches as described in corresponding readme files (either patch contains own runinstaller or patch is installed with Oracle's opatch command interface).

Database in General


8.4 Installation of Oracle 10.2 (R2)


LSNRCTL: Check if the correct host name is used in the following files: /oracle//102_/network/admin/lsnrctl.ora /oracle//102_/network/admin/tnsnames.ora /sapmnt//profile/oracle/tnsnames.ora

8.5 Installation of SAPDB / MaxDB

For SAP DB 7.3 NetApp ONTAP Version 7.0.1R1P1 or higher GD release is required.

SAPDB 7.3: If you receive the following error messages, the process for calculating the database space could not check the available space on NFS mounted file systems: Sum of all datadevspaces to small for the SAP System Please change the value of the parameter MIN_DB_SIZE in the DATABASE.R3S file: Section: [ADMINDBSIZE_IND_ADA] MIN_DB_SIZE=15000

Database in General


The following screenshot shows an example for the SAPMS4 user:

Afterwards, the transaction db50 has to be called according to OSS note 588515.

8.5.1.1 Same SID on Different Platforms

When installing a SAPDB/MaxDB database with the same SID on different platforms, i.e. an application server on Linux and the central instance on Solaris, there is a problem with the compatibility of the file .XUSER.62. To solve this problem, the following activities should be performed:

Create a subdirectory /sapdb//XUSERD on both platforms (on the NetApp Filer they will be located in different directories containing the OS).

Move the corresponding file /home_sap/adm/.XUSER.62 to the subdirectory you created (repeat this step for the other platform).

Create a soft link as follows:

cd /home_sap/adm ln -s /sapdb//XUSERD/.XUSER.62

Database in General


Due to the fact that LDAP recognizes the OS, the correct /sapdb directory will be mounted.

During the installation of a dialog instance, you are asked to enter the host names of the central and database instances. We recommended using the virtual names (e.g. ciml4, dbml4-se) in this case since the file .XUSER.62 is replaced when the dialog instance is installed and must contain the virtual database name.

8.5.2 MaxDB 7.5/7.6 /etc/opt/sdb Before installing MaxDB 7.5/7.6 check the existence of file named /etc/opt/sdb. If the file does not exist create it manually and insert these lines:

[Globals] IndepData=/sapdb/data IndepPrograms=/sapdb/programs SdbOwner=sdb SdbGroup=sdba

Copy the file to the /sapdb/sql/sdb directory (from the Application Node). Check / Create the following directories (from the Application Node):

an1:~# mkdir /sapdb/data/config an1:~# mkdir /sapdb/data/config/install

Database in General


8.5.3 MaxDB 7.6

During the MaxDB 7.6 DB installation, a password for sqd has to be set. Please type in the correct password (defined in LDAP) of the user sqd.

Database in General


8.5.4 IO Balancing The SAPDB parameter _FBM_LOW_IO_RATE (currently only valid for releases lower than 7.4) can be set to 0 to avoid any balancing of disk usage between devspaces and to avoid higher snapshot sizes (the default value is 10). This means IO balancing will be stopped if the IO rate of all devspaces per second is higher than the parameter value 10.

8.5.5 Directories for sapdata / saplog The path to the SAPDATA directories is /sapdata/sapdata (analogously for SAPLOG); a link (only versions lower MaxDB 7.6) from /sapdb//sapdata is created automatically.

Except sapinst from SAP Version 6.40, the SID has to be added. Please enter /sapdata/sapdata1/, e.g.: /sapdata/sapdata1/P11 Set the SAPLOG volumes accordingly. /saplog/saplog1/P11

8.6 SAP Installation with Database DB2 We support DB2 database only at administrative level. That means you cannot specify a SAP system with DB2 database within the Planning Tool.

8.6.1 Software Requirements To run DB database within a SAP system you need at least the software shown below:

FlexFrame for SAP, 4.1A00 FlexFrame Autonomous Agents, V30K15 up SAP NetWeaver 2004s (SR1/SR2) R/3 4.7x200 with kernel release 640, Patch 189 R/3 4.6C SR2 with Kernel 4.6D_EXT 64-Bit, Patch 23xx DB2 LUW 9.1 FP3SAP for Linux x86_64_Bit (Download from http.//service.sap.com) DB2 LUW 9.1 FP3SAP for Solaris (Download from http.//service.sap.com)

Optional it could be necessary to install DB2 LDAP-Plug-in from

http://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=swg-dm-db2ldap

It needs a registration at IBM company.

Database in General


8.6.2 Pre-Installation Steps Stop the FA-Autonomous Agents during installation time

Create a new SID instance using ff_sid_adm.pl: ff_sid_adm.pl --op add --pool pool1 -sid LB4 --sapversion 7.0 --db DB2V91:dblb4-se:10.10.12.159 --sap ci:57:cilb4:10.10.10.57:cilb4-se:10.10.12.57 --sap ascs:55:ascslb4:10.10.10.55:ascslb4-se:10.10.12.55 --sap app:58:app58lb4:10.10.10.58:app58lb4-se:10.10.12.58 --sap scs:56:scslb4:10.10.10.56:scslb4-se:10.10.12.56 --db2srvsapdb2LB4:60000,DB2_db2lb4:60001,DB2_db2lb4_1:60002, DB2_db2lb4_2:60003,DB2_db2lb4_END:60004 The script ff_sid_adm.pl prepares the system files /etc/hosts and /etc/services of the application nodes known in the specific pool. The script writes a log file in /FlexFrame/volFF/pool-pool1/db2 to show you the changes made in system files. There will also a log file with LDIFs of all LDAP modifications done with this SID.

Run ff_sid_mnt_adm.pl if you want to relocate your directories for SAPDATA/SAPLOG to a volume of your own.

Run ff_setup_sid_folder.sh: ff_setup_sid_folder.sh pool1 LB4

8.6.3 Installing DB2 Today the installation is not fully integrated within sapinst. So you need to perform an additional step to make the DB2 software available for the installation of SAP later.

There are two ways to install the database software. Both methods are possible. The second method should be sufficient with SAP installation.

We recommend you to install the software to an SID specific directory. Our recommendation would be /db2//. Install software using db2setup.

On the path your DB2 software DVD is available (example Linux) you make a switch to /LINUXX86_64/ESE/disk1 and call ./db2setup. You have to choose an standard installation with no creation of a DB2 instance. Use the dasusr for your installation. dasusr1 is available with FlexFrame. If you want to use another one you must fulfill the requirement of the users name,

Database in General


dasusr. You can create that user using ff_user_adm.pl. The primary group of the user is always dasadm. We recommend to use the predefined group dasadm1. Otherwise you can create a group of your own by ff_group_adm.pl from the Toolbox-CD.

Install software using response file

You create an response file with the following content:

PROD=ENTERPROSE_SERVER_EDITION FILE=/db2/LB4/db2soft LIC_AGREEMENT=ACCEPT INSTALL_TYPE=TYPICAL LANG=EN CONFIG_ONLY=NO

and run

db2setup I en l -t -r In both cases you are complaint that the installation could not handle the link from /usr/local_bin/db2ls to /db2/LB4/db2soft/install/db2ls. Please create that link after running db2setup, not before !!! If you do not install DB2 for the first time, delete the link before running db2setup and create it again afterwards. Otherwise db2setup will not finish with a log message. In Linux you create the link on control node using (example) /FlexFrame/volFF/os/Linux//root_img/usr/local/bin: ln s /db2/LB4/db2soft/installation/db2ls db2ls

8.6.4 Installing DB2 License With R/3 4.7 the DB2 instance is created manually before SAPINST will be started with DB2 instance installation and the license files must be applied before SAPINST is called (see exceptions in document for R/3 4.7).

The unlimited DB2 license can be downloaded from http://service.sap.com (please refer to SAP note 816773). Copy it to a file system that can be accessed from an application node.

Install the license as db2 user by the following commands: db2licm a db2ese_c.lic db2licm a db2dpf.lic db2licm a db2so.lic db2licm a db2hf.lic

Database in General


You check the license installation with

db2licm l You should have installed permanent licenses for

DB2 Enterprise Server Edition DB2 database partitioning DB2 storage optimization Homogeneous federation for DB2

8.6.5 LDAP-Plugin Sometimes it is possible to get error messages while connecting to the database. In that case we recommend to install the available LDAP-Plugin. You can request that Plugin from IBM site

http://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=swg-dm-db2ldap

Copy the IBMLDAPauthserver.so (example Linux) to /home_sap/db2/sqllib/security64/plugin/server. Copy the IBMLDAPSecurity.ini to /home_sap/db2/sqllib/cfg and adapt the following lines to your system environment:

LDAP_HOST = -se -se USER_BASEDN = ou=People,ou=pool1,ou=Pools,,dc= GROUP_BASEDN = ou=Group,ou=pool1,ou=Pools,,dc= SEARCH_DN = cn=root,ou=LDAPadmins,ou=FlexFrame,,dc= SEARCH_PW =

Database in General


8.6.7 Changes in System Files The virtual db hostname (db-se) is written to /etc/hosts by ff_sid_adm.pl. The services needed for DB2 are inserted into /etc/services (you can look for the service names also into LDAP searching for services like DB2_db2 and sapdb2). ff_sid_adm.pl tries to modify the system files of all possible files in a pool. But if you exchange your image or run ff_new_an.sh the changes are (maybe) lost. In that case you should look into the logs written by ff_sid_adm.pl mentioned above and restore the changes manually.

8.6.8 Post Installation Steps After you installed your database you have to adapt the database catalog using (executed as db2): db2 uncatalog db db2 catalog db as l db2 uncatalog node NODE db2 catalog tcpip node NODE remote db-se server sapdb2 remote_instance db2 db2 catalog db l as at node NODE

If you executed these command, you should have a configuration like this:

db2 list database directory

System Database Directory Number of entries in the directory = 2

Database 1 entry:

Database alias = Database name = L Node name = NODE Database release level = b.00 Comment = Directory entry type = Remote Catalog database partition number = -1 Alternate server hostname = Alternate server port number =

Database in General


Database 2 entry:

Database alias = L Database name = Local database directory = /db2/ Database release level = b.00 Comment = Directory entry type = Indirect Catalog database partition number = 0 Alternate server hostname = Alternate server port number =

db2 list node directory

Node Directory Number of entries in the directory = 1

Node 1 entry: Node name = NODE Comment = Directory entry type = LOCAL Protocol = TCPIP Hostname = db-se Service name = sapdb2


9 SAP in General

9.1 ASCS ABAP Central Service Instance in General

A central services instance consists of the message server and the enqueue server:

Message server

Only one message server can run on each AS Java or AS ABAP usage type. The message server handles the communication between the dialog instances and also supplies information to the SAP Web dispatcher about load balancing.

Enqueue server

The enqueue server contains a lock table that handles logical database locks plus infrastructure locks set by Java server process. The enqueue server also synchronizes data in a Java cluster. In usage type AS ABAP, the enqueue server handles only locks on data objects.

9.2 SAP Java Cryptographic Toolkit During the SAPINST installation process you will be asked about the SAP Java Cryptographic Toolkit. If you select this option, some files of the JAVA directory have to be modifiable.

However, the root file system (Linux) where the java directory is located is mounted read only (/usr file system at Solaris) and can not be modified. In that case, logon to the Control Node, mount the resources of the OS images and do the following steps:

9.2.1 Linux

SLES9_x86: JAVA_HOME is /opt/IBMxxxxxxxx The example below shows SLES8 (SUN Java)

control1:~#> cd /FlexFrame/volFF/os/Linux/FSC3.2xxx/root_img/usr/java control1:~#> cp -rp mount -t nfs filerxxxx:/........var-xxxxxxxx/ /usr/java/ (e.g. = j2sdk1.4.2_12) Now the JAVA files are writeable.

SAP in General


If the installation has finished successfully, do the following:

appnode1:~#> umount /usr/java/ control1:~#> cd ..../var_img/var-xxxxxxxx/ c

Date post:	12-Oct-2015
Category:	Documents
Upload:	spacnav
View:	72 times
Download:	0 times