+ All Categories
Home > Documents > DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance,...

DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance,...

Date post: 25-Apr-2018
Category:
Upload: trinhanh
View: 287 times
Download: 3 times
Share this document with a friend
19
TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors 17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany „Deep dive into DB2“ Bruno Friess DB2
Transcript
Page 1: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

„Deep dive into DB2“

Bruno Friess

DB2

Page 2: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

Agenda:

TSM/DB2 installationDownload, installation, and then: where is it?DB2 instance creation

DB2 process overviewMulti­threaded engineProcess and threads

Settings for dbm and dbTuning for performance, scalabilityMemory and virtual memory manager

Database reorganizationDB2 logs

db2diaglogFODC, trace...

HADR – high availability disaster recovery

Page 3: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

Download and installation

Where to download?http://ftp.software.ibm.com/storage/tivoli­storage­management/ is still there.

● You can use IBM FixCentral as well.● Never use double­zeros (6.3.0.0 etc.)● There are maintenance and patch levels● Packages including the license files are available for paying customers

It's one file. Extract the file and start with installation over CLI or Java.

Installation logfiles are located in /var/tivoli/tsmDefault DB2 Install Path is /opt/IBM/db2/V9.7  but TSM uses /opt/tivoli/tsm

If the installation is not successful:Use the un­installer located in /opt/tivoli/tsm/_uninstIf DE hangs: delete /var/ibm/common/acsi and /usr/ibm/common/acsi

TSM installation is using db2_install, which will install all needed packages, but will not perform user and group creation, instance creation, or configuration.TSM un­installation is using db2_deinstall.

Page 4: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2? Where is it?

After a successful installation, there is no rpm or lpp inventory.rpm -qa  or  lslpp -l  doesn't show anything about DB2.

Useful: /usr/local/bin/db2ls

With version 6.2:> db2lsInstall Path Level Fix Pack Install Date Installer UID ---------------------------------------------------------------------------------/opt/tivoli/tsm/db2 9.7.0.2 2 Sun Feb 27 17:58 2011 CET 0

With version 6.3:> db2ls

Install Path Level Fix Pack Install Date Installer UID ---------------------------------------------------------------------------------/opt/tivoli/tsm/db2 9.7.0.5 5 Wed Mar 20 22:16 2013 CDT 0

Page 5: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 instance creation

A DB2 instance is an environment in which you store data and run applications(= a database manager, a virtual server environment or „container“). Run the db2icrt command. 

DB2DIR/instance/db2icrt ­a AuthType ­u FencedID InstName

Using the user tsm01 and instance tsm01:> db2icrt -a SERVER -u tsm01 tsm01

Three remaining commands...> db2 update dbm cfg using diagpath <any_path>> db2 update dbm cfg using dftdbpath <home_of_user>> db2set -i tsm01 DB2CODEPAGE=819

...end of TSM installation guide.(Other instance commands: db2idrop, db2ilist...)

→ Now? What is going on? 

fenced user ID instance name

Page 6: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 process overview

FIREWALL

DB2 Data Serverdb2sysc

Application Level Database Level per Request

Instance Level db2start

dbsysc

db2ipccm

RemoteListeners

dbaiothr

RemoteClientAgent

LocalClient

Application IdleAgentPool

db2agent

db2agent

db2agntpdb2agntp…db2agntp

db2loggwdb2pfchrdb2pclnrdb2loggrdb2dlock

db2lmrdb2med

db2wdog

db2vend

db2fmp

db2acd

tiny hole for the NSA

Page 7: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 common threads

db2syscSystem controller thread, which is responsible for startup, shutdownand management of the running instance

db2agentCoordinator agent that performs database operations on behalf of applications

db2agntpActive subagent spawned if INTRA_PARALLEL is set to YES

db2pfchrAsynchronous bufferpool prefetcher (NUM_IOSERVERS)

db2pclnrAsynchronous bufferpool cleaner/writer (NUM_IOCLEANERS)

db2loggrManipulates log files to handle transaction processing and recovery

db2loggwWrites log records to the log files

db2dlockThread responsible for deadlock detection

db2panic„panic agent“ writes dump/log files during a crash (FODC) or when a limit is reached

Page 8: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 common processdb2sysc

Main DB2 system controller or enginedb2acd

Autonomic computing daemon. Performs client side autonomic tasks, such as healthMonitor, maintenance (reorg) and scheduling

db2wdogDB2 watchdog daemon cleans up resources if db2sysc abnormally terminates

db2vendFenced vendor process runs outside of the engine

db2fmpFenced processes that run user code outside the firewall for stored procedures and user defined functions

> ps -ef | grep db2root 10193 1 0 Aug06 ? 00:07:59 /opt/tivoli/tsm/db2/bin/db2fmcd root 12360 1 0 Aug06 ? 00:00:00 db2wdog 0 rlin01 12373 12360 0 Aug06 ? 02:27:38 db2sysc 0 rlin01 12389 12360 0 Aug06 ? 00:10:22 db2acd 8a6678,14,1e014,2,0,1,11fc0,388009,2,6800crlin01 13287 12360 0 Aug06 ? 00:11:20 db2fmp (8a6678,14,1e014,2,0,1,31fc0,388009,2,77802frlin01 24860 12360 0 Aug06 ? 00:00:08 db2fmp (8a6678,14,1e014,2,0,1,91fc0,388009,2,ab68039

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12373 rlin01 18 0 2164m 300m 258m S 0.3 7.9 147:40.70 db2sysc 17448 root 15 0 12764 1148 824 R 0.3 0.0 0:00.20 top

Linux top

Page 9: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 settings for dbm and db

Settings for the database manager> db2 get dbm cfg

Settings for the database (you have to connect to TSMDB1)> connect to TSMDB1 > db2 get db cfg> disconnect TSMDB1

IBM: „No change needed“

Before changing, please consider the TSM 6.4 performance guide.Read the DB2 manuals, especially the overview of the performance impact.Some of them needs a stop/start.

How to change/update:> db2 update dbm cfg using <parameter> <value>> db2 update db cfg using <parameter> <value>

To reset all (!!) values to their default:> db2 reset dbm cfg> db2 reset db cfg

Page 10: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 settings for dbm and db

Potentially parameters of dbm to change:

name online impact explanation

dftdbpath yes none Path to the database meta-data

diaglevel yes low Sets the diag level, default is 3, range is 0-4

diagpath yes none Path to the DB2 diag files

intra_parallel no high Enables/Disables Intrapartition parallelism IBM Lab recommends „NO“, default is -1

max_querydegree yes high Max number of parallel threads in one SQL statementIBM Lab recommends 1, default is ANY (or -1)

notifylevel yes medium Admin notification level (file instance.nfy), default is 3, 1 is enough

Potentially parameters of db to change:

name online impact explanation

auto_maint yes medium Maintenance values (Reorg, runstats, auto db backup..)

dft_degree yes high Switch off intrapartition parallelism, default is 1 IBM Lab recommends „1“

longindexbuild yes none Needed for HADR with ON, default is OFF

and: newlogpath, mirrorlogpath, overflowlogpath, failarchpath...

Page 11: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 memory 

STMM – Self tuning memory manager

Default setting for TSM is to configure DB2 memory usage to AUTOMATIC.

DBMemPercent Auto

Within „get db cfg“ you will see:Self tuning memory (SELF_TUNING_MEM) = ONSize of database shared memory (4KB) (DATABASE_MEMORY) = AUTOMATIC(135180)Database memory threshold (DB_MEM_THRESH) = 10Max storage for lock list (4KB) (LOCKLIST) = AUTOMATIC(4120)Percent. of lock lists per application (MAXLOCKS) = AUTOMATIC(97)Package cache size (4KB) (PCKCACHESZ) = AUTOMATIC(29586)Sort heap thres for shared sorts (4KB) (SHEAPTHRES_SHR) = AUTOMATIC(268)Sort list heap (4KB) (SORTHEAP) = AUTOMATIC(53)

In the db2 diaglog you will see: CHANGE : CFG DBM: "Instance_memory" From: "20" <automatic> To: "780850" <automatic>

Log information is written into: [DIAGPATH]/stmmlog

Page 12: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 memory 

What does this mean?

If you have one TSM server on one machine, then automatic is ok.If not, then it is difficult: TSM allows only a range from 10 to 99% of the memory.

What if you have one machine with 32 GB RAM and three instances?What happens when you have 512 GB RAM and one instance? Or 20 instances??

TSM is using:> db2pd -dbptnmemDatabase Partition 0 -- Active -- Up 1 days 17:07:21 -- Date 09/13/2013 08:25:21Database Partition Memory Controller StatisticsController Automatic: YMemory Limit: 119984660 KBCurrent usage: 66347392 KBHWM usage: 74097600 KBCached memory: 6702912 KB

Individual Memory Consumers:

Name Mem Used (KB) HWM Used (KB) Cached (KB)========================================================APPL-TSMDB1 58048 160000 10624DBMS-tsmp01 126464 126464 48448FMP_RESOURCES 22528 22528 0PRIVATE 157376 409280 32320DB-TSMDB1 65967104 73406400 6611520

Total memory

TSM + DB2

db2sysc

Page 13: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

Database reorganization (see IBM technote 1452146 or the presentation of N.Pott)Is reorg needed?  db2 reorgchk current statistics on table all >db2reorgchk.out

Settings of the database:Automatic maintenance (AUTO_MAINT) = ON Automatic database backup (AUTO_DB_BACKUP) = OFF Automatic table maintenance (AUTO_TBL_MAINT) = ON Automatic runstats (AUTO_RUNSTATS) = ON Automatic statement statistics (AUTO_STMT_STATS) = ON Automatic statistics profiling (AUTO_STATS_PROF) = OFF Automatic profile updates (AUTO_PROF_UPD) = OFF Automatic reorganization (AUTO_REORG) = OFF

Is Reorg on?

TSM options:(6.1.5/6.2.3 or higher)

all are onlinechangeable(setopt)

Page 14: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

Database reorganization ….now it's reorganized, but I still have no more space in the filesystem.To get the free space which is at the end of the tables now, try this:

You can reduce the space of the database in the filsystem by „reducing“ it.This is for DB2 9.7 and only if this feature is switched on.

> db2 connect to tsmdb1> db2 set schema tsmdb1> db2 "select reclaimable_space_enabled from \table(mon_get_tablespace('',-1)) where tbsp_id in (2,4,5,6)"

If all tablespaces show, that reclaimable_space_enabled=1 then you can start with reducing:

> db2 ALTER TABLESPACE USERSPACE1 REDUCE MAX> db2 ALTER TABLESPACE IDXSPACE1 REDUCE MAX> db2 ALTER TABLESPACE LARGESPACE1 REDUCE MAX> db2 ALTER TABLESPACE LARGEIDXSPACE1 REDUCE MAX

Remarks:● All those ALTER process are slow, please start one after another● Can be done during TSM operating (online)● But consider I/O on your disks / filesystems

Page 15: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 log files 

Set the diaglog size and location:> db2 update dbm cfg using diagpath <path_to_diaglog>> db2 update dbm cfg using diagsize 1024

If you have more space, then >1024 is ok.

Inside this log, there are a lot of messages and most of them are difficult to read.IBM needs this file for error analysis.

Content of the [DIAGPATH]-rw-rw-rw- 1 rlin02 tsmsrv 28765146 Sep 13 14:52 db2diag.0.log-rw-rw-rw- 1 rlin02 tsmsrv 58231 Oct 15 2011 db2diag.log-rw-r----- 1 rlin02 tsmsrv 6291316 Aug 6 15:29 db2eventlog.000drwxrwxrwx 2 rlin02 tsmsrv 8192 Sep 12 20:55 eventsdrwxr-x--- 3 rlin02 tsmsrv 8192 Aug 6 10:31 FODC_Panic_2013-08-06-10.30.49.236060-rw-rw-rw- 1 rlin02 tsmsrv 67042 Sep 13 14:41 rlin02.0.nfy-rw-rw-rw- 1 rlin02 tsmsrv 763 Oct 15 2011 rlin02.nfydrwxrwxr-t 2 rlin02 tsmsrv 8192 Sep 12 08:27 stmmlog

DB2 memory logs

Admin notification

Automatic FODC

FODC – First occurence data collection(a kind of dump files collection)Old FODC directories can be deleted / archived

0 to 9

Page 16: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

DB2 log files 

Strange files in [DIAGPATH] or in sqllib/db2dump:Hundreds or thousands of *.dump.bin, *.stack.txt, *.apm.bin and *.trap.txt.For more information see IC72708: „Dump files fills up filesystem“TSM will report „ANR1545W Add more space to the file system.“Check this place and delete the files (as a workaround)Old files can be deleted while TSM / DB2 is runningFiles are needed by manual FODC command

Page 17: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

HADRHigh availability disaster recovery.

It replicates data in real time from a primary database to one or more standby databases.Multiple standbys are introduced with DB2 10.1 (see TSM 7.1).

To use HADR with TSM no additional licensing is necessary!!!

Easy solutionIs integrated into DB2Only needs TCP (no additional software / hardware)Setup only requires a few configuration parameters 

(primary/standby network address, and hadr_syncmode)Only 3 commands to manage: start/stop/takeover HADRChoose the best sync mode for your business: sync, nearSync, async, superAsyncAutomatic reconfiguration after failover or role switch in multiple standbys

But...Multiple standbysDelayed replay 

starts with DB2 version 10

Page 18: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

HADRQuick installation...

1. Priming the database● Backup of the primary database to disk ● Drop database on second node● Restore database on second node

2. Use the same ports for this instance in /etc/services on both machines

3.  Configure HADRdb2 update db cfg for tsmdb1 using hadr_local_host exaix01.exstor.dedb2 update db cfg for tsmdb1 using hadr_local_svc 60010db2 update db cfg for tsmdb1 using hadr_remote_host exaix02.exstor.dedb2 update db cfg for tsmdb1 using hadr_remote_inst tsm01db2 update db cfg for tsmdb1 using hadr_remote_svc 60010db2 update db cfg for tsmdb1 using hadr_syncmode SYNCdb2 update db cfg for tsmdb1 using hadr_timeout 120

4. Start HADRdb2 start hadr on db TSMDB1 as standby   (on standby server)db2 start hadr on db TSMDB1 as primary   (on primary server)

5. Failover with HADRShutdown DB2 on primary serverdb2 takeover hadr on db TSMDB1 by force (on standby server)

Swap this on primary and secondary

Page 19: DB2 „Deep dive into DB2“ - exstor.de · „Deep dive into DB2 ... Tuning for performance, scalability Memory and virtual memory manager Database reorganization DB2 logs db2diaglog

TSM Symposium 2013: Tivoli Storage Manager: Future Expectations – Partners and Exhibitors

17.-20. September 2013, Hilton Hotel Gendarmenmarkt, Berlin, Germany

Bruno [email protected]: 52.511992,13.392902


Recommended