1

‘Why’ and ‘How’Migration Guide to ASE 11.9.2/12

Ian AnnisPrincipal ConsultantSybase Professional Services (UK) March 2000

UKSUG MasterclassUKSUG Masterclass

2

Presentation MissionPresentation Mission

Migration is something that no organisation should undertake lightly. It is a process that requires justification and planning in order to achieve a smooth transition to a new environment.

This session will cover the pre-upgrade, migration and post-upgrade issues that need to be addressed in performing a migration to ASE 11.9.2 and ASE 12.0

3

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

4

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

5

Why Migrate ?Why Migrate ?

Vendor Push End of Life (EOL) Policy

Version Support (Industry practice -> last 2 releases) Platform Support (Critical mass - support costs)

New Features Performance Enhancements (query optimiser, parallel support, ...) Functional Enhancements (row level locking, 64-bit support, ...) Availability Enhancements (HA, recovery speed, rebuild index, ...)

Emergency Bug Fixes (EBF)

Client Pull ‘Latest Version’ Syndrome Change in Environment

Platform change Performance Issues

Old application not keeping up with growth in business

6

Sybase End of Life (EOL) ExampleSybase End of Life (EOL) Example

End of Support Notification for Adaptive Server Enterprise 11.5.x on Compaq Alpha NT Adaptive Server Enterprise 11.9.1, and Adaptive Server Enterprise 11.9.3 on all platforms

Jan 19, 2000Dear Valued Sybase Customer:

This letter provides notification of the end of support for certain older versions of Sybaseproduct offerings. We are announcing the end of support for certain older versions of AdaptiveServer Enterprise. Specifically, Sybase will no longer offer support for the following products:

Product End of Support DateAdaptive Server Enterprise 11.5.x Jan 31,2001 (for Compaq Alpha NT)Adaptive Server Enterprise 11.9.1 Jan 31, 2001Adaptive Server Enterprise 11.9.3 Jan 31, 2001

June 30, 2001(for Silicon Graphics platform only)

In addition to the products listed previously, Sybase will also be discontinuing support for thefollowing hardware / operating system platforms.

Platform End of Support DateCompaq Alpha NT Jan 31, 2001

Until the End of Support Date, Sybase will fix only Priority 1 bugs for the above versions.

For all of our active support customers, Sybase will provide a free license upgrade to AdaptiveServer Enterprise 12.0. Sybase Professional Services is available to assist you, on a fee basis,with migration to these current versions of Sybase's database offerings. We stronglyrecommend that you upgrade to a newer version.

7

Sybase EOL Current PositionSybase EOL Current Position

End of Support Notification for Older Versions of Sybase Products

March 9, 1999Sybase will no longer offer support for the following products:

Product End of Support DateSybase SQL Server 11.0.3 Jun 30, 2000 (OpenVMS to 31/12/2003)Adaptive Server Enterprise 11.5.x Dec 31, 2000Adaptive Server Enterprise 11.9.1 Jan 31, 2001Adaptive Server Enterprise 11.9.3 Jan 31, 2001 (June 30, 2001 for SGI platform only)Open Client/Open Server 10.0.4 Dec 31, 2000 (OpenVMS to 31/12/2003)Replication Server 11.0.3 Jun 30, 2000

In addition to the products listed previously, Sybase will also be discontinuing support for thefollowing hardware / operating system platforms….

As part of the rationalization of supported hardware/software platforms, a number of platformsare no longer supported from 1998. For lists of platforms which have been dropped, go tohttp://www.sybase.com/support/techdocs and look for the following Document IDs:

20363 20406

8

Architecture IssuesArchitecture Issues

What else is changing / needs to change Certification Issues

What does certification give you (Sybase monthly update) http://www.sybase.com/support/techdocs, Doc ID 1010821

Operating System Upgrades 32/64-bit support Feature enhancement / Bug Fix / Patches

Problem Upgrades HPUX 9->10 -> 11 Sun Solaris 2.5.1 - 2.6 with LVM version 2.4 (keep versions in line) NT Service Packs

Product Compatibility Sybase SQL Server 11.0.3 base product for Y2K Will it work with third party product ‘xyz’

Scope of overall change required

9

Sybase Adaptive Server ReleasesSybase Adaptive Server Releases

Former releases of Adaptive Server Enterprise (ASE) include: SQL Server 11.0 (released in Dec 1995)

SQL Server 11.0.3 is baseline release for Y2K compatibility Adaptive Server Enterprise 11.5 (released in Sept 1997) Adaptive Server Enterprise 11.9.2 (released in Oct 1998) Adaptive Server Enterprise 12.0 (released Dec 1999)

Supported upgrades to Adaptive Server 11.9.2 are from versions: - 10.0.2.7 - 11.0.3.3 - 11.5 - 11.5.1 - 11.9.1

If you are running a version of Adaptive Server or SQL Server that is earlier than those listed here, it is recommended that you upgrade to one of these versions before upgrading to 11.9.2.

10

Adaptive Server Enterprise (ASE)Adaptive Server Enterprise (ASE)

Designed for Performance OLTP/TPC-C benchmark records SMP linear scalability (LMM, LPM, Parallel Support)

Designed for 24*7 Operation Zero requirement for database downtime

On-line backup & recovery, and On-line maintenance tasks Resilience Architecture for:

Disks, Processors, Machine, Location High Availability (HA) support in ASE 12.0 OpenSwitch and Replication Server support for Warm Standby

Designed for Low Cost of Ownership Multi-threaded Kernel

< 100 KB memory per user more users supported per processor

DBA Overhead Less maintenance required (less scope for human error ?)

11

Changes in SQL Server 11.0.xChanges in SQL Server 11.0.x

New Server Engine LMM, LPM

ANSI SQL Level 2

Syntax

Display format

Data Conversions Money Integer to Char

New Optimizer

Subquery Processing

Aggregates with EXISTS

BETWEEN

Comments

Required some changes to application SQL

12

Key Features of ASE 11.9.2Key Features of ASE 11.9.2

Key New Features: Data Row Locking Performance Improvements for Dynamic SQL Better Query Optimization

Key Benefits Flexibility in locking granularity Dramatically higher performance with applications that have lock

contention No performance degradation for existing applications

13

11.9.2 Optimizer Statistics11.9.2 Optimizer Statistics

Statistics now stored in two system tables Column level stats (distribution, density values and date

and time of last modification) are stored in sysstatistics. Table and Index level statistics are stored in systabstats. Space for statistics is limited only by the size of the DB.

No guarantee of good performance Update statistics does guarantee up to date statistics. There are no rules about when to run update statistics. Statistics provide accurate information about the data to

the optimizer. The more statistics available the more information the optimizer has

to work with. Remember to run ‘sp_recompile <table_name>’ for pre-compiled

objects to access the updated statistics. New ‘optdiag’ tool to view and simulate statistics for DB tables

14

Key Features of ASE 12.0 (Avatar) Key Features of ASE 12.0 (Avatar) Key Focus AreasKey Focus Areas

High Availability (HA) Reduction In Planned Downtime Active-Active Fail-Over and Single Node

Recovery Enhancements

Performance and Scalability Re-design Optimizer’s Top Half

Query Processing Extensibility OLTP and DW / DSS Performance

Sort Merge Joins Abstract Query Plans

Native Object Support Import Java Classes into Database Objects = Abstract Data Types

Extend Native Type System Methods = User Defined Functions

Use in any SQL SELECT list or WHERE clause

Distributed Transaction Mgmt Full XA, MS/DTC and DTC-XA ASE as Transaction Coordinator

15

Migration SurveyMigration Survey

Who’s carried out an ASE Migration ?

What route did you use ?

Did you read the Installation Guide ?

How much pain did you suffer ?

16

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

17

Migration ProcessMigration Process

Planning & Pre-Upgrade Tasks Analyze existing system and business requirements Decide and prepare on migration route Plan migration tasks

Migration Tasks Implement migration route

Post-Upgrade Tasks Enhance system for new features Test performance and usability

18

Major SAFE/EM Migration ActivitiesMajor SAFE/EM Migration Activities

Planning the migration project Establishing the current configurations Documenting business requirements Developing compatibility analysis Planning for application and platform migrations

Administering the migration project Designing and developing migration scripts for

data servers Recording source and target configurations Migrating data and application code Performing quality-assurance testing and

corrective actions

19

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

20

Analyze Existing SystemAnalyze Existing System

Document Existing Architecture Servers, clients, network config and application description

H/W & S/W configuration and sizing

Operational Business Requirements Availability Requirements - allowable downtime periods Maintenance Procedures - backup plans & schedule Service Level Requirements

Current Performance Metrics Transaction profiles by application Server performance (sp_sysmon)

Additional Business Requirements Priority list for application migration and dependencies Resource issues (IT and human)

21

Analyze Sybase ConfigurationAnalyze Sybase Configuration

What is currently running record configuration file or ‘sp_configure’ record SQL interface file

What disk devices and their usage record current disk device configuration map databases and segments to devices record dump devices

Scripts for Database Objects Should already have them, else reverse engineer with:

Sybase Central (generate DDL) PowerDesigner DataArchitect Third party tools, e.g. Cyrano

22

Plan the UpgradePlan the Upgrade

Be Prepared at every stage Once you know what you have - you can identify what needs

to change

Advanced Reading Prepare yourself for the task in hand - read the manuals

Determine Migration Route Based on the requirements gathered to date, choose the

appropriate method Cutover without replication phased cutover parallel with replication

Develop Migration Plan Plan activities and resources required

23

Produce Migration PlanProduce Migration Plan

Plan the whole process Migration Strategy Fallback

what if it didn’t work, how do I stay operational Application Test Suite

plan and develop tests for functional and performance testing Bridging Strategy

how minimize impact to end-users (linked to migration strategy) Environment

consider the whole process - not just the ASE upgrade part Scheduling

plan when it happens set milestones and reviews to be able to know when each stage is

finished

24

Resources To Perform Migration ProcessResources To Perform Migration Process

Documentation Installation Guide (platform specific) Release Notes (platform specific) What New in ASE Release 11.9.x/12.x Performance & Tuning Guide System Administration Guide

Tech Support my.sybase.com Technotes: http://www.sybase.com/support/techdocs Manual: Migrating to Sybase ASE 11.5

Sybase Professional Services Installation & Migration Support (SAFE/EM) Performance & Tuning

25

Migration StrategyMigration Strategy

Determine Migration Strategy Cutover without replication

easiest option with minimal resource demands highest risk as it requires downtime during critical migration phases

and subject to lengthy recovery on fallback Phased cutover

enables managed/planned migration of an application at a time can use any route to achieve it’s aim dependant on risk/resource/..

Parallel with Replication minimal system downtime, hence good for 24x7 environments extra set-up and H/W resource required for replication server easy fallback to existing server easy comparison with existing system for performance, functionality

Note: Whenever possible, upgrade test and development databases first. Upgrade the production system after testing.

26

Migration Routes to ASE 11.9.2Migration Routes to ASE 11.9.2

Determine Migration Route ASE Upgrade Process

sqlupgrade to upgrade server - full cutover DIY (manual patch) - full or phased cutover

Fresh Installation of New Server BCP old databases - full or phased cutover Dump & Load old databases - parallel, full or phased cutover Replication Server - parallel or phased cutover

SybaseAdaptive Server

27

Migration Route ArchitectureMigration Route Architecture

SybaseReplication Server

Old Server New Server

New Serverexecutables

Database Object DDL +BCP Data

Database Dump + LoadTransaction Dump + Load

Sqlupgrade or DIY

BCP

Dump & Load

Replication

Sqlupgrade or svrbuild

28

Moving a Database to Another ServerMoving a Database to Another Server

If a database is to be reproduced exactly on the same hardware platform: Copy any needed definitions from the old database using

Sybase Central (generate DDL) or use existing scripts Make a copy of the old database using dump database Create a new database using create database (from scripts !) Load in the new database contents using load database

If the target database is not identical to the original database: Copy all definitions using Sybase Central Create the new database as needed using Sybase Central’s

DDL scripts Copy the data out of the old database using bcp Copy the data into the new database using bcp

29

Database UtilitiesDatabase Utilities

ASE Build Utilities An ASE database utility is a program executed from an

operating system prompt or GUI interface that assists in system administration srvbuild - a UNIX-based server installation utility srvbuildres - a UNIX-based server installation utility where a GUI

is not available Server Config - a Windows-NT based server installation utility dsedit - an editor for creating and modifying interfaces files bcp - a program that copies data from a database to an operating

system file, and vice versa optdiag - a program for detecting inefficient space usage in tables

using data-only locking sqlupgrade - a UNIX-based server installation utility to upgrade an

existing server

30

Single sybase directory Files for all components are stored in this directory

Important sybase subdirectories bin

All executables (isql, bcp, and so on) charsets

Character set and sort order localization files init

Installation log files install

Install programs, RUNSERVER files, the error log scripts

Database install scripts

Directory Structure: Pre-ASE 12.0Directory Structure: Pre-ASE 12.0

31

Directory Structure: Pre-ASE 12.0Directory Structure: Pre-ASE 12.0

32

ASE default character set and sort order The default character set depends on the platform

Windows platforms default to cp850 Digital Unix, IBM AIX, and Sun Solaris default to iso_1 HP defaults to roman8 These character sets support US English and include accented

characters for most European Languages The default sort order is binary sort order Alternative options can be chosen at installation time

Post installation character sets and sort orders Whilst they can be changed, it is not advisable as it requires

significant manipulation of existing data. These changes are not recommended unless absolutely necessary.

View the current character set and sort order sp_helpsort

Character Sets and Sort OrderCharacter Sets and Sort Order

33

Pre-upgrade ChecklistPre-upgrade Checklist

General preparation Read the documentation - installation guide to hand Verify software release with OS release + patches required Run the pre-upgrade option within sqlupgrade

check reserve words, system database sizes, DB options, etc. Check and backup databases

Run dbcc checkdb, dbcc checkalloc, and dbcc checkcatalog on all your databases including the master database.

If the dbcc's do not return any errors, make backups of all your databases including the master database.

Drain replication queues & stop replication (see Installation Guide)

Turn off Sybase mirroring if it is enabled

Upgrade your operating system first If possible, run existing SQL Server on the new operating system for several

days/weeks to troubleshoot any problems caused by the operating system change before upgrading to ASE 11.9.2/12

34

Backup Your DatabasesBackup Your Databases

Keep current backups of all databases prior to upgrading, regardless of the route chosen System Databases (master, model, sybsystemprocs) User Databases (as required) Bulk Copy output of master-only system tables to help in disaster

recovery scenarios from: sysdatabases sysdevices syslogins sysusages

You may need to rebuild master if: Adaptive Server fails during the upgrade process (disk problems,

etc.) to duplicate it on another machine it’s easier to keep up-to-date backups than to rebuild master from

scratch

35

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

36

Build the Server EnvironmentBuild the Server Environment

Prepare the IT environment for the upgrade Identify ASE release (and EBF) to be applied Prepare the H/W (sufficient resource) + OS patches as

identified in the Installation Guide New ASE releases usually increase memory requirements

Prepare scripts as appropriate for chosen Migration Path Server level scripts - disk init, server configuration, … Database Object scripts - reverse engineering or prepared scripts

Make required application changes remove new reserved words identified in prepared

(sp_checkreswords) changes in syntax, e.g. SQL92 support

Verify and Configure Existing Server if it is being upgraded increase memory allocated to ASE increase size of sybsystemprocs (to 60Mb) copy your own ‘sp_’ for loading after the upgrade

37

sqlupgrade Overviewsqlupgrade Overview

Automatic Phased Process using sqlupgrade The sqlupgrade Phases

Phase 1: Runs the preliminary test to make sure the upgrade will work properly.

Phase 2: Runs the upgrade program, which actually performs the specific upgrade tasks.

Phase 3: Completes the upgrade by recreating the sybsystemprocs database by running the installmaster and installmodel scripts.

The sqlupgrade log is located in $SYBASE/init/logs/sqlupgradeMMDD.NNN..

38

Sqlupgrade Phase 1 - pre-upgrade Sqlupgrade Phase 1 - pre-upgrade

Boot the previous version of SQL Server to: a. Copies the interface file entry from the earlier interfaces file to

the new interfaces file

b. Runs preupgrade tests, checking for free space and database options. The program is located in $SYBASE/upgrade/preupgrade

c. Checks for reserved word conflicts. File is created in $SYBASE/init/logs/checkres.dmp

d. Checkpoints all databases

e. Saves the database segment information in a file located in $SYBASE/init/logs/usage.sql. This file is removed after a successful upgrade.

f. Shuts down the original SQL Server

2. Creates a runserver file, RUN_servername, in the install dir 3. Starts the new server with a new configuration file,

$SYBASE/servername.cfg. This step recovers all databases.

39

Sqlupgrade Phase 2 - upgrade Sqlupgrade Phase 2 - upgrade

Run the $SYBASE/upgrade/upgrade to: a. create temp files to manage the progress of the upgrade

(sqlcmds.n) sqlcmds.0 - contains most of the upgrade's SQL commands; could be used to check

current status of upgrade

sqlcmds.1 - contains updates to system tables

sqlcmds.2 - contains generic functions and remapping commands; could be used to check the status of remapping the query trees.

sqlcmds.3 - contains a list of changes to the system table sysindexes

b. Ensures the old SQL Server version is supported for the upgrade.

c. Runs the appropriate tasks, depending on the version of SQL Server that is being upgraded. To view what is actually being done at this point, view the sqlcmds.* files above for more details.

d. Allows updates to system tables.

e. Sets the version of the SQL Server to the new version number. Once it has done this step you CANNOT rerun the upgrade program.

40

Sqlupgrade Phase 3 - new system build Sqlupgrade Phase 3 - new system build

Runs the $SYBASE/upgrade/upgrade to: a. Turns off allow updates to system tables

b. checkpoints all databases

c. Updates the runserver file.

d. Shuts down and restarts the new release Server.

e. Runs the installmaster and installmodel scripts.

Note: it is safer to drop and re-create all compiled objects (stored procedures, triggers, etc.) in the new environment to ensure new query plans are generated

41

Benefits With sqlupgrade RouteBenefits With sqlupgrade Route

Automated Upgrade Generally, no intervention is required

Speed As the data is not being upgraded or reloaded, then it is

generally a fast process to run through

When to use this approach Recommend for development/test systems

only use approach on production servers with care and confidence of success, and where a window exists to upgrade, I.e. not 24x7 applications

Acceptable approach for small to medium systems, or very large systems where an alternative server/disk space is not available

42

Issues With sqlupgrade RouteIssues With sqlupgrade Route

Lack Of Control How handle errors during the upgrade process One-way operation

can recover (same with EBF) to restart upgrade after correcting fault, e.g. read only file not updated, disk or log space, etc.

Recoverability Poor - see above

Must have backups to recover to Service may be unavailable whilst the executables and dumps are

re-loaded

43

DIY Route OverviewDIY Route Overview

Manual Process using srvbuild phase by phase As an alternative to using sqlupgrade:

Manually install new ASE release

create the databases then load the old server dumps from backup

When the online database function is run, it will upgrade the old database to the current release.

You cannot load the old master database dump, you can only load user databases.

Transactions logs can also be loaded in the same way to enable a phased hand-over between the two servers

44

Benefits With DIY RouteBenefits With DIY Route

Clean Installation No legacy issues to address on new server Starting point is clean installation containing only the

required system databases

Recoverability This is a safe method as it leaves the original Server intact

45

Issues With DIY RouteIssues With DIY Route

Requires more DBA effort (time issue)

See Dump & Load issues

46

BCP Route OverviewBCP Route Overview

Manual Process using srvbuild and BCP Build a new server and copy data across:

Manually install new ASE release using srvbuild

create the databases as per original size or larger

create the database objects with the prepared DDL scripts

load the old data from BCP files into the new server tables

rebuild indexes and triggers

47

Benefits With BCP RouteBenefits With BCP Route

Clean Installation No legacy issues to address on new server

Platform Independence Use -c option to pass BCP data in ASCII mode

Data Clustering Data loaded and allocated in extents contiguously Forced re-creating of all indexes (if using fast BCP) Statistics recreated by index creation

Device Allocation Easier to change device location/size as recreated from

scratch

48

Issues With BCP RouteIssues With BCP Route

Volume of data to migrate (time issue) Consider using fast BCP approach (no indexes, triggers,..)

Drop all the indexes and triggers on the table Copy the data into the table Recreate the indexes and triggers Dump the database (as operation is minimally logged)

Configure and run parallel BCP load into partitioned table Running concurrent sessions of large bulk copy jobs into specific

partitions substantially increases performance

49

Issues With BCP Route (cont)Issues With BCP Route (cont)

Downtime or incremental transaction update As soon as the data is BCP’ed out, no more transactions

can enter the old server unless a method is devised to capture any new transactions against the old server.

50

Dump & Load Database Route OverviewDump & Load Database Route Overview

Transfer data to new server on same platform Transactions must be loaded in the correct order, and they all have to be

there load transaction without prior load database will fail

Issuing a load database takes a database off line Eliminates administrator’s need to mark databases dbo use only when loading a database

and transaction logs set ‘online database <dbname>’ after loading is complete

Old Server New Server

51

Rules for Loading DatabasesRules for Loading Databases

Database Loading Restrictions Database must not currently be in use Can load only dumps made on same machine type The database into which you load must exist, and it must be

at least as large as the dumped database Use sp_helpdb to see amount of space allocated to a database

Loading data into an existing database overwrites its data; partial loads are not possible

To create a new database for loading, use the for load option to create database

Dump Compatibility When an Adaptive Server installation is upgraded to a new

release, all databases associated with that server are automatically upgraded Upgrade is on a per-database basis, for upgrading a database or

transaction log dump from any previous release to the current one

52

Benefits With Dump & Load RouteBenefits With Dump & Load Route

Speed Major factor is that the new server can be prepared

independently ready for the first dump. Dump & load is relatively fast - even for large volumes of

data, and database is upgraded when it is made online

Up-to-Date Keep applying transaction log dumps to keep new server

up-to-date Downtime for switch over is minimised

Fallback Easy to fallback as old server still exists in line until first

transaction in new server - same issue on capturing new transaction data

53

Issues With Dump & Load RouteIssues With Dump & Load Route

Compatibility Only works on same hardware platform (sometimes OS

release restrictions as well)

For load option New server with physical layout from previous server

maybe inefficient use of storage - need indexes re-created may have data loaded into log segments (see SA manual on

recovering for incorrect segment mapping after load)

Recoverability Once new data has been entered the database dump can

not be loaded into the previous version recovery to previous version would be via the BCP route.

54

Replication Route OverviewReplication Route Overview

Manual Process using srvbuild and Rep Server Build a new server and create replication configuration:

Manually install new ASE release using srvbuild

create warm standby configuration in replication server from old database to new database

materialise the data for the database

test and check performance of new server prior to switch-over

switch-over active database to new server and capture transactions back to the old server

55

Benefits With Replication RouteBenefits With Replication Route

Clean Installation No legacy issues to address on new server

Platform Independence Data conversions handled between servers

Availability and Recoverability Best method for true 24x7 operation where continuation of

service must be maintained Can be configured in warm standby mode to switch and

get new transactions being replicated into old server - allows for rapid fallback if problems are found on the new server

Valid functional and performance testing with up-to-date data prior to switch-over

56

Issues With Replication RouteIssues With Replication Route

Effort required (time & resource issue) Requires resources and skills to set up and use replication

server May already have expertise in-house

Materialisation process can take a long time on large production systems Ensure optimal configuration of replication server for the task in

hand Not a major issue as the operational system carries on running

during this process Ensure adequate spare capacity to handle materialisation overhead

on the old server

57

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

58

Testing the New ServerTesting the New Server

Stability & Performance Prior to Roll-out The primary goal of testing is to ensure that after migration:

Application behavior is predictable.

Application and operational service levels are preserved or exceeded.

The test and production systems are stable and the data is safe.

The upgrade is successful and does not adversely impact the production system

Note: testing strategies are not covered in this presentation

59

Verification and Use of New ServerVerification and Use of New Server

Verify the version level of Adaptive Server. To verify that you are at the new version level, connect to isql

and run the following commands: 1> select @@version

Look for “11.9.2” in the version string. 1> sp_configure "upgrade version"

sp_configure should return the Run Value “11920”.

Re-enable Replication If replication server is in use

reset the secondary truncation point restart replication server enable the rep_agent on appropriate databases

Configure Client connections

60

Post-Upgrade Steps - Update StatisticsPost-Upgrade Steps - Update Statistics

Update Statistics 11.9.x statistics have been improved now on all columns better support for large tables (more distribution steps) defaults to previous server statistics after pre 11.9.x load preferred method to update stats:

run optdiag on each user table (backup of pre-upgrade stats) run update statistics on each user table run sp_recompile on each user table check new stats with optdiag see: http://www.sybase.com/support/techdocs “Optimizer Statistics

After Upgrading to ASE 11.9.2”, Doc ID 20461

61

Post Upgrade Steps - Data Only LockingPost Upgrade Steps - Data Only Locking

Identify and Reduce Lock Contention Use ‘sp_sysmon’ and ‘sp_object_stats’ to identify possible

lock contention issues consider using (only where required)

‘alter table <tablename> lock datapages’ ‘alter table <tablename> lock datarows’

62

What if …..What if …..

Serious errors / server won’t start Resort to backup or previous server Switch client interface files back to old server Manual patching/upgrade of the problem (Tech Support)

Poor Performance Testing should have identified obvious performance

problems Run standard commands to identify problem area, e.g.

sp_sysmon, sp_object_stats (11.9.2), etc. Ensure statistics are up-to-date

What about new transactions Capture transaction text (difficult !) BCP out data daily to be able to recreate new data Replication of new transactions back to old server

63

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

64

Key Features of ASE 12.0 Key Features of ASE 12.0 Key Focus AreasKey Focus Areas

High Availability (HA) Reduction In Planned Downtime Active-Active Fail-Over and Single Node

Recovery Enhancements

Performance and Scalability Re-design Optimizer’s Top Half

Query Processing Extensibility OLTP and DW / DSS Performance

Sort Merge Joins Abstract Query Plans

Native Object Support Import Java Classes into Database Objects = Abstract Data Types

Extend Native Type System Methods = User Defined Functions

Use in any SQL SELECT list or WHERE clause

Distributed Transaction Mgmt Full XA, MS/DTC and DTC-XA ASE as Transaction Coordinator

HA - High AvailabilityHA - High Availability

ASE HA Objectives in this Release Reduction In Unplanned Downtime

Near-seamless Fail-Over of One ASE Node Onto Another, Cluster Configured, Companion Node

High Performance Fail-Over (No Start-Up Overhead) High Performance Recovery After Fail-Over Integration With HW/OS Vendor HA Systems

Reduction of Planned Downtime Back Up Server Performance Enhancements On Line Index Reorganization

66

SystemDB

systemprocs

security

Interfaces File

Master

User User

S1 Establish Companion

SystemDB

systemprocs

securityMaster

S2

ReplicateUsers/Logins

EstablishProxy DBs

Establish Companion

User

Interfaces File

HA System

HA - Establish CompanionHA - Establish Companion

HA - Reduction in Planned DowntimeHA - Reduction in Planned Downtime

Reduction in backup & recovery overhead Order of Magnitude Back-up and Restore Performance

Improvements

Reduction in time for index maintenance Random inserts and deletes in an index cause:

Drop in space utilization Declustering (Scan of page chain in key order may cause random

I/O) Online index rebuild preferred to recreating indexes using

reorg rebuild <tablename> <indexname> Copy all index rows to new pages and deallocate old pages under

an intent lock (an extent at a time) - DOL indexes only high concurrency operation with minimal logging and no deadlocks achieve clustering and desired space utilization goals

68

Java In The DatabaseJava In The Database

Java Is Becoming a Generally Accepted Computing Platform

Java Offers The Promise of Unified Programming Model Across All Tiers Develop, Debug, and Deploy Objects/Logic Components and Data

Objects Across Client, Data Stores and Middle Tiers

Java Offers a Rich Environment for Database Programming Complete, Object-oriented Programming Language Provides

Tremendous Extensions to SQL Capabilities Non-proprietary and Portable Language “Pointer-safe” Language for Integrity of Database Kernel

Java in Adaptive Server EnterpriseJava in Adaptive Server Enterprise

Goals in This Release: Extend ASE's T-SQL

Language Capabilities With Java

Use Java Classes As: User Defined Functions User Defined Datatypes

Embed Java Virtual Machine Inside the Server

Provide Utilities to Install Java Classes Into a Database

Enable Remote Java Debugging

70

DB 2ASE OracleNew York TokyoLondon

ClientApplications

Tra

nsac

tion

Ser

vers

and

TP

Mon

itors

Trends in Distributed ComputingTrends in Distributed Computing

Distributed Data Architectures Increasing in frequency

Business units make decisions Multiple database vendors

Yet, data is a corporate asset ! Some data requires a “tightly consistent”

model

N-tier Application Architectures Can provide high-end scalability

and fault tolerance Multiple Application Servers

providing same services

Both Require Support for Advanced Distributed Xact APIs, e.g.XA & DTC

What is Distributed Transaction What is Distributed Transaction Management (DTM) ?Management (DTM) ?

A Transaction Management Framework A framework consisting of 3 components

Transaction Monitor/Manager or Transaction Coordinator Application Client Resource Manager(s)/Dispenser(s)

A conceptual framework where several Resource Managers participate in doing a unit of work (transaction)

Transactional scope limits the failures occurring across execution domains

Transaction boundaries are demarcated with API Transaction outcome and commitment policies are

implemented within this framework XA and DTC are two models that are currently available in

industrial strength Tuxedo/Encina/TopEnd/MTS-MSDTC/CICS

DTM Support Within ASE 12.0DTM Support Within ASE 12.0

ASE Transaction Manager / Co-ordinator ASE Transaction Manager understands different

transaction types (Local, Internal, Subordinate and External)

The transaction states are validated through a state machine

Various Protocols supported by the external transactions (XA, ASTC, MSDTC) are driven through a protocol table

During recovery, prepared push-model external transactions are re-instantiated and Server is made available as soon as possible

Increased Query Execution OptionsIncreased Query Execution Options

Begins re-architecting “top half” of the Optimizer & Execution Engine for extensibility of new join and access strategies

First Extension Will Support Parallel and Serial Sort/Merge Joins Sort/Merge Joins Between OMNI Proxy Tables or Between Local

and OMNI Proxy Tables Smart Transformation of WHERE Clause Predicates Improved Selectivity Estimation for LIKE Predicates Join transitive closure Abstract Query Plans Support for up to 50 tables in a join clause Execute Immediate

Optimizer Enhancements in ASE 12.0Optimizer Enhancements in ASE 12.0

Sort Merge Joins Ordered joins provide clustered access to joining rows; result in less logical and

physical I/Os. Can exploit indexes that pre-order rows on joining columns or create work tables as

required Sort Merge Join Algorithm (4 distinct types supported) - Often Better Performance

for DW/DSS Queries Than Nested Loop Join of ASE 11.x LIKE Clause

New costing to improve selectivity and qualifying row estimates Join Transitive Closure

Provides the optimizer with additional join paths and, hopefully, faster plans. SARG transitive closure was added in ASE 11.5

Predicate Transformation Significant performance improvement in queries with limited access paths (i.e. very

few possible SARGS/Joins/OR’s that can be used to qualify rows in a table) Increase maximum number of non-RI tables per query

from 16 user tables and 12 work tables to 50 user tables and 14 work tables

Sort Merge Join ExampleSort Merge Join Example

PartClustered on

p_partkeyPartsupp

Clustered on ps_partkey

LineitemClustered on l_orderkey

PartClustered on

p_partkeyPartsupp

Clustered on ps_partkey Lineitem

Sorted on l_partkey

select … from part, partsupp, lineitemwhere p_partkey = ps_partkeyand ps_partkey = l_partkeyand ps_orderkey = l_orderkeyand p_type = ‘CD’

Unsorted Accessto innermost table

Sorted Accessto innermost table

76

Managing Optimizer Changes After UpgradeManaging Optimizer Changes After Upgrade

What happens to the installed base when the optimizer is enhanced? Most find it better but some find it worse…

What could go wrong with the Optimizer? Statistics may not apply to the data that is now in the table The query plan used for a stored procedure may not be

applicable to the query at hand The buffer cache model and the actual buffer cache usage at run

time could differ

One solution to all these problems would be to implement rules based optimization. However: Rule based decisions could be sub-optimal as they require the

developer to have a knowledge of the eventual data layout Developers very often have very little knowledge of how to write

efficient query plans and there is a considerable overhead in writing/testing them

77

Curing Unexpected Optimizer BehaviorCuring Unexpected Optimizer Behavior

What are the options for improving the optimizer and getting rid of unexpected behavior? Implementing a better and more dynamic cost model Implementing some form of extremely flexible rules based

optimization Allowing good query plans to be captured and re-used

78

Abstract Query PlansAbstract Query Plans

An abstract query plan is a persistent, human readable description of a query plan, that’s associated to a SQL statement

It is not syntactically part of the statement

The description language is a relational algebra

Possible to specify only a partial plan, where the optimizer completes the plan generation

Stored in a system catalog sysqueryplans

Persistent across: connections Server versions (i.e. upgrades)

79

Where will AP’s be used?Where will AP’s be used?

Application providers don’t want to include vendor specific syntax in their queries

In general, users don’t want to modify a production application to solve an upgrade optimizer problem

Still, it’s possible to include them if so desired Example:

select c1 from t1 where c2 = 0 plan ‘(I_scan () t1)’

Abstract query plans are captured and reused: set plan dump ‘new_plans_group’ on set plan load ‘new_plans_group’

80

Directory Structure: ASE 12.0Directory Structure: ASE 12.0

Top-level directory

Subdirectories for each component that include the following Executable program for the component Installation and configuration tools for the

component Display-related files needed by the component

Naming convention for subdirectories includes a component identifier and the software release version Examples:

ASE-12_0/ (location of ASE files) OCS-12_0/ (location of Open Client files) jConnect-4_2/ (location of jConnect 4.2 files)

81

Overview of ASE 12.0 Directory StructureOverview of ASE 12.0 Directory Structure

82

Important ASE-12_0 SubdirectoriesImportant ASE-12_0 Subdirectories

bin Executable files for most server utilities

init ASE installation log files

install Install programs, RUNSERVER files, the error log

scripts Optional database installation scripts

Note that some executables (such as isql and bcp) are located in the OSC-12_O/bin subdirectory

83

ASE 12.0 RequirementsASE 12.0 Requirements

Operating system levels for ASE 12.0: Solaris 2.6 (2.7 for 64-bit) HP/UX 11.0 (32 & 64 Bit) AIX 4.3.1 (32 & 64 Bit) Intel NT 4.0 with Service Pack 4 Digital Unix 4.0D

Minimum disk space Standard installation: 678 MB (Full installation: 700 MB) Custom installation: Depends on components chosen

New License Manager SySAM - Sybase Software Asset Management to install

Upgrade from Versions 11.0.x, 11.5, 11.5.1, 11.9.2

84

AgendaAgenda

Why Migrate ?

How Migrate ?

Planning & Pre-Upgrade

Migration Steps

Post-Upgrade

ASE 12

Summary

85

Key Lessons For A Successful MigrationKey Lessons For A Successful Migration

Plan for Success Plan the whole process Read the available documentation to prepare yourself/team Highlight and overcome potential problem areas by trailing

migration approach and scripts on development/test systems prior to production roll-out

Execute the Plan Follow the procedures and your migration plan Be prepared with the recovery fallback option for your

chosen strategy

86

SPS Migration InvolvementSPS Migration Involvement

Plan & Execute Migration Services Rapid Delivery of Migration Services Risk minimisation through SPS

experience of similar migrations Product Knowledge to exploit new

product features Professional approach using

documented Quality Standards & SAFE/EM™ methods

Assured delivery Single source of technology,

experience and methodology ISO 9001 TickIT of both technology

and staff© Copyright United Feature Syndicate 1997

87

ConclusionConclusion

Successful and painless migrations don’t just happen by accident, it requires you to: Use a proven and structured

migration approach Plan and execute the whole

migration task Follow the documentation - it

exists to help you

Please ask questions...

88

Sybase ContactsSybase Contacts

Sybase Technical Support, North America:1-800-8-SYBASE (to order product upgrade )

Sybase Technical Support, UK:(01628) 597111 (to order product upgrade)

Sybase Web Site www.sybase.com

Comments to:Ian Annis Principal Consultant

Sybase Professional Services

(01628) 597448 ian.annis@sybase.com

89

‘Why’ and ‘How’Migration Guide to ASE 11.9.2/12

Ian AnnisPrincipal ConsultantSybase Professional Services (UK) March 2000

UKSUG MasterclassUKSUG Masterclass