Going APEX 5: A Real Life Adventure

Dr. Eckhart Guthöhrlein, Dr. Alex Kohn pRED Informatics, Roche Diagnostics GmbH

Features: Most Wanted

APEX at Roche

Migration Strategy

Developer’s Cut

Lessons Learnt

Features: Most Wanted

APEX at Roche

Migration Strategy

Developer’s Cut

Lessons Learnt

APEX 5: MOST WANTED FEATURES

What‘s New in APEX 5?

• Page Designer

– Cohesive User Experience

– Better Visual Representation

– Drag and Drop

– Enhanced Code Editor

• Universal Theme

– Theme Roller

– Template Options

– Responsive Design

• New HTML5 charts

• Interactive Reports

– Multiple Reports on a Page

– Pivot

– Fixed Headers

• Modal Dialogs

• LDAP Integration for App Builder

Features: Most Wanted

APEX at Roche

Migration Strategy

Developer’s Cut

Lessons Learnt

APEX AT ROCHE

The Roche Group

• Founded 1896 in Basel, Switzerland

• Employing > 82000 people

• Clear focus on healthcare

• Roche Innovation Center Penzberg

• Pharma Research and Early Development (pRED) Informatics

• Provide informatics and bioinformatics services and tools to researchers

Roche Innovation Center Penzberg pRED Informatics

Located between Munich and the Alps in Southern Bavaria

APEX at Roche Pharma Research

• over 2000 employees at seven sites around the world

• >100 APEX applications

– Data visualization, Reporting, Process analysis and optimization

– Inventories, Request Handling, Project Management

– Masterdata and controlled vocabulary

– Platform administration for APEX and DWH (security, monitoring)

– many ad-hoc solutions for services and projects

What Does Our pREDi APEX Platform Offer

Beyond the Standard?

ASMA

• Central place for the

security management

of all APEX

applications

• PL/SQL package for

querying the Roche

LDAP

• Integrated into APEX

Application Builder Alex Kohn, APEX: Zentrale Rechteverwaltung aller Applikationen leicht

gemacht, DOAG 2014.

What Does Our pREDi APEX Platform Offer

Beyond the Standard?

ASMA

Automatic Sign-On using

Windows Credentials

Single Sign

On

What Does Our pREDi APEX Platform Offer

Beyond the Standard?

Application

Template

• Pre-configured

authentication &

authorization schemas

based on ASMA

• Roche branded Theme

ASMA

Single Sign

On

What Does Our pREDi APEX Platform Offer

Beyond the Standard?

Application

Template

ASMA

Single Sign

On

APEX

LOVs

Pre-configured LOVs such

as Roche Users, Cell Lines,

Projects

What Does Our pREDi APEX Platform Offer

Beyond the Standard?

Application

Template

ASMA

Single Sign

On

APEX

LOVs

APEX

Utils

Various utilities such as e-

mail, logging, native excel

export, etc.

What Does Our pREDi APEX Platform Offer

Beyond the Standard?

Application

Template

ASMA

Single Sign

On

APEX

LOVs

APEX

Utils

Report

Generator

Jasper Reports compiler

Current APEX Infrastructure

apexdev.roche.com

DEV

DEV1 DEV2

/db2//db1/

APEX4.2

APEX4.2

Example URL:

http://apexdev.roche.com/db1/f?p=doagdemo

Current APEX Infrastructure

apexdev.roche.com apex.roche.com

apextst.roche.com

DEV

DEV1 DEV2

/db2//db1/

APEX4.2

APEX4.2

TST1 TST2

/db2//db1/

TST

APEX4.2

APEX4.2

PRD1 PRD2

/db2//db1/

PRD

APEX4.2

APEX4.2

Features: Most Wanted

APEX at Roche

Migration Strategy

Developer’s Cut

Lessons Learnt

MIGRATION STRATEGY

Goals for the Migration

• Assess the necessary effort before transforming the entire environment

• Avoid peaks in workload for developers, testers, and users

• Regular users should not notice anything (ideally)

– no downtime

– no disruptive changes to the applications

• Ongoing application development must be possible

– project schedules not to be affected

– application deployment from dev to test and production still possible

Get the pREDi APEX Platform Ready for 5.0

• Started with fresh APEX 5 instances on new databases

• Set up APEX and ORDS for heterogeneous versions

• Upgraded APEX pREDi platform to work with APEX 5

– LDAP integration for workspaces

– PL/SQL packages upgraded to support changes in APEX 5

– ASMA: Upgraded to universal theme

– Roche Master and Roche Template application

• Based on Universal Theme

• Roche logo integrated on instance level for Universal Theme

• Contains pre-configured authentication and authorization

schemes

Migration Strategy, Variant 1: En Bloc

apexdev.roche.com apex.roche.com

apextst.roche.com

DEV

DEV1 DEV2

/db2//db1/

APEX5.0

APEX5.0

TST1 TST2

/db2//db1/

TST

APEX4.2

APEX4.2

PRD1 PRD2

/db2//db1/

PRD

APEX4.2

APEX4.2

Migration Strategy, Variant 1: En Bloc

apexdev.roche.com apex.roche.com

apextst.roche.com

DEV

DEV1 DEV2

/db2//db1/

APEX5.0

APEX5.0

TST1 TST2

/db2//db1/

TST

APEX5.0

APEX5.0

PRD1 PRD2

/db2//db1/

PRD

APEX4.2

APEX4.2

Migration Strategy, Variant 1: En Bloc

apexdev.roche.com apex.roche.com

apextst.roche.com

DEV

DEV1 DEV2

/db2//db1/

APEX5.0

APEX5.0

TST1 TST2

/db2//db1/

TST

APEX5.0

APEX5.0

PRD1 PRD2

/db2//db1/

PRD

APEX5.0

APEX5.0

Migration Strategy, Variant 2: Step-by-Step

apexdev.roche.com

DEV1 DEV2

/db2//db1/

APEX4.2

APEX4.2

Migration Strategy, Variant 2: Step-by-Step

apexdev.roche.com

DEV1 DEV2

/db2//db1/

APEX4.2

APEX4.2

APEX5.0

APEX5.0

Migration Strategy, Variant 2: Step-by-Step

apexdev.roche.com

DEV1 DEV2

/db2//db1/

APEX4.2

APEX4.2

DEV3 DEV4

APEX5.0

APEX5.0

/db3/ /db4/

Migration Strategy, Variant 2: Step-by-Step

apexdev.roche.com

DEV1 DEV2

/db2//db1/

APEX4.2

APEX4.2

DEV3 DEV4

APEX5.0

/db3/ /db4/

DEVX

APEXV.?

/dbX/

⋯APEX5.0

Cleaning Days

ID APPLICATION_NAME OWNER WORKSPACE retire Target DB Status

1000 Roche Master Application APEX_SECURITY APEX_SECURITY n all DONE

1001 Roche Template Application APEX_SECURITY APEX_SECURITY n all DONE

171 Security Management APEX_SECURITY APEX_SECURITY n all DONE

197 IDRM Warehouse Monitoring IDRM_MASTERDATA IDRM n db3 DONE

118 Master Data Management (MDM) IDRM_MASTERDATA IDRM n db3 IN PROGRESS

116 Media Management MEDIA_MANAGEMENT IDRM y --- RETIRED

144 PCV Controlled Vocabulary PCV IDRM n db3 NOT STARTED

146 People Pages PEOPLE_PAGES IDRM y --- RETIRED

224 Project Planning REQUEST_SYSTEM IDRM n db4 DONE

107 Genealogy Browser GENEALOGY LMN n db3 NOT STARTED

119 DEMOTaxonomy Management APEX_PREDI_PZ PREDI y --- RETIRED

106 TAXONOMY TAXONOMY n db4 NOT STARTED

... ... ... ... ... ... ...

• Inventory of existing applications

− responsible persons and key users

− obsolete applications

• Synchronize DEV/TST/PRD environments

− enforce best practice for deployment process

Features: Most Wanted

APEX at Roche

Migration Strategy

Developer’s Cut

Lessons Learnt

DEVELOPER’S CUT

Deployment to Test and Prod Environments

• Development on DEV only; (TST,) PRD must be runtime only

• No developer access to workspaces via the APEX application builder

• Deployment using SQL scripts

• Application export using

1. application builder, or

2. command line with Oracle’s Java export tool

${ORACLE_HOME}/apex/utilities/oracle/apex/APEXExport.class

$ java oracle.apex.APEXExport -db ${DB_INSTANCE_DEV} \

-user ${USERNAME} -password ${PASSPWD} \

-applicationid ${APPID} -expPubReports

–skipExportDate

Deployment Using SQL*Plus

-- connect as target parsing scheme or APEX_050000

set define on whenever sqlerror exit sql.sqlcode rollback

accept TARGET_WORKSPACE char prompt 'Target workspace: ' accept APP_FILE char prompt 'Application file to import: '

prompt set target workspace id and metadata offset declare l_workspace_id number; begin select workspace_id into l_workspace_id from apex_workspaces where workspace = '&TARGET_WORKSPACE'; apex_application_install.set_workspace_id(l_workspace_id); -- override workspace id apex_application_install.generate_offset; -- generate metadata offset to avoid collisions -- apex_application_install.set_application_id(702); -- override application id

-- apex_application_install.generate_application_id; -- override application id

-- apex_application_install.set_schema('NEWSCHEMA'); -- override parsing schema

-- apex_application_install.set_application_alias('NEWALIAS'); -- override application alias

end; /

prompt import application export file @&APP_FILE

prompt set application in RUN_ONLY BEGIN wwv_flow_api.set_build_status_run_only(wwv_flow.g_flow_id); END; / commit;

Semi-Automated Application Deployment

and Version Control

APEX

DEV

Scheduled export

(+ on demand)

APEX

TST

APEX

PRD

APEX Cockpit (ASMA)

One-click deployment • Scheduled APEX application export from the

development instance to a Subversion repository

• APEX Cockpit adds a deployment workflow to

ASMA

• Administrators can deploy applications to

TST/PRD at the click of a button

on demand

APEX Architecture with ORDS

Static

files

• Images

• HTML, CSS

• JavaScript

Web Listener

Web Container

ORDS

Browser

APEX4.2

APEX4.2

APEX?.?

•••

APEX5.0

APEX5.0

ORDS 3.0 Issues

• Initial plan was to upgrade both ORDS and APEX

• ORDS upgrade to first release of version 3 failed

– Bugs while executing configuration

– UCP Errors

– Failed to load images

• Stayed with ORDS 2, works fine with APEX 4.2 and 5.0

• Personal lesson learned:

Never upgrade to first version again but wait for first bug-fix release

• Should be fixed now with ORDS 3.0.2.x

ORDS 2 Configuration:

One Listener, Multiple Databases

ORDS/url-mapping.xml

<?xml version="1.0" encoding="UTF-8"?>

<pool-config xmlns="http://xmlns.oracle.com/apex/pool-config">

<pool base-path="/db1" name="apexdev1”/>

<pool base-path="/db2" name="apexdev2"/>

</pool-config>

ords +-- url-mapping.xml # URL path to DB mapping +-- defaults.xml # common settings for all db connections +-- conf +-- apexdev1.xml # specific for apexdev1 APEX_PUBLIC_USER +-- apexdev1_al.xml # apexdev1 APEX_LISTENER +-- apexdev1_rt.xml # apexdev1 APEX_REST_PUBLIC_USER +-- apexdev2.xml +-- apexdev2_al.xml +-- apexdev2_rl.xml +-- ...

APEX Static Files

• The application server requires a copy of the APEX images directory

Default: /i/

• Each APEX versions needs different static files (CSS, JavaScript)

• Therefore, the images directory of each APEX version

must be put in a separate folder location,

e.g. /i4.2.6.00.03/ and /i5.01.00.06/

• The image prefix must be set in the Oracle APEX installation by executing

the reset_image_prefix.sql script of the APEX installation.

. +-- webapps +-- i4.2.6.00.03 +-- i5.0.1.00.06 +-- ReportGenerator +-- ROOT +-- roche-apex

SQL> @<apex directory>\utilities\reset_image_prefix.sql Enter the Application Express image prefix [/i/] /i5.0.1.00.06/ . . . Image Prefix update complete

Transparent Migration by URL Rewriting

• Application moved: apex/db1/f?p=doagdemo to apex/db2/f?p=doagdemo

• Goal: Automatically redirect to new location

• without changing the application

• without notifying the users

Browser

Web Listener

Web Container

ORDS

UrlRewriteFilter

/db1/f?p=doagdemo /db2/f?p=doagdemo

red

irect

http://tuckey.org/urlrewrite

UrlRewriteFilter Configuration

• One redirect rule for each application

• Example:

– application moved from db1/f?p=doagdemo to db2/f?p=doagdemo

– default to /db1 if not specified

– UrlRewriteFilter installed at ROOT http://apexdev.pez.roche.com/

<urlrewrite use-query-string="true">

<rule>

<from>^(\/db[^2])?\/f\?p=(doagdemo|999)(:.*)?$</from>

<to type="redirect" last="true">

http://apexdev.pez.roche.com/db2/f?%{query-string}

</to>

</rule>

</urlrewrite>

Image generated using regexper.com under CC license 3.0 (creativecommons.org/licenses/by/3.0)

UrlRewriteFilter: Alternative Rule and Options

<urlrewrite use-query-string="true">

<rule>

<from >^(?!\/db2\/).*\/f\?p=(?:doagdemo|999)(:.*)?$</from>

<to type="redirect" last="true">

http://apexdev.pez.roche.com/db2/f?p=doagdemo$1

</to>

</rule>

</urlrewrite>

• log matches and affected users

• add <run> actions for direct notifications

• …

LDAP Authentication for Developers

• Killer feature: easy LDAP integration for developers

• We all know how good passwords looks like...

• But now, at last:

no more separate accounts for APEX development,

including administration in the INTERNAL workspace

APEX Login via LDAP

in a Secured Enterprise Environment

Enter credentialson login page

Verify username and workspaceAPEX backend

Get Distinguished Name (DN)APEX_AUTH PL/SQL package

LDAP lookup

AuthenticationAPEX Backend

LDAP auth

Application Builder

Application Server Oracle Database Active Directory

LDAP Configuration (1)

LDAP Configuration (2)

LDAP Configuration (3)

LDAP Configuration (4)

LDAP Integration: APEX_AUTH Package (1)

/**

* Get the distinguished name for the given LDAP user.

* Returns null if no unique match is found.

*

* @param p_cn user id

* @return distinguished name for p_cn

*/

-- Tip: PLDoc (http://pldoc.sourceforge.net) for API documentation function getDistinguishedName(p_cn in varchar2) return varchar2 is l_retval pls_integer; l_session DBMS_LDAP.session; l_attrs DBMS_LDAP.string_collection; l_message DBMS_LDAP.message; l_entry DBMS_LDAP.message; l_dn varchar2(256) := null; begin -- ...

LDAP Integration: APEX_AUTH Package (2)

-- ... -- Connect to the LDAP server. l_session := DBMS_LDAP.init(hostname => g_ldap_host, portnum => g_ldap_port);

l_retval := DBMS_LDAP.simple_bind_s(ld => l_session, dn => g_ldap_user, passwd => g_ldap_passwd);

-- search for users and persons l_attrs(1) := 'distinguishedName'; l_retval := DBMS_LDAP.search_s(ld => l_session, base => g_ldap_base, scope =>

DBMS_LDAP.SCOPE_SUBTREE, filter =>

'(&(objectClass=user)(objectCategory=person)(cn=' || p_cn || '))', attrs => l_attrs, attronly => 0, res => l_message); -- ...

LDAP Integration: APEX_AUTH Package (3)

-- If there is a unique result, get its distinguished name. if DBMS_LDAP.count_entries(ld => l_session, msg => l_message) = 1 then l_entry := DBMS_LDAP.first_entry(ld => l_session, msg => l_message); l_dn := dbms_ldap.get_dn(ld => l_session, ldapentry => l_entry); end if;

return l_dn;

end;

-- Further improvements:

-- * handle non-unique results

-- * additional filter criteria

-- * exception handling, logging

Upgrading to Universal Theme

• Not mandatory, legacy themes still work

• Thorough testing is a must after migration to Universal Theme

• Example ASMA: application was unusable after automatic update

• Bottom line: upgrade simple applications only (or invest time)

Excel Export Plugin for Interactive Reports

• APEX plugin providing native MS Excel export for Interactive Reports

– column selection and naming

– filtering and sorting

– rounding of numeric values

• Integrated into the IR toolbar

• Extensive use of Javascript and queries to internal APEX tables

• IRs have been rebuilt from the inside out in APEX 5

• How big is the effort for porting the plugin to APEX 5?

Excel Export from IR Menu (APEX 4.2)

1 written by Vikram, http://www.APEX-plugin.com/oracle-APEX-plugins/dynamic-action-plugin/ir-actions-menu-item_171.html

• “IR custom action menu“ plugin1

• extends menu using JavaScript

• Details can be found in: Eckhart Guthöhrlein and Bernhard Schirm,

Tweaking APEX: A New Approach for Dynamic Interactive Reports,

DOAG 2013

IR Custom Actions

window.location.href = 'f?p='

+ $v('pFlowId') + ':9:' + $v('pInstance') +

'::NO:2:PAGE_ID,BASE_REPORT_ID,ROUND:'

+ $v('pFlowStepId') + ','

+ $v('apexir_REPORT_ID') + ',0'

Excel Download Page (APEX 4.2)

set via page link to

$v('APEXir_REPORT_ID')

$v('pFlowStepId')

0 or 1

Excel Plugin: Overview

1. Get SQL for the current Interactive Report

– underlying query becomes inner SELECT statement

– the user‘s column selection become the outer SELECT statement,

preserving column ordering

– sorting is converted into an ORDER BY clause

– filters and searches are parsed into a WHERE clause

2. Set column labels using a hash table with the corresponding column

names.

3. Build Excel XML and return it as a BLOB

– Extensions to PL/SQL “Excel creator“ by Anton Scheffer1

1http://technology.amis.nl/2011/02/19/create-an-excel-file-with-plsql

Excel Plugin: APEX Tables

Table/View Contains

APEX_application_page_ir Interactive Reports

APEX_application_page_ir_rpt Customized reports

APEX_application_page_ir_col Report columns

APEX_application_page_ir_grpby Sorting, format masks

APEX_application_page_ir_cond Filters, searches, highlighting

IR to SQL: Example Queries (1)

-- queries of all IRs in application 999

select to_char(ir.sql_query) sql_query,

page_id,

ir.interactive_report_id

from apex_application_page_ir ir

where ir.application_id = 999;

-- column information for IR in a region on page 2

select r.region_id,

ir.interactive_report_id,

col.*

from apex_application_page_ir ir

join apex_application_page_ir_col col

on col.interactive_report_id = ir.interactive_report_id

join apex_application_page_regions r

on r.region_id = ir.region_id

where ir.application_id = 999

and ir.page_id = 2

and r.region_id = 36113592999419938;

IR to SQL: Example Queries (2)

-- filters on selected IR on page 1 select condition_column_name, condition_operator,

condition_expression,

condition_expression2,

condition_sql,

condition_expr_type

from apex_application_page_ir_cond cond join apex_application_page_ir_rpt r on r.application_id = cond.application_id and r.page_id = cond.page_id and r.report_id = cond.report_id where cond.application_id = 999 -- optional and cond.page_id = 1 -- optional and cond.condition_type = 'Filter' and cond.condition_enabled = 'Yes' and r.base_report_id = 36190965823294501;

IR Custom Actions: JavaScript for APEX 5

• IR classes and IDs have been completely overhauled (multiple IRs)

• Plugin configuration: JavaScript for custom action window.location.href = 'f?p=' + $v('pFlowId') + ':9:'

+ $v('pInstance') + '::NO:4:PAGE_ID, REGION_ID,ROUND:'

+ $v('pFlowStepId') + ',' + '#_region_id_#,0‚‚

• Actions menu: plugin code replaces placeholder by actual region id

• Complete rewrite of plugin code for creating the action menu entries

… + '::NO:4:PAGE_ID, REGION_ID, ROUND:‘

+ $v('pFlowStepId')

+ ',17869460278264420,0'">

// extracted from id attribute of

// action menu class

// $('div[class="a-Menu"]').attr('id')

• Multiple Interactive Reports per page => $v('apexir_REPORT_ID')

• Plugin and Menu: Parameter Region ID instead of Report ID

• Plugin: get Report ID for Region ID base_rept_id := apex_ir.get_last_viewed_report_id

(p_page_id => v('PAGE_ID'), p_region_id => v('REGION_ID'));

• PL/SQL backend: add report ID filter to where clauses

window.location.href = 'f?p=' + $v('pFlowId') + ':9:' + $v('pInstance') +

'::NO:2:PAGE_ID,REPORT_ID,ROUND:'

+ $v('pFlowStepId') + ','

+ $v('apexir_REPORT_ID') + ',0';

// APEX 4.2

'::NO:4:PAGE_ID, REGION_ID,ROUND:‘

+ $v('pFlowStepId') + ‘,’

+ '#_region_id_#,0'

// APEX 5.0

Excel Plugin: Changes for APEX 5.0

Features: Most Wanted

APEX at Roche

Migration Strategy

Developer’s Cut

Lessons Learnt

LESSONS LEARNT

Summary

• The migration project went smoothly and is a big success.

• With application-wise migration comes great flexibility

– parallel operation of APEX 4.2 and 5.0

– transparent for end-users, non-disruptive for developers

– no peaks in workload as for bulk migration

• Upgrading an application from APEX 4.2 to 5.0

– usually straightforward, most things work

– some refactoring is needed for non-standard customizations

– switch to Universal Theme needs manual editing and thorough testing

– overall effort is reasonable

Do it!

Doing now what patients need next