Page 1 of 63

Commonwealth Bank of Australia CommSee Solution Case Study

Brenton Webster

Solutions Architect

Microsoft Corporation

Contributing authors: Valentine Boiarkine, Technical Writer (Blade/Wadeware), Robert Brunet, Senior Consultant (Microsoft), Anna Liu, Architect (Microsoft)

With additional contributions from: Ed Gallimore (CBA), Stuart Johnson (CBA)

December 2006

Page 2 of 63

Contents

Summary ..........................................................................................................................................4 Introduction .......................................................................................................................................4

Who Is Commonwealth Bank? .....................................................................................................4 What Is CommSee? .....................................................................................................................4 Project Background ......................................................................................................................5

Environment Overview......................................................................................................................7 Integration Strategy ..........................................................................................................................8

User Interface / Front-End Integration ..........................................................................................8 Back-End Integration ....................................................................................................................8

Business Drivers ...............................................................................................................................9 Technical Drivers ........................................................................................................................... 10 Solution Requirements .................................................................................................................. 11

Business Requirements ............................................................................................................ 11 Functional Requirements .......................................................................................................... 11

Architectural Overview ................................................................................................................... 12 CommSee Client Overview ....................................................................................................... 13 Service Oriented Framework Architecture (SOFA) Layer ......................................................... 13 Private Services ........................................................................................................................ 13 IFW (Public) Services ................................................................................................................ 14 BizTalk Server ........................................................................................................................... 14 Data Stores ............................................................................................................................... 14

Solution Design .............................................................................................................................. 15 User Interface Tier ..................................................................................................................... 16

Smart Client .......................................................................................................................... 16 Benefits Achieved by Utilising the Composite Smart Client Application Paradigm .............. 17 WinParts Design Pattern ....................................................................................................... 18 Asynchronous Functionality .................................................................................................. 19 Rich Visualisation of Business Data ..................................................................................... 20 Benefits of Implementing Visualisation of Business Data ..................................................... 28

Service Tier ............................................................................................................................... 30 SOFA Layer .......................................................................................................................... 30 Private Services .................................................................................................................... 32 Public Services ..................................................................................................................... 33 Public Service Design Approach .......................................................................................... 33 Benefits of Service Oriented Architecture ............................................................................. 34 Benefits of Building Frameworks .......................................................................................... 35 BizTalk Orchestrations .......................................................................................................... 36

Caching, Performance, and Scalability ..................................................................................... 36 Security...................................................................................................................................... 37 Availability ................................................................................................................................. 38 Monitoring .................................................................................................................................. 39

Monitoring in the Data Centre ............................................................................................... 39 Monitoring on the Clients ...................................................................................................... 39 Server Response Instrumentation ........................................................................................ 42

Infrastructure Architecture, Facts, and Figures ............................................................................. 45 Facts and Figures ...................................................................................................................... 46

Development Approach ................................................................................................................. 47 Vertical Slice Development Methodology ............................................................................. 47 Benefits of Vertical Slice Development Methodology ........................................................... 47 Web Service Abstraction ....................................................................................................... 49 Development Tools ............................................................................................................... 49 Build Management ................................................................................................................ 50

Page 3 of 63

Test Approach ............................................................................................................................... 51 Unit Testing ........................................................................................................................... 52 Ensuring Continual Integration.............................................................................................. 52 The Dev-BA Testing Environment ........................................................................................ 53 Load Testing ......................................................................................................................... 53

Deployment and Operations Approach ......................................................................................... 54 Packaging ............................................................................................................................. 54 Client Deployment ................................................................................................................. 54 Server Deployment ............................................................................................................... 55 Versioning ............................................................................................................................. 55

Future Planning ............................................................................................................................. 56 Visual Studio 2005 / .NET Framework 2.0 ............................................................................ 56 Visual Studio Team System .................................................................................................. 56 SOFA Layer – Preparing for WCF ........................................................................................ 56 CommSee UI – Preparing for WPF....................................................................................... 58

Lessons Learned ........................................................................................................................... 60 Conclusion ..................................................................................................................................... 63 Special Thanks .............................................................................................................................. 63

Page 4 of 63

Summary This case study describes how the Commonwealth Bank of Australia designed, developed, and implemented its CommSee application – a relationship banking solution, custom-built by the Commonwealth Bank of Australia using Microsoft® .NET technologies. This solution was developed as a Microsoft® Windows® Forms–based Smart Client that consumes standards-based .NET Web services. The Web services are responsible for orchestrating data from a variety of back-end data sources including mainframes, databases, and various backend legacy systems. At the time of this writing, CommSee has been successfully deployed to 30,000 users at more than 1,700 sites across Australia. After considering the business needs that CommSee was designed to address, this case study will examine the solution architecture, technical project details, and best practices employed in this major software development effort.

Introduction

Who Is Commonwealth Bank? The Commonwealth Bank (CBA) is one of Australia‟s leading financial institutions, with businesses in New Zealand, Asia, and the United Kingdom. The company is one of Australia‟s leading providers of integrated financial services including retail, premium, business and institutional banking, funds management, superannuation, general and life insurance, brokerage services, and finance company activities. With over 130,000 distribution points across the country and businesses overseas, the Commonwealth Bank is a diverse organisation with a long history in the Australian banking industry. CBA maintains more than 1,000 branches and has over 40,000 employees globally, with more than 35,000 staff in Australia, making it one of Australia's largest employers.

What Is CommSee? Built on the .NET framework, CommSee consists of a Windows Forms Smart Client front end connecting to back-end standards-based ASP.NET Web services. Web services orchestrate and synchronise data and processes from many different back-end legacy and mainframe systems via Microsoft® BizTalk® Server and Microsoft Host Integration Server (HIS). This data and

functionality is then surfaced for use by the CommSee Smart Client and other applications across the bank. The Smart Client provides an intuitive, productive, high-performing, and graphically rich environment for the CBA‟s employees, enabling unprecedented visibility into a customer‟s interaction history with the bank. In addition, advanced data-visualisation capabilities enable faster, better decision making for employees. The Windows Forms Smart Client integrates existing Win32, VB6, HTML, and Java applications, running them side by side with newer .NET components and thereby enabling CBA to build on and leverage previous investments in technologies such as Microsoft® Visual Basic®, .NET, Oracle, IBM mainframe systems, and the

Java 2 Platform, Enterprise Edition (J2EE). This strategy has allowed CBA to achieve business benefits from a single integrated application today without having to rewrite all of its applications up front; instead, CBA has migrated and continues to migrate applications over time. CommSee introduces a new paradigm of customer-centric banking operations. Prior to implementing CommSee, departments of CBA would use many different applications – each with

Page 5 of 63

its own disparate data sources – in order to manage its customer relationships. Because branch tellers had to access multiple applications using different login credentials, it was impossible to access all of a customer‟s information in a single location, presenting significant administrative and management difficulties. CommSee was designed to overcome these difficulties by providing a single view of the customer‟s information within one application.

Project Background Like many large organisations, the Commonwealth Bank of Australia had long struggled with old, inefficient processes and disparate legacy systems that it used to manage customer accounts. In 2003, looking around the marketplace for financial services in Australia, CBA realised that it needed to make some significant changes in order to stay ahead of the competition. Overwhelmingly, the response that came back was that CBA needed to offer the very best in customer service. In order to achieve this goal, CBA launched an ambitious program called “Which New Bank?” that was aimed at transforming its internal processes and systems to empower its staff with the tools and technology needed to better serve their customers. This service transformation was based on three interdependent goals: excellent customer service through engaged people supported by simple processes. One of the key components of the overall Which New Bank initiative was the CommSee project. CommSee began as a customer relationship management (CRM) platform that was to be based off an existing internal application CBA had built to support its Premium Financial Services (PFS) division. This application, code-named “Republic,” was a CRM solution, custom-built in Visual Basic 6, which provided members of the PFS division a rich user interface (UI) and intelligent data-visualisation capabilities over consolidated customer information. The departments using the “Republic” application soon realised its potential, and adoption began to spread to other divisions across CBA until there were approximately 3,000 users leveraging the application. One of the key advantages of the application architecture was its extensibility, which allowed developers to quickly incorporate new functionality by easily plugging in new components. “Republic” was chosen to be the basis of the new CommSee application. However, in order to support the increased user load (from 3,000 to 30,000 users) and to meet the requirements for increased functionality because this application would now be used by the entire bank, some significant architectural changes were required. In addition, CBA was interested in taking advantage of newer capabilities available in the .NET Framework 1.1. From the beginning, CommSee was architected with the future direction of technology in mind. A strong layered architecture and the use of common frameworks on both the client and server side mean that the application is built in a solid and consistent manner, making it easier to change moving forward. When CommSee was architected in 2004, it was done with future technologies in mind (such as Windows Presentation Foundation, previously code-named “Avalon,” and Windows Communications Foundation, previously code-named “Indigo”), with the intent that CommSee would be able to continuously evolve to leverage and exploit those technologies (and the versions beyond) at the appropriate time. At the time of writing, the current version of CommSee is built using the following technologies and CBA is actively planning the migration to the next generation of the Microsoft technology stack:

Page 6 of 63

Windows Server 2003

ASP.NET Web Services 1.1

Windows Forms 1.1

.NET Framework 1.1

Web Services Enhancements (WSE) 2.0

BizTalk Server 2004

Host Integration Server 2004 Development of CommSee coincided with the company-wide rollout of Windows XP. Prior to this rollout, CBA relied primarily on the Microsoft® Windows NT® 3.51 and OS2 operating systems, as well as the OS2 subsystem that is part of NT 3.51; therefore existing applications were based on either OS2 or Microsoft® Win32® and written in languages such as C, C++, and Visual Basic. As part of the operating system upgrade, a large effort was made to migrate existing OS2 applications to Win32, a highly labour- and resource-intensive undertaking. After the legacy OS2 applications were ported to Win32, it was a commonsense decision to preserve this investment. Therefore, instead of rewriting these applications on the .NET Framework, it was decided to preserve these front-end assets and integrate them within the CommSee interface. Although CommSee began as a CRM application, it evolved into a solution that delivered CRM and much, much more. CommSee now provides a single common platform and infrastructure upon which CBA can deliver almost all of its business functionality and processes to its users. CommSee provides frameworks for UI design, service design, and integration into existing systems. These frameworks are now used as a mechanism for standardising software development and delivery throughout CBA. CBA decided to adopt a “build instead of buy” approach, investing in long-term, extensible, tailor-made solutions understood and maintained by its own staff. Microsoft .NET technology was adopted because it was seen as a natural progression from the previously implemented Visual Basic 6–based rich client application. This choice allowed CBA to utilise its employees‟ existing skill set aligned with Microsoft technologies, and offered flexibility in terms of vendor support and involvement. The development of CommSee began in earnest in July 2004 and the first version was placed into a limited production setting in September 2004. Since then CBA has released a new version of CommSee every 3 months – to date 9 consecutive quarterly releases – continually improving the solution, adding features and expanding the user base. After successful smaller-scale production deployments, the national rollout of CommSee began in April 2005 and was completed in November 2005, since which time all of the banks employees have been using CommSee. CommSee was delivered successfully on time and within budget, and has now very much proven itself in production. At the time of writing CommSee is used by approximately 30,000 users at more than 1,700 sites across Australia including over 1000 branches, all call centres and all business banking centres. The solution is now providing CBA with many benefits including:

Increased user satisfaction,

Increased user efficiency

Increased customer satisfaction

Reduced training costs

Reduced customer wait times

Shorter lines in branch locations These are only a few of the benefits provided by CommSee, a continuously evolving solution that grows in accordance with changing business processes and requirements.

Page 7 of 63

Environment Overview The Commonwealth Bank of Australia is a large organisation with a diverse infrastructure based on many different generations of technologies. The central infrastructure is hosted in Sydney, with separate infrastructure parts distributed across more than 1,700 sites. The primary back-end system is a mainframe data farm, which is accessed through HIS servers. Other back-end systems deployed by the company include an Oracle database server, a human workflow solution, a Documentum document repository, and a wide variety of other back-end data sources. The mainframe data farm stores all transaction-related data and performs key banking business processes such as nightly batch processing, account debiting and crediting, and account maintenance. The mainframe schema and functionality were defined many decades ago, and since the mainframe offered high accuracy and performance, it was not advisable to extend or change this information. However, over time CBA started offering new products and services and collected more information about its customers. To accommodate the new data this required it was necessary to implement a highly scalable database solution such as Microsoft® SQL Server™ or Oracle. Because CBA had already adopted a database standard, an Oracle database was used as the primary CommSee data store. Essentially, the Oracle database extends the existing mainframe data store through capturing additional information; for example, the CRM capabilities of CommSee are sourced from Oracle databases. As part of this project, emphasis was placed on ensuring data accuracy through avoiding duplication and maintaining consistency across the multiple data sources. To meet this requirement, the company implemented a background synchronisation service between Oracle databases and the mainframe. This allows enforcement of the “single source of truth” pattern – where a single attribute uniquely identifies customers and accounts across all data sources, thereby avoiding data duplication and stale data. Prior to implementing CommSee, CBA maintained numerous separate business applications, ranging from legacy OS2 and Win32 applications to Visual Basic 6 applications. Each application would serve only one business function, thus complicating decisions that spanned different business functions. The number of disparate applications made it impossible for CBA personnel to have a common view of customer information. CommSee was designed to solve this problem by consolidating all these applications into a central source. Microsoft .NET technology was chosen as the platform for CommSee development because of its ability to implement Web services and a composite .NET Windows Forms interface.

Page 8 of 63

Integration Strategy Integration was one of the primary goals of the CommSee solution. Although in a perfect world it would be ideal to completely rewrite all of the applications – both front-end and back-end – it is simply not practical from a time, investment, and risk-mitigation standpoint for any significant project such as CommSee. Integration and reuse of existing systems was critical for CBA to be able to manage risks as well as preserve and leverage its existing investments. Often when people talk about Service Oriented Architecture (SOA), a heavy emphasis is put on the reuse and integration of back-end assets via services, in particular by using Web services. Typically, little attention is paid to reusing front-end assets. CBA realised that in order to be successful, it needed to focus on the reuse and integration of its existing front-end and back-end assets.

User Interface / Front-End Integration The goal of retaining existing client-side assets such as Microsoft® ActiveX® controls, Win32, Visual Basic 6, and HTML applications was achieved by adopting a composite Smart Client architecture built using Windows Forms .NET. The composite Smart Client is capable of hosting existing applications – in some cases in-process, and in other cases using techniques such as Dynamic-Link Library (DLL) injection.

Back-End Integration The existing mainframe system is the key data source and provides critical business functionality to CBA. In order to integrate with the mainframe cluster, CBA chose HIS and initially deployed a separate HIS server at each branch site, making Commonwealth Bank of Australia one of the largest HIS users in the world. The adoption of a service-oriented approach has eliminated the need for branch HIS servers, which have been replaced with a centralised HIS server farm. Where previously the legacy applications would access the mainframe through a local HIS server running at the branch, CommSee uses Web services that connect to the central HIS server farm for their data access needs. Commonwealth Bank has implemented a modular Web service–based infrastructure built on Microsoft® .NET technologies to communicate with Oracle and back-end mainframe and other legacy systems. CommSee uses BizTalk Server as a pluggable component for long-running orchestrated processes and to consolidate calls to multiple back ends.

Page 9 of 63

Business Drivers CommSee provides bank staff with access to all information about the relationship between the customer and CBA. This represents a major departure from the original “Transactional Banking” concept, where only relevant transactions are visible to specified personnel and an overall view of the customer relationship is unavailable. The following business needs have driven the design and development of CommSee:

Improve customer retention. Today, people frequently change banks because of special promotions, dissatisfaction, and low switching costs.

Maximise return on each customer by offering across-the-board solutions.

Facilitate better, faster decisions through availability of complete, accurate data.

Reduce paperwork through the ability to digitally store and present imaged documents such as signatures and photos.

Use software systems to connect multiple branches and departments. This greatly reduces costs and offers customers convenient services such as on-the-spot loan approval and extra-fast cheque clearance.

Page 10 of 63

Technical Drivers From the start, the CommSee solution was conceived to be built as a Smart Client on the .NET Framework. Smart Client was chosen to fulfil the requirements for a rich user experience and a next-generation data visualisation. The vision of CommSee designers was to exploit the power of the client to deliver a rich, high-performance interface and offload processing from the Web service servers to the clients where appropriate. Smart Clients presented a number of key advantages over a Web-based approach. Leveraging the processing power of the client computers allowed CBA to:

Gain more control over bandwidth usage.

Increase performance.

Reduce bandwidth, server, and data centre costs.

Provide a far richer experience than is possible in the browser.

Effectively integrate local hardware (such as card readers).

Integrate with existing Win32, VB6, HTML and Java applications. To develop the rich, highly visual, and interactive UI that CommSee required would be difficult, if not impossible, to develop effectively within the browser, and in 2004 would have required a heavy investment in dynamic HTML (DHTML) and JavaScript. Today‟s various Asynchronous JavaScript and XML (AJAX) toolkits (which were not available in 2004) would make it much easier to minimise the amount of JavaScript and DHTML that would have to be written (and maintained), and would enable developers to build a richer, more bandwidth-friendly solution within the browser. However, AJAX would still not be able to provide the level of bandwidth management and visual capabilities that can be achieved with a Smart Client application that fully exploits the power of a local PC. A Smart Client application is also able to work effectively in low-bandwidth environments and in offline scenarios where no connectivity is available. For CommSee, CBA made a business decision and chose not to support offline capabilities at this time. Finally, one of the key points of difference that steered CBA toward choosing Smart Client over a Web application was the fact that a Smart Client would be able to integrate with existing client-side assets, such as existing Win32, Visual Basic 6, HTML, and Java applications. Using this approach, these legacy applications are able to function seamlessly within CommSee while being gradually replaced over time, thereby spreading the initial investment over several years. Originally, a Terminal Services solution was also considered, as it would address most of the project requirements. However, careful analysis showed that this was not a feasible solution for accommodating 30,000 branch users. As with the Web-based approach, the Terminal Services–based approach would not offer the advantage of integration with existing client-side assets and would require a large investment in new server hardware and networking infrastructure. The .NET Framework was seen as the way forward because it offered the capability to bring existing client applications together into a single source. Familiarity with Microsoft technologies offered greater developer productivity. A key technological requirement of CommSee was future readiness. Choosing .NET meant that the product could be designed to work well with Windows Presentation Foundation (WPF), Windows Communication Foundation (WCF), and other future Microsoft technologies.

Page 11 of 63

Solution Requirements

Business Requirements Extensibility was seen as a key consideration in addressing the business drivers of the application. The business requirements that led to the choice of CommSee were to provide a composite application shell that can incorporate all current and future applications; to provide a single point of access to all line-of-business applications; to provide an integrated, complete view of customer information; and to avoid stale and duplicate information.

Functional Requirements The solution was envisaged to be both long-term and future-proof – that is, the solution was to be designed, to the extent possible, to be in line with the future direction of Microsoft technologies, such as WFP and WCF. The solution was required to incorporate existing initiatives for defining a single vocabulary for all business data and processes and for adopting a single identifier for each customer. The solution requirements were formulated to address the critical “pain point” of separation between the front and back office, by consolidating different business functions to allow better communication between customer-facing staff and back-office staff. This is accomplished through inter-departmental referral. For example, in the past, when a customer who came into a branch to deposit a cheque also asks a question about insurance, a bank teller was unable to efficiently communicate this customer‟s needs to the insurance department. Using CommSee, however, the bank teller performing the cheque deposit transaction can add a note to the customer‟s needs analysis record that refers the customer to the insurance department. The insurance department is able to easily access the information gathered by the bank teller and follow up on the customer‟s inquiry. Such scenarios result in higher customer satisfaction and allow CBA to provide additional services to each customer.

Page 12 of 63

Architectural Overview The following diagram represents an architectural overview of the CommSee solution.

Figure 1. CommSee Solution Architectural Overview CommSee is a complex, multi-tiered application spanning many technologies and business processes. It was designed to address the challenges of business complexity and integration by using existing front-end and back-end assets. Essentially, the CommSee architecture can be broken down into four tiers: the Client Tier, two Web service tiers (the Private Service Tier and the IFW Service Tier [or Public Service Tier]), and the Data Tier. Each tier is designed to be modular and independent, and essential services such as security, configuration, and instrumentation are carried through the four tiers. CommSee‟s UI and Web service layers are built

Page 13 of 63

on their respective frameworks, ensuring that new UI components and functionality can be incorporated easily, without requiring a change to the overall solution. The use of frameworks allows CBA to impose a uniform standard through the UI and Web service components, and to maximise productivity by abstracting common logic.

CommSee Client Overview The CommSee client is a .NET Windows Forms Smart Client written in the C# language. The client is known as “the Harness” because it acts as a container for multiple UI components known as “WinParts.” The Harness provides common services such as state management and navigation to each of the WinParts. Every WinPart is a modular component that is designed to be independent of other WinParts. The WinParts are either .NET Windows Forms user controls, or a hosted unmanaged application (such as Win32, Visual Basic 6, HTML, or Java applications) that runs in a container control. (This is achieved through the DLL Injection technique discussed later in this case study.) The key advantage of using WinParts is that they allow for integration of multiple disparate applications into one common interface. Each .NET WinPart calls its respective private service for data to display. The existing Win32 applications get data directly from their respective data sources. For more information on the design of the CommSee Client, see the User Interface Tier section of the Solution Design chapter.

Service Oriented Framework Architecture (SOFA) Layer Service Oriented Framework Architecture (SOFA) is the framework used for building the private and public/IFW Web services implemented in CommSee. SOFA provides common services such as configuration, authentication, authorisation, and logging. All CommSee Web services are based on the SOFA framework, thereby taking advantage of its common design patterns and functionality. The SOFA framework is based on the Enterprise Development Reference Architecture (EDRA) developed by the Microsoft patterns & practices team. EDRA facilitates the development of Web services in distributed applications, providing a common set of capabilities to all Web services based on EDRA. At the time CommSee was architected and designed, EDRA was seen as a way to quickly build a framework that could be used for the consistent development of CommSee Web services. This choice was made knowing that WCF was under development and that when WCF was eventually released, it would be Microsoft‟s strategic and recommended technology choice for building service-oriented applications. CBA‟s decision to use a common, well-layered framework for all of its Web services means it is well positioned to port SOFA to WCF without making any changes to application services or business logic. CBA already has plans to make this transition. For more information on the SOFA layer, see the Service Tier section of the Solution Design chapter.

Private Services Private services – also known as the presentation services – are closely coupled to the UI. These Web services are called only by the UI, and are specifically tuned to accept data from and return

Page 14 of 63

data to the UI in the way expected by the UI. The solution is designed in such a way that one private service is coupled to many WinParts – where each WinPart calls its own [WebMethod] on a specific Web service (.asmx). Each private Web service groups related functionality for use by many WinParts. Each WinPart is independent which helps reduce overhead associated with maintaining shared components and also helps facilitate parallel development. Every private service is based on the SOFA framework to provide the interface with common functionality such as authentication and authorisation. For more information on private services, see the Service Tier section of the Solution Design chapter.

IFW (Public) Services Information Framework (IFW) services, also known as public services, expose standard banking processes via Web services based on the IFW XML Schema Definition (XSD)–compliant schema. All IFW services are built on the .NET Framework, and the service interface is created from elements of the IFW XSD schema. IFW is an integration model provided by IBM that describes objects and attributes involved in banking transactions. The public services accept an IFW-compliant XML request and return XML that adheres to the IFW schema. These Web services are designed to provide banking functionality to a range of consuming applications within CBA, including CommSee. The IFW services are built using the same SOFA framework used for private services and are designed for high throughput using optimisation techniques such as caching. The IFW services are the primary interface to the mainframe system and also connect to other back-end data sources, such as Oracle databases. For more information on public services, see the Service Tier section of the Solution Design chapter.

BizTalk Server BizTalk Server is used by IFW services to orchestrate long-running transactions, such as the opening of an account, which can span multiple days. Implementing BizTalk orchestrations allowed CBA to streamline such long-running business processes and make them more accurate. BizTalk exposes and consumes Web services in its capacity as the connectivity mechanism. A custom Work Item Management (WIM) solution is used as the human workflow component in BizTalk orchestrations. For more information on BizTalk Server implementation, see the BizTalk Orchestrations section of the Solution Design chapter.

Data Stores CBA uses a variety of data stores. The primary data stores are the Microsoft® Active Directory®

directory service, the mainframe system, Oracle databases, and the Documentum Imaging solution. CommSee is designed to integrate with and connect to all of the existing systems within CBA, allowing CBA to derive more value from its existing investments in these systems. For more information on data stores accessed by CommSee Web services, see the Service Tier section of the Solution Design chapter.

Page 15 of 63

Solution Design Selecting the correct architecture that meets these complex business requirements, while being highly extensible to accommodate future business needs, has been a challenge. Rising to this challenge, Commonwealth Bank and Microsoft collaborated closely to create the optimal solution design. Due to the scalability and extensibility of the CommSee design, CommSee functionality and application size grew threefold within a year of the initial rollout, and more business functionality is being implemented every day. The user base grew from an initial pilot of 600 in September 2004 to approximately 10,000 users a year later. At the time of this writing, there are approximately 30,000 users running CommSee throughout Australia. The following section explores in detail the architecture of the CommSee solution, the best practices used in the design process, and the requirements successfully addressed by each design decision.

Page 16 of 63

User Interface Tier CommSee introduces two new paradigms of UI design: a composite interface framework and interactive presentation of business information. The key functional goals for the interface were preserving investment in existing front-end systems and providing a single source of customer information to the user through rich visualisation of business data. The key operational goals of the client are configurable deployment, high performance, and bandwidth minimisation.

Smart Client CBA chose to leverage a Smart Client as its architectural approach for the CommSee interface because it allows the ability to provide a highly graphical and interactive user experience, allows for offline capabilities (although at this time, CBA doesn‟t make use of this capability), makes use of the full power of a Windows XP client while minimising bandwidth, and finally provides integration capabilities necessary to preserve existing front-end assets.

Figure 2: Composite Interface – the Harness In order to provide the user with a single source of customer information and consolidate the vast number of existing applications, the concept of the “Harness” was devised. Essentially, the Harness is a container for numerous independent applications, providing common services such as state management and navigation. The complete UI consists of the Harness (which provides navigation between workspaces), the workspaces (logical groups of WinParts), and WinParts (singular, composable, and independent pieces of functionality designed to be hosted within the Harness framework). The Harness is both a framework for developing new content and a graphical container for different sources of

Page 17 of 63

content. The Harness utilises the concept of a workspace or a dashboard, where multiple applications are grouped logically and reside on relevant screens. Web-like navigation between the workspaces has been adopted due to user familiarity and intuitive actions. The Harness is a .NET Windows Forms application written using C#, and contains multiple .NET user controls called as WinParts. The services delivered by the Harness framework are security, authorisation, user interface services, messaging, and access to devices specific to financial services, such as personal identification number (PIN) pads, smartcards, and signature pads. Apart from consuming these common services provided by the Harness, each WinPart hosted within the Harness is required to be modular and independent of one another. This modular framework allows rapid development of easily pluggable new content. The WinParts contained within each workspace are independent of each other. Client configuration files determine the workspaces and WinParts the individual user sees, as well as their layout. Most WinParts are .NET Windows Forms user controls written using C#. These WinParts are described as “compliant” because they are built on the CommSee framework, can utilise UI services provided by the Harness, and communicate by calling private Web services. Each compliant WinPart is developed to provide a single logical piece of business functionality. In addition, the Harness was designed to host existing Win32, Visual Basic 6, and HTML applications without the need to alter them. These applications hosted within the Harness are described as “non-compliant” WinParts. To display the screen output of a Win32 executable within a .NET Windows Form, a special Windows application programming interface (API) known as DLL Injection had to be adopted. DLL Injection takes advantage of a Windows debug API that allows you to “inject” native code from a .dll or .exe file into an already running process. Although DLL Injection is a Windows debug API and is not officially supported, it allows organisations to rapidly integrate their existing Win32 applications into a .NET interface and migrate them to .NET at a later stage. As opposed to non-compliant Win32 applications, existing Visual Basic 6 applications can be hosted in-process; DLL Injection was not necessary to integrate them into the Harness. The ability to host existing applications within the composite interface meant that the new solution could be rolled out and ready to use as the single source of all customer information from the start.

Benefits Achieved by Utilising the Composite Smart Client Application Paradigm

The CommSee client is an implementation of the composite Smart Client pattern. A composite Smart Client allows the client solution to be composed of a number of discrete WinParts that are integrated together within a host environment to form a fully coherent Smart Client solution. The composite pattern is a very important pattern that has been used in enterprise line-of-business Smart Client solutions by Dell (http://members.microsoft.com/CustomerEvidence/Search/EvidenceDetails.aspx?EvidenceID=3981&LanguageID=1) and Thomson Financial, and by many call centre applications built on the Microsoft Customer Care Framework (http://www.microsoft.com/serviceproviders/solutions/ccf.mspx), to name just a few. One of the key advantages of this approach is that the WinParts provide business logic and functionality at a higher level of abstraction, along with a more solution-oriented integration model than simple UI controls or autonomous plug-ins that do not interact with each other.

Page 18 of 63

In addition to the overall CommSee UI structure, the Harness provides key Smart Client capabilities, such as deployment, asynchronous communication, update, data caching, and so on. This means that the WinParts can leverage these capabilities instead of having to implement them themselves. This allowed the solution to be developed extremely quickly, as WinPart developers could focus on their piece of the business logic rather than the underlying plumbing. This approach also provided a number of advantages in terms of a separation of roles between the content developer and the solution designer. The content developer is focused on implementing the business logic required to provide specific business-focused functionality. The solution designer is interested in being able to define a solution to a business problem at a higher and more broadly focused level of abstraction. This approach also makes possible a level of flexibility and agility that allows even ad-hoc solutions to be quickly defined and deployed. These are sometimes referred to as low-road solutions – solutions that deliver value now by providing the majority of the required functionality quickly and efficiently – as opposed to high-road solutions that require much more analysis, design, and development effort for little incremental benefit. The composite Smart Client architecture fits extremely well into SOA. CommSee Web services define their granularity based on business function. This is a natural way for SOA to be developed and to evolve. The WinParts bind directly to each service, thereby ensuring simplicity and granularity. There is a natural correspondence between the service agents and the WinParts that make up the CommSee composite Smart Client solution. This architecture allows the developers of the business capabilities, as well as the Web services that expose them, to develop the UI and client-side logic that can take maximum advantage of those services. Because the WinParts that make up the CommSee UI are loosely coupled – that is, there is no hard dependency between different WinParts because the Harness provides a standard mechanism for them to interact with each other – these modules can be independently updated and developed. This effectively extends the benefits of loose coupling to the client. Of course, not all services will be integrated and coordinated on the client (for transactional, availability, and reliability reasons), but many can be, and those that can‟t are implemented behind a façade service that uses a server-side orchestration engine such as BizTalk to coordinate these services on behalf of the client. The success of the CommSee UI has proven the value of composite Smart Clients for the next generation of business applications.

WinParts Design Pattern

Each WinPart is designed as a modular, independent UI piece and is hosted within the Harness. The adoption of this design pattern has helped CBA achieve its goal of a flexible and extensible interface. All WinParts inherit from a common base class that provides them with basic services and information. Much of the underlying infrastructure – such as security, device access and asynchronous communication with services – is abstracted away from the WinPart, allowing the developer to focus solely on the business content provided by the WinPart. Because a decision was made to host all customer information within one central source, business functionality is always implemented as WinParts. The wide use of WinParts as a standard has allowed the development team to invest time in the creation of custom Microsoft® Visual Studio® designers, code generators, and tools to streamline development of WinParts.

Page 19 of 63

Workspace layouts are configured at design time by using custom designers and layout controls. Layouts can be changed dynamically through the use of code or a configuration file. A custom-built flow-layout control and designer governs the layout of WinParts dynamically. Because the requirements for screen real estate change dynamically, this control is able to respond dynamically through optimal space sharing made possible by a built-in recursive algorithm. As an example, when a WinPart is designed, the developer may specify “I need at least X pixels vertical height”. At run time a WinPart may have retrieved 20 rows of data from a service, or it may have retrieved 2 rows. The WinPart can request how much space it would ideally want from the workspace. The layout infrastructure takes into account the minimum requirements and ideal requirements for all of the WinParts on the screen and adjusts their size dynamically. WinParts are commonly hidden and displayed according to the user‟s role and credentials. The flow layout responds to optimise the real estate at run time. This approach provides great customisation and personalisation potential for the interface. For reasons of security and maintainability, users are not given the ability to change their own screens, and therefore screens are configured optimally for them by means of the built-in layout infrastructure.

Asynchronous Functionality

Because each WinPart must operate independently of the others, and because of the requirement for high performance, responsive interface asynchronous logic is widely used throughout the CommSee interface. Each WinPart loads data asynchronously, thus ensuring that the entire interface is responsive during processor-intensive or long-running operations. To achieve this, the WinParts make asynchronous calls to background threads, which then execute a synchronous request/response Web service call to a private Web service that retrieves the data and returns it to the client. Once the data has been returned to the client, a callback method on the main thread is called to signal to the UI that the data is now available. The following diagram demonstrates the asynchronous operations made by the CommSee UI (Note that object and method names not representative of actual names).

Harness

Workspace

WinPart 1

WinPart 2

WinPart 3

WinPart 4

Navigation Bar

State Machine

Workspace State:

* Loading

* Loaded

* Error

WinPart State:

* Loading

* Loaded

* Error

1. NavigationBar.StartLoadAllWinParts()

2. StateMachine.SignalState(Loading)

Private Service 4

Private Service 3

Private Service 2

Private Service 1

2.

3.4.

5.

Page 20 of 63

Figure 3: Asynchronous Operations

1. The State Machine is a class instantiated by the Harness to keep track of the state of the workspace and each WinPart.

2. The navigation bar is used to navigate between workspaces in a forward/backward

fashion. For example, the first workspace the user sees may be the User workspace, showing the user‟s default screens, WinParts presenting search functionality, and user-specific WinParts. The navigation bar can be used to navigate to a Customer Information workspace, which presents WinParts related to customer information. The navigation bar loads a different workspace by presenting the workspace user control and calling its StartLoadAllWinParts method asynchronously. The workspace signals to the State Machine that its state is Loading.

3. The workspace presents a collection of related WinParts. The workspace calls the

StartLoad method of each WinPart asynchronously, and the UI remains responsive. 4. Each WinPart signals to the State Machine that its state is Loading, and on a

background thread calls its respective private service. 5. When the call to the service is complete, the WinPart makes a callback to the main UI

thread to present the data from the Web service. 6. After the data is presented by the WinPart, it signals to the State Machine that its state is

now Loaded. 7. After all WinParts have signalled that the state is Loaded, the entire workspace state is

set to Loaded, thus enabling extra functionality that requires all workspaces to be Loaded and ready.

8. If one of the WinParts experiences an exception during its load, it does not affect the

operation of other WinParts; however, it signals to the State Machine that the state is Error. The State Machine then sets the workspace state to Error, which enables global exception handling.

Other operations performed by WinParts, such as Save operations, are also asynchronous. The asynchronous methodology allows the UI to be responsive during operations, and enables the WinParts to operate completely independently of each other. This methodology also ensures that if one of the private services experiences a delay (for example, during an increased load on the Web service server or database), this delay does not affect the performance of other WinParts.

Rich Visualisation of Business Data CommSee reinvents the concept of a line-of-business application interface. Most such applications adopt plain two-dimensional, text-intensive interface designs, heavy on standard controls such as text boxes, tabs, grids, and paper reports. This design approach is based on the design of paper forms and reports – the predecessors of computer software. CommSee introduces a breakthrough concept of information design – finding graphical, interactive ways to present business information by leveraging the rich visualisation capabilities of the Smart Client interface. Utilising the full power of the Microsoft Windows GDI+ API, the CommSee interface exposes previously ordinary data as interactive animations, achieving a graphically rich interface. For example, relationships between customers are depicted as a web of

Page 21 of 63

human figures, connected to each other with lines; the user has the ability to navigate this web through rotating, expanding, or collapsing each person and relationship. The following screenshots represents the CommSee Relationship Editor screen. This screen allows the user to visualise relationships between customers and drill down further into each customer‟s details. This screen is highly dynamic, using graphics to meaningfully represent data that would otherwise be difficult to analyse.

Figure 4: Relationship Editor Another example of CommSee‟s innovative graphical design is a three-dimensional (3-D) rotating pie chart showing six categories of customer information, with each populated category displayed in colour and categories that need analysis displayed in black and white. Designers of this screen have researched aspects of human psychology related to colour processing, where the brain is more intensely stimulated by colour than by gray or black-and-white imagery, and is more stimulated by imagery than text. The customer service representative intuitively wants to “colour in” the gray areas of the pie chart, and therefore feels more motivated to ask the customer questions about future needs. This approach has resulted in a great increase in the amount of customer information collected by bank tellers. This information is then used for more targeted marketing, thereby increasing product and service sales. The advantages of presenting the

Page 22 of 63

customer needs as a 3-D object (as opposed to, say, a table of information) also include using less screen real estate, providing visual clues to the user, and providing more intuitive usability. The following screenshot shows the customer details screen highlighting the completed needs analysis pie chart. Clicking on each “slice” of the pie chart allows the user to drill down into this customer‟s needs regarding a particular service, such as brokerage or insurance. It also allows the user to populate the “slices” with information gathered from the customer.

Figure 5: Customer Details Adopting this paradigm has provided many great benefits, some of which were not envisioned at the time of the design. These benefits include the following:

Equipped with powerful visualisation tools, users are now able to make faster, better decisions, thereby increasing customer satisfaction. Complex concepts are more easily understood by users if presented in a friendly, visual manner.

When the user navigates between workspaces, the new workspace screen slides into view from the right side of the screen to the left edge, covering the entire screen within approximately one second. Before the animation begins, each WinPart makes background calls to its respective services and then awaits the return of data from those

Page 23 of 63

services. For example, if it normally takes 1.5 seconds to get data back from a service and display it on the screen, by the time the animation finishes, the requests only have 0.5 second remaining before they are fully complete. This serves to further increase the user‟s perception of the application‟s already high performance, thus providing an even better user experience. In addition, the direction of the screen movement provides visual feedback as to how the user is navigating through the application.

Asynchronous loading and layout of data means that the UI is never locked up, with the user waiting for data to load. Avoiding the “white screen of death” (an unwanted symptom of a synchronous call on the UI thread) has tremendously increased acceptance among heavy application users.

Rich visualisation allows users to understand relationships between customers and connect with the customer‟s environment.

A “fun to use” application influences the morale of users, many of whom spend their entire day looking at line-of-business software. These users can experience the equivalent of “line hypnosis,” a syndrome in which factory workers lose awareness and concentration due to the repetition of monotonous tasks. A rich and varied UI breaks up monotonous tasks into highly visual and interactive actions, keeping the user alert and interested.

Presenting a new, highly visual, and distinctive interface has created excitement among project teams and CBA management, thereby winning management buy-in and support for the project.

The rich visualisation techniques were designed by CBA and developed solely by using .NET Windows Forms and GDI+ interfaces, without the use of third-party applications or components. Commonwealth Bank‟s focus on a new UI paradigm makes CommSee intuitive, appealing, easy to use, and highly productive. The results of deploying this new paradigm to 30,000 users have shown that the rich visualisation of business data is a key aspect of next-generation applications and can provide phenomenal benefits to a wide variety of applications. The following screenshots illustrate the information design pattern adopted by CBA.

Page 24 of 63

Organisation Hierarchy – this screen allows the user to search and intuitively navigate through the organisational hierarchy of CBA and drill down into the details of each person or department. The screen is used to select personnel for referrals, task assignments, and similar functions.

Figure 6: Organisation Hierarchy

Page 25 of 63

User Details – this screen allows a member of the Human Resources (HR) department to view their own or another person‟s details. Note that on the left, the hierarchy of the person‟s position in the organisation is displayed.

Figure 7: User Details

Page 26 of 63

Interactions History Workspace – this workspace allows the user to view and categorise their interactions with a customer. Note that each interaction type has a corresponding icon and a user note.

Figure 8: Interactions History Workspace

Page 27 of 63

Record Interaction Slide Space – this screen allows the user to view specific customer information related to the interactions between CBA and the customer. Note that on the right, a meeting notice is displayed showing the purpose of an in-person meeting with the customer, the personnel attending the meeting, and a duration timer.

Figure 9: Record Interaction Slide Space

Page 28 of 63

Process Detail Workspace – the following screen represents a certain process as a Gantt chart showing the duration and status of each process task.

Figure 10: Process Detail Workspace

Benefits of Implementing Visualisation of Business Data

Visualisation involves much more than simply making patterns easy to see. In the CommSee client, visualisation is highly interactive, permitting the user to perform multidimensional analysis across many variables at once. With a large application such as CommSee, one issue is simply dealing with the large volume of information. This volume of information makes it difficult for the customer service representative to quickly scan through the information and understand what is happening. Multivariate data, such as relationships between customers, is particularly challenging because it is difficult to see relationships across many variables simultaneously. Combining visuals to see multiple attributes at once, in addition to using interaction to „slice and dice‟ across all the dimensions simultaneously, provides a very powerful means to achieve multivariate correlation.

Page 29 of 63

Data visualisation makes the slice-and-dice process extremely intuitive – much more so than other techniques, including pull-down menus (which require pre-existing structure) and pivot tables. By combining multiple visuals for multiple variables, the customer service representative has a quick and easy way to interact across many dimensions at once. Leveraging the rich visualisation in the CommSee client has achieved greater user satisfaction and increased sales of CBA‟s products and services by providing visual tools to gather and analyse customer information, which can be used in targeted promotion and offering of services.

Page 30 of 63

Service Tier

All business functionality used by the CommSee interface, with the exception of the injected Win32 applications (which continue to use their existing back-end infrastructure), is implemented as a set of standard Web services. The Web service layer is divided into two distinct parts: private services that are tightly coupled to WinParts and provide specific information requested by the user interface, and public services that provide functionality exposed as XML messages to a wide variety of consumers, including CommSee private services and other consumers within CBA. In the future, CBA can choose to expose some of these services to business partners in business-to-business (B2B) scenarios.

SOFA Layer

The Service Oriented Framework Architecture (SOFA) framework is an extensible Web service framework that CBA developed by extending the EDRA offering from the Microsoft patterns & practices group. This framework provides Web service enhancements in the form of encapsulating common Web service functionality, such as authentication, authorisation, and caching. Every CommSee service is built on top of the SOFA framework and utilises these common services. SOFA provides a set of templates and attributes that the developers use to set up their development environment and implement specific services. SOFA services are attribute-based. When a developer creates a new Web service, it is marked with an attribute to indicate that this service will be using SOFA, and with further attributes specifying the details of authentication, authorisation, caching, and assembly binding information. At compile time, custom-built tools perform forward engineering, and implementation code is generated based on the attribute values. The following code snippet illustrates how, using assembly-level attributes such as SofaSoapHeader and SofaEmitWebService, a Web service is marked as a SOFA Web service and handler implementation will be generated at compile time based on the attribute values. Each method is marked with the SofaMethodAttribute attribute to indicate that method implementation should be generated at compile time. This attribute/build approach is how CommSee plans to migrate to a Windows Communication Foundation (WCF) version of SOFA. The forward engineering tools will be modified to emit WCF Web services, and the elements of WCF that exist in SOFA will be replaced with native WCF implementation.

using System;

using System.Xml;

using CommBank.Ifw.Core;

using CommBank.Ifw.Messages.CurrentVersion;

using CommBank.ServiceOrientedFramework.Tools.Emit.Core;

using System.Web.ServicesProtocols;

using CommBank.Ifw.Messaging.Soap;

// declare out of band header for passing end user identity info

[assembly: SofaSoapHeader("UserIdentity", typeof(UserIdentityHeader),

SoapHeaderDirection.In)]

// specify emitted web service assembly namespace and asmx name

[assembly: SofaEmitWebService("CommBank.Ifw.ArrangementEnquiries",

Page 31 of 63

"ArrangementEnquiries")]

namespace CommBank.Ifw.ArrangementEnquiries.Sofa

{

/// <summary>

/// Summary description for ServiceGateway.

/// </summary>

public class ServiceGateway

{

private ServiceGateway()

{

}

#region RetrieveTermDepositArrangement

/// <summary>

/// Static wrapper used by Sofa forward engineering

/// </summary>

/// <param name="request"></param>

/// <returns></returns>

public static void ValidateRetrieveTermDepositArrangement(XmlDocument

request)

{

new RetrieveTermDepositArrangement.Service().ValidateRequest(request);

}

/// <summary>

/// Static wrapper used by Sofa forward engineering to specify webservice

method name

/// </summary>

/// <param name="request"></param>

/// <returns></returns>

[SofaXmlMethodAttribute( "RetrieveTermDepositArrangement",

typeof(Message), typeof(Message), SchemaManager.DefaultNamespace,

"ValidateRetrieveTermDepositArrangement", SueApplication="CommBank.Ifw",

SueOperation="RetrieveTermDepositArrangement")]

public static XmlDocument RetrieveTermDepositArrangement(XmlDocument

request)

{

return new

RetrieveTermDepositArrangement.Service().ExecuteRequest(request);

}

#endregion RetrieveTermDepositArrangement

// etc...

}

}

Essentially, SOFA provides a chain of handlers through which the request passes prior to executing implementation code. The following diagram represents capabilities provided by SOFA.

Page 32 of 63

SOFA Web Service Execution

Class marked with

attribute to emit

SOFA xml at

compile time

Web method

signature

Config

Files

determine

bindings

Client

RequestAuthentication

handler

Audit / Log

handler

Cache

handler

Input

Business

Action

SUE

Authorization

Web ServiceMSMQ

Listener

Service

Oracle

Caching

Assembly

Business

Action

Assembly

Figure 11: SOFA Web Service Execution SOFA offers the following advantages:

Separating the service interface from internal service implementation allows for deployment scenarios optimised for scalability, reliability, security, performance, and availability.

Separating business logic from cross-cutting concerns such as logging, monitoring, or raising business events allows the developer to focus only on implementing the business logic, with infrastructure functionality being provided by the framework.

Separating business logic from the underlying transport so that multiple transports can be used to access a single service implementation.

Private Services Private services are stateless and are called by the UI accepting and returning typed DataSets that derive from a common CommSee dataset object. Ordinarily, returning typed DataSets would not be recommended as a best practice, because typed DataSets are not broadly interoperable. For CommSee, there is a distinct separation between private services, which are used exclusively for the UI and optimised to return data in the way the UI wants it, and public services, which are designed to be broadly interoperable, exposing XSD-compliant schemas. These goals of developer productivity and friendliness of the interface on the one hand, versus an XSD-compliant, broadly interoperable and versionable interface on the other, are often at odds with each other. Having this split between the private and public services allows each layer to be optimised to perform its task well, and requires no compromise in the service interface. Private services perform a variety of functions, including communicating with public services or working directly with data sources. Private services are based on the SOFA framework, which provides

Page 33 of 63

them with standard functionality such as authentication and authorisation. Each private service is marked with appropriate SOFA attributes that generate relevant code at compile time.

Public Services

Each public service is designed to be stateless and modular and is built to provide relevant information for each specific business functionality type. Currently, the complete set of services is deployed to a single server. (Multiple physical servers host identical sets of services for scale and redundancy). However, the services can run in any combination across multiple servers. A public service can be atomic, composite, or a long-running orchestration:

An atomic service is a service that performs a single business action.

A composite service represents a business process and combines calls to multiple atomic services, but it is still a synchronous call.

A long-running orchestration represents a complex business process; for example, a process that needs to run over several days. This type of business process is triggered by calling an atomic or composite service, which in turn spawns a BizTalk orchestration, involving components such as human workflow, the results of nightly batch processing, and other IFW services.

The Web services can talk to data sources such as the Oracle database, the mainframe via Host Integration Server (HIS), and possibly some of the other 30+ back-end systems. Currently, CommSee includes approximately 160 public services. All public services are based on business objects derived from the IFW schema, which describes the objects, functions, and attributes involved in standard banking processes. The public services accept an XML request compliant with the IFW schema, and return business data to the caller in the form of XSD-compliant XML messages based on the IFW schema.

Public Service Design Approach

Most public services represent either a business process already implemented by a legacy application, such as the mainframe system, or a new business process defined by a business analyst. The service selection process is as follows:

1. Identify the business process.

2. Identify specific actions performed during the lifetime of the business process.

3. Map granular components of the business process down to a single business action.

4. Each business action becomes an atomic service, with one method to perform the business action.

5. The entire business process becomes a composite service, calling each business action

in the correct order.

6. If the business process involves timing (for example, if the process spans several days), human interaction, or complex decision-making, the service calls a BizTalk orchestration that implements the complex process.

Page 34 of 63

The key challenge in selecting business functionality to be implemented as unified public services has been that each disparate application used its own vocabulary for describing the business processes. The adoption of IFW as the standard unified schema has allowed CBA to consolidate its business processes into a single meaningful model. The large IFW object model can be opened and customised using Rational Rose modelling technology. An XSD schema depicting a business process that is a part of the full model can be generated by using custom tools. The XSD schema is then used as the basis for the interface of the public service. All of the public services are built on the .NET Framework. Developing stable service interfaces ensures the resiliency of deployed services.

Benefits of Service Oriented Architecture

CommSee implements Service Oriented Architecture (SOA) throughout the solution design – including user services (such as authentication and authorisation), private services providing data to the user interface, and public services providing business data and business processes to a variety of consumers. Service orientation is a key architectural trend for building enterprise systems. The architects of the CommSee solution have started implementing service orientation by classifying all functionality – from the mainframe application to the authorisation logic – as a logical service provider. The functionality of each service provider is exposed through interfaces, and the service consumers are bound to the service interface, not implementation. This allows the flexibility of re-binding the service consumer to another implementation of the service. After the services have been identified and refined, the service calls are mapped and orchestrated into business processes. One key factor in successfully incorporating SOA into CommSee has been the insistence on separating the service interface and the service implementation. Each service consumer references the interface only, and is bound at run time to a certain service implementation, which may change over time without affecting the service consumers. This approach also opens the capability of different service implementations exposing the same interface, thereby allowing the client to be dynamically rebound to a different implementation, and offering the developers a choice of services without the need to significantly change the consumer code. For the architects and developers of CommSee, adopting SOA has offered the advantage of creating a highly extensible and dynamic application, while minimising project dependencies and development time. Supporting the ability of the consumer to select the appropriate service provider has allowed CommSee to be sensitive to the content and context of a specific business process, with the ability to incorporate new functionality over time without the need to restructure the application. Operationally, building CommSee using the service-orientated approach meant that diverse systems across CBA could be effectively integrated into the solution over time, while ensuring flexible, configuration-based deployment. The aggregation of multiple systems‟ information and business logic into coherent logical services has allowed diverse and redundant systems to be addressed through a common framework. Adopting service orientation and building extensible, reusable frameworks has also allowed CBA to protect its existing IT investments without inhibiting the deployment of new capabilities and continuously improving the implementation of the services. Essentially, adopting SOA has ensured that technology investments are protected from difficulties associated with changing business requirements.

Page 35 of 63

From a business perspective, investing in developing frameworks and implementing service orientation allows CBA to bring IT investments more in line with its business strategy. Mapping diverse business information and processes into a single logical model allows CBA‟s business analysts to understand the cost tradeoffs associated with investments in people, systems, and sourcing. For many years, Microsoft has evangelised service orientation as a key building block for creating connected systems that leverage networks to link the humans and systems that drive business processes. Making SOA the core of enterprise development allows an enterprise to more effectively meet business challenges through integration of services and devices. With its emphasis on a common model for invoking capabilities, service orientation is at the core of an effective connected-systems strategy. To achieve transparency and coherency, CBA has created connections. CBA connected systems to consolidate management of customer information. For example, CBA connected human and technical capabilities to create consistent business processes, and it connected workers to create collaborative teams. The next logical step is to connect organisations to create effective value chains. In order for SOA to be effective, two key aspects of redefining business processes had to be put in place:

Standardised business vocabulary

Before the entire functionality of banking processes could be implemented and exposed as reusable services, CBA had to agree on a standard vocabulary that can be used to define these business processes. For example, while some applications would use “Customer Address” as a five-field structure, others would use it as one free-form field. This led to confusion and data-consolidation difficulties. The IBM IFW model has provided a means to redefine all business processes as subsets of a complete, standard schema. Every public service accepts only an IFW schema–compliant request and returns an XML message based on the IFW schema. This ensures that all consumers of IFW services use standard data structures and standard names for business actions, thus allowing true integration and reusability.

Service design approach

The service design approach outlined above has also played a key role in the success of SOA implementation. Prior to being able to implement each business process as a set of granular Web services, a process had to be implemented for service identification and approval. This process is followed every time a new service is proposed. Following a formal process ensures that all provided services are consistent in terms of functionality and granularity.

Benefits of Building Frameworks

The use of frameworks has contributed greatly to the success of the CommSee project. In an enterprise environment, such as a banking environment, line-of-business software shares many common components; for example, branding, layout, authorisation, and data access. In order to implement the complex business functionality required by CBA, the common services had to be abstracted away from the content, allowing the developer to focus on implementing business

Page 36 of 63

functionality. Creating frameworks of base classes and development patterns has allowed CBA to achieve these goals. The two key frameworks in CommSee are:

Harness UI framework. For more information on the Harness UI Framework, see the User Interface Tier section earlier in this chapter.

SOFA framework. For more information on the SOFA Framework, see the SOFA

Layer section earlier in this chapter.

BizTalk Orchestrations Most public services are atomic transactions, each of which represents a single business action. BizTalk is used to orchestrate and monitor long-running transactions. A public service that initiates a long-running transaction calls a Web service published by BizTalk to start the orchestration. All Web services exposed by BizTalk are private, secure, and do not require additional authentication and authorisation. BizTalk Business Activity Monitoring (BAM) is used to track the instances of BizTalk orchestrations. The powerful capabilities of BAM allow users to track the progress of each long-running transaction and view meaningful business information involved in the running orchestration. When human interaction is required, BizTalk orchestration uses CBA‟s custom Work Item Management solution as its human workflow engine. Currently, BizTalk is used across various business areas, for example Demand Deposit, Home Loan and Personal Loan Origination. These business processes could run across multiple days. It is used for all financial transactions to guarantee one-time delivery, and is used in situations where hundreds of WebService calls are required to service a business process that would typically time out if called in a synchronous fashion. BizTalk implementation has brought an increase in efficiency and accuracy to these complicated processes.

Caching, Performance, and Scalability CommSee has been developed as a set of independent vertical segments with very little data shared between the respective segments of the UI, private services, and public services. This framework offers the great advantage of maintainability, scalability, and a rapid parallel-development approach. However, it also means that the disparate segments of the UI can access the same back-end data multiple times, which can lead to performance degradation. When building enterprise-scale distributed applications, architects and developers face many challenges. Caching can help to overcome some of these challenges, including the following:

Performance. Caching improves application performance by storing relevant data as close as possible to the data consumer. This avoids repetitive data creation, processing, and transportation.

Scalability. Storing information in a cache helps save resources and increases scalability as demands on the application increase.

Page 37 of 63

Availability. By storing data in a local cache, the application may be able to survive system failures such as network latency, Web service problems, and hardware failures.

To take advantage of these capabilities, caching was implemented as part of the SOFA framework for public services. Essentially, each public service is marked with attributes stating what data it reads and changes. The SOFA framework caches the data accessed by the services on the Web server, and the next time the same data is requested, it is retrieved from the cache. When a Web service alters this data, the cache is automatically invalidated, and the next time the service is called, the cache will be refreshed. Implementing this caching mechanism has tremendously increased the performance of the solution. To take advantage of caching, it makes sense that a given client makes all of its requests from the same Web service server during a session. Alteon load balancers are set to enforce the affinity of a client connection to a single server. Note that all services are designed to be stateless, so server affinity is not a requirement but is a performance optimization in relation to caching. The data chosen to be cached had to be carefully considered. Caching infrequently changing data such as customer names, account lists, and contact details has proven to be the best approach. Conversely, fluid, frequently changing data such as bank balances is not cached. The cache expiry can be configured for each service, and is typically set anywhere from 2 seconds (to act as more of as de-duplicator) up to 60 seconds. The cache interval is kept low to avoid out-of-date data. Similarly, reference data is cached locally on the client while transactional data is not cached. Client-side caching is implemented though the ServiceAgent class.

Security Maintaining tight security while ensuring a user-friendly single sign-on has been a top priority for CommSee. Because CBA had already invested in a Microsoft Active Directory infrastructure, it was a natural progression to use Active Directory for authentication purposes. CBA maintains a single Active Directory forest with multiple domains. The Windows security infrastructure is leveraged to provide single sign-on capabilities. Windows Integrated Authentication is enabled in Internet Information Services (IIS) on all Web service servers. CBA‟s authorisation strategy is based on the Authorization Manager (AzMan) feature of Microsoft® Windows Server™ 2003. Parts of the user interface and the SOFA framework that require authorisation make Lightweight Directory Access Protocol (LDAP) calls to AzMan authorisation data stored in the Active Directory. Although all Web service requests are authenticated, only sensitive requests are authorised, for performance reasons. All financial transactions are treated as sensitive requests and must be authorised; in addition, various other transactions are marked as sensitive if they are only allowed to be accessed by users in certain roles. The CommSee client application is considered trusted and, given most services, can be accessed by all roles; most services do not require additional authorisation beyond the fact that they are a valid authenticated user on the system. Authentication occurs once, when the user logs onto Windows. When CommSee starts up, it obtains a secure token from the CommSee authentication service based on the user‟s Windows token. Authentication is checked on each service call by cryptographically verifying that the supplied token was issued by the CommSee authentication service.

Page 38 of 63

Note that CommSee is treated as a “trusted channel” and that authorisation is handled differently if a public service is being called by something outside the CommSee system. Although most public services do not require additional authorisation when being called from within CommSee (for example, from a CommSee private service), if applications other than CommSee are calling the public services, each transaction must be authorised. The Single User Entity (SUE) service facilitates the calls between the SOFA services and authorisation data stored in Active Directory. When a user connected to the CBA network logs on, a Windows Domain Controller authenticates the user through a normal logon procedure. However, with remote mobile users such as mobile mortgage managers, a domain logon is impossible because the user is only connected to the Internet and not the internal network. In this situation, the user can re-enter their logon details into CommSee and pass the encrypted user name and password to the SUE service. The SUE service then validates these details against the Active Directory and authenticates the CommSee client. All traffic to private and public Web services is encrypted by using an industry-standard Secure Sockets Layer (SSL) algorithm. The following diagram summarises the security infrastructure implemented by CommSee.

Figure 12: CommSee Security Infrastructure

Availability CommSee is a solution that provides critical banking services to 30,000 users around Australia. Naturally, high availability measures had to be undertaken to ensure that the solution provides the required level of service.

Page 39 of 63

High availability in CommSee is achieved through redundancy; that is, Web services are hosted on many identically configured Web servers. Alteon load balancers are used to direct client traffic to the Web service servers, ensuring that the Web services are always available. Asynchronous operations behind each WinPart ensure that the interface is responsive even when a part is processing a delayed response. Redundancy is also implemented as a HIS server farm, a BizTalk server farm, and clustering on the Oracle server. There are no single points of failure in the system.

Monitoring When deploying an application to 30,000 users as in the case of CommSee, monitoring of the application for exceptions, performance leaks, and general trends is critical. Monitoring is important not only on the servers in the data centre, but also on the 30,000 distributed clients.

Monitoring in the Data Centre Like any large, mission critical application, CommSee requires a dedicated team for monitoring and operations. The operations team is responsible for monitoring servers in several environments, including testing, training, staging and production. The team constantly monitors servers for trends and anomalies, allowing them to react quickly if any alerts are raised. Performance counters are used extensively to monitor the health of the servers as well as the CommSee Web services. For example, thresholds such as CPU, memory and disk usage are monitored to ensure they are within acceptable parameters. All IFW and private services are built with a common framework (SOFA) which means they all expose an extensive and standard array of performance counters that are used in monitoring the system. BMC Patrol is installed on all servers and used as the primary tool for monitoring.

Monitoring on the Clients Because each client instance of the application runs on a separate computer, and sometimes on completely different hardware, gathering diagnostic information can be a great challenge. Even after the information has been gathered, it can be difficult to consolidate and analyse trends. In order to effectively monitor the distributed clients that are part of CommSee, the operations team created a “Flight Recorder” component that is embedded into the client application and used to gather important information about how the client is running. The data gathered by the Flight Recorder component is asynchronously uploaded to a Web service that records the Flight Recorder information in a database. The CommSee monitoring team uses Crystal Reports to analyse the information and determine important trends. In order to prevent generation of an overwhelming amount of log information, by default clients will only upload the log information to the server when they encounter an error, such as an unexpected shutdown, or if that particular user has been flagged by the operations team to have their log information uploaded an analysed. At any given time, a representative set of users (approximately 1 percent of the total user population) are set to upload all their log information for general analysis. This enables the operations team to understand how users are leveraging CommSee, and to understand how the application is performing from the client‟s perspective.

Page 40 of 63

The following table details the monitoring challenges faced by a large enterprise application such as CommSee, and describes how the Flight Recorder component addressed these challenges.

Challenge How the Flight Recorder subsystem addressed this challenge

Difficulties in gathering information from 30,000 desktops

Each instance of CommSee creates a log file that records diagnostic information. The Flight Recorder component uses trace listeners to obtain and record the output of System.Diagnostics objects and custom trace outputs.

Difficulties in identifying problem areas when gathering information during normal, error-free operation

When the CommSee client shuts down normally, it deletes the flight log file without uploading it to the Flight Recorder Web service. However, if the CommSee client has experienced an abnormal shutdown (such as a crash or a silent shutdown), the next time the CommSee client starts up, it will upload the contents of the flight log file to the Flight Recorder Web service. This ensures that only important information is gathered.

Difficulties in consolidating a massive amount of information

The Flight Recorder Web service writes the contents of each uploaded log file to a database, structuring the information for easier analysis.

Difficulties in making decisions based on a massive amount of information

Special summarised reports are created to identify problem areas, such as the most common exceptions, total number of silent shutdowns, and so on.

Difficult to identify performance trends over time

Having a database makes trend analysis easier because data can be summarised and trend graphs can be created.

Page 41 of 63

The flight recorder works in the following way:

1. The CommSee client application generates a log file during normal operation. This file includes diagnostic traces and error messages if they were encountered. These messages are built into the Harness framework, thus ensuring that the content developer does not need to write a special logging infrastructure.

2. When the CommSee client shuts down normally, the log is destroyed. However, if the

CommSee client shuts down unexpectedly (silent shutdown or crash), the log file remains on the client.

3. When CommSee starts up, it checks for the existence of the log. If the log exists, it is

uploaded to a data upload service.

4. The service stores the information in the database.

5. The database is analysed by using Crystal Reports or a similar tool. The following diagram summarises the architecture of the Flight Recorder component.

Figure 13: Flight Recorder Architecture

Page 42 of 63

Server Response Instrumentation One of the key parts of the flight recorder monitoring system is the Server Response Instrumentation, which sends raw performance instrumentation data from the SOFA servers back to the client machines by adding to the server‟s existing HTTP response to transport the data back to the client. This allows detailed performance data from the server request to be sent back to the client machine and processed, thereby offloading computation from the server and making more effective use of the powerful PCs that exist on the user‟s desktop. The client machine processes the raw performance data from the server transparently, along with the existing performance data already gathered on the client machine. The gathered performance data from the client and server machines is persisted in files or databases for use by operations personnel and performance reporting. Note that even though not all clients are set to upload their client logs (only about 1 percent are set for this), all clients are set to upload the server response instrumentation data they collect and process. The architecture of Server Response Instrumentation was designed to utilise the existing HTTP communication channel between the client and the server machines. The HTTP request and HTTP response objects are used to transport the raw data between the client and server machines for any single request. Server Response Instrumentation functions in the following way:

1. Raw performance data is sent back to the client machines by adding to the server‟s HTTP response using Simple Object Access Protocol (SOAP) Handlers and SOAP Headers. The SOAP Handlers intercept the HTTP call and add the extra data to the SOAP Headers. The raw performance data is compressed along with the entire HTTP message.

2. The existing server Instrumentation Framework is utilised without the need for any

changes or extra tracing. A trace listener that gathers the raw performance data from the instrumentation framework and places it directly into the HTTP context is utilised. The SOAP Handler then executes the business action which makes the usual calls to the Instrumentation Framework to record performance data.

3. When the request is completed, the same SOAP Handler adds some final raw

performance data into a specific key in the HTTP context object.

4. The CommSee DataComms DLLs get the raw performance data from the HTTP context key and place it in the SOAP Response Header to be sent back to the client.

5. The client side DataComms DLLs get the data from the HTTP Response Header and

places it into thread local storage on the client machine.

6. The CommSee client application checks thread local storage for any raw performance data and processes the raw instrumentation data through its existing channels.

The following diagram summarises the operation of the Server Response Instrumentation component.

Page 43 of 63

Figure 14: Server Response Instrumentation

Page 44 of 63

Processing the instrumentation data on the servers directly would require many extra threads as well as CPU time on the servers, which would degrade their overall performance. With the current setup for the SOFA servers, each virtual directory has its own copy of the Instrumentation Framework, and at least one background thread would be required for each virtual directory (60+ threads). Sending the instrumentation data back to the CommSee client machine for processing helps reduce the load on the servers. Because the processing of the instrumentation data is typically CPU-intensive, the processing is done transparently on a background thread so it has no noticeable impact on the performance of the client application. Another advantage is that the performance information specific to a server request is available immediately on the client machine without the need for any database queries and so on.

Page 45 of 63

Infrastructure Architecture, Facts, and Figures The following diagram summarises the infrastructure involved in the deployment of CommSee.

Data CentreBranch Site

Smart Client

Local AD

SOFA farm

AD Farm

HIS farm

BizTalk Farm

Mainframe

Smart card

Other Application Servers, Oracle

All services

authenticated,

some authorized

Figure 15: Deployment Overview In CommSee enterprise infrastructure deployment, each branch site hosts a Domain Controller, which is able to authenticate the user locally. The user is authenticated by using their Windows credentials and a smartcard. The central data centre hosts the SOFA farm of Web servers, the HIS farm, the BizTalk farm, application servers, Oracle database servers, and mainframes. When a branch site CommSee client makes a request to a public Web service, the request is authenticated in the central Active Directory. Some Web service requests are authorised, to ensure that the user has the appropriate permissions to access the Web service‟s functionality. Depending on the type of Web service, it may make calls to the mainframe (through the HIS farm), BizTalk servers, the Oracle database, or other application servers.

Page 46 of 63

Facts and Figures Smart Client Approximately 30,000 clients are deployed across the enterprise at more than 1,700 sites, including every branch, all call centres, business banking centres, and so on. SOFA Farm At the time of this writing, the SOFA farm has 20 dual CPU Intel servers, each with 12 gigabytes (GB) of RAM. The farm generally runs at a sustained load of 800 transactions per second (tps) for six peak hours a day. During the peak periods, the SOFA farm generally runs at about 40 percent capacity. Each physical server has the complete set of public and private services deployed to it. Currently, this amounts to approximately 2,500 private services and 160 public services on each server. In addition, each server runs at least two versions of CommSee, the current version and one version prior, raising the total deployed services per server to approximately 5,320. HIS The HIS farm currently consists of 6 standard dual CPU Intel servers. Biztalk The BizTalk Server supports the high load requirements demanded by CBA. Currently, these requirements are two standard dual CPU, 2-GB RAM BizTalk servers configured as a farm, and one underlying SQL server supporting long-running orchestrations for opening approximately 5,000 accounts / 10,000 transactions per day. Scalability testing has shown that the current setup has the capacity to support more than 150,000 transactions per day. Oracle The Oracle database farm runs on the Sun E25 platform. The farm is configured as an 8-node cluster, with nodes 1-4 having 24 processors and 90GB RAM each, and nodes 5 and 6 having 8 processors and 30 GB RAM each.

Page 47 of 63

Development Approach CommSee development can be divided into two logical areas: framework development and content development. Frameworks include the Harness and SOFA frameworks, as well as the underlying infrastructure. Content development involves the creation of CommSee WinParts and associated business functionality. At the beginning of the project, the frameworks were developed first to provide a base set of services and design patterns. New content was then developed (and continues to be developed) by more than 200 members of the content development team, while the frameworks are continually refined.

Vertical Slice Development Methodology

The key to the success and rapid growth of CommSee is the concept of developing modular, independent parts of the solution, each of which delivers specific business functionality. These parts are known as “vertical slices” because they encompass the full vertical infrastructure of CommSee frameworks, from the UI to the database queries. Each slice consists of a WinPart, a private service, and possible back-end implementations such as stored procedures or public services. A business analyst (BA) is assigned to oversee the functionality of a vertical slice, and a developer is responsible for implementing this functionality. This approach was adopted because smaller teams are more manageable and enforce better communication standards. The developers on each team write all of the code associated with a particular vertical slice, frequently collaborating with and obtaining feedback from the BA through the Dev-BA environment, an isolated replica of the test environment that allows developers to quickly deploy new builds of CommSee and receive rapid feedback from the BAs. This methodology allows rapid, parallel development of independent, task-oriented, hot-pluggable functionality. This fosters an iterative, incremental approach to development; makes possible incremental delivery; helps to mitigate risk; and integrates the varying skill sets of developers. Adopting a hybrid methodology focused on developer independence and the separation of parts of the solution has enabled more than 200 developers to work together seamlessly but independently of each other.

Benefits of Vertical Slice Development Methodology The vertical slice model was used to isolate developers into smaller, more manageable teams that develop independent pieces of content. This approach eliminated the difficulties associated with using many shared components in such a large project; for example, the need to constantly re-test all consumers of the shared component when the shared component has changed. This approach also reduces the need for communication between hundreds of developers, as well as the delays associated with such communication. The following diagram summarises the use of vertical slice methodology by the CommSee development team.

Page 48 of 63

Figure 16: Development Methodology The CommSee project has demonstrated that an independent, vertical-slice approach is highly suitable to very large development teams, and that avoiding excessive communication and dependencies can bring unexpected advantages.

Page 49 of 63

Web Service Abstraction CommSee employs an attribute-based approach to Web service development. This means that the Web methods themselves do not entail implementation but instead are marked with the appropriate attributes to link the method to the implementation assembly. The advantage of this approach is that developers do not need to spend time writing implementation code for the Web services but instead can focus on implementing the business logic; the Web service implementation and binding is performed at compile time by custom generators. Using custom code generators, the Web service code could be generated automatically, allowing the complexity of the SOFA framework to be abstracted away from the developer. The separation of the service interface from the service implementation, and the reusability of infrastructural functionality through frameworks, offers the benefit of consistency in service development, focusing the resources on the implementation of business logic rather than the re-creation of existing infrastructure.

Development Tools The following primary tools are used by the CommSee developers and architects to build CommSee functionality and automate routine development tasks:

Visual Studio 2003 and .NET Framework 1.1. Visual Studio 2003 is the primary development tool used by all CommSee developers to implement the user interface and Web service functionality. CBA has also started the process of migrating CommSee to .NET Framework 2.0 and Visual Studio 2005 (see the Future Planning chapter later in this document).

BizTalk Server tools. Integrated into Visual Studio .NET, the BizTalk set of tools is used to create, monitor, and debug orchestrated business processes.

nUnit. nUnit is a unit-testing framework for managed code development. The CommSee development team uses nUnit to incorporate extensive unit tests into CommSee. nUnit takes advantage of many built-in .NET features such as custom attributes‟ reflection-related capabilities to create and monitor accurate unit tests. Currently, CBA is considering upgrading its unit testing infrastructure to Visual Studio Team System to take advantage of better integration into the Microsoft suite of products (see the Future Planning chapter later in this document).

CruiseControl.NET. CruiseControl.NET is an automated integration server that directly monitors the CommSee team‟s Microsoft® Visual SourceSafe® repository and automatically builds and validates the checked-in code. When the build is complete, CruiseControl.NET notifies the developer as to whether the changes that they committed integrated successfully or not.

Wise Installer. Wise Installer is an enterprise-level installation package creation tool used by the CommSee team to create automated MSI (Windows Installer) files for the CommSee client and services.

WiX. The Windows Installer XML (WiX) is a tool set that builds Windows installation packages described by an XML file. The CommSee team has integrated WiX into its build

Page 50 of 63

process to build MSI and MSM setup packages from the command line by using XML configuration files.

XMLSpy. XML Spy is used by the CommSee development team as the main integrated development environment (IDE) for aspects of XML development such as XML editing and validation, XSD schema creation, and XSL editing and transformation.

Rational Rose and the IBM IFW schema model. Rational Rose is an object-oriented Unified Modelling Language (UML) software design tool intended for visual modelling and component construction of enterprise-level software applications. The IFW service team uses Rational Rose to view and edit the IFW schema.

Custom tools and custom code generators. The CommSee development team has created a wide variety of customised tools used for automating development tasks such as initial vertical slice creation, service attribution, and more.

Custom Visual Studio plug-ins for developing WinParts. These plug-ins were created to automate the initial creation of WinParts, a core component of the CommSee UI. The generators create the WinPart class, inheriting and referencing appropriate components.

Simplified schema generator. This custom tool is used by the IFW Service team to generate XSD schemas from the IFW object model. The tool allows the developer to highlight the relevant parts of the IFW object model and generates an XSD file that is later used to create an IFW-compliant business object.

Dependency tool. This custom tool is used by the IFW Service team to analyse the impact of a proposed change to the database on public Web services and other systems.

Build Management

The CommSee development team uses a multi-stage build environment to deliver any changes to the testing team and other developers. Visual Source Safe is used as the source control tool, with plans to upgrade the environment to Visual Studio 2005 Team System. The team enforces a strict policy that code must compile and pass unit tests prior to check-in. Build tools such as WiX and CruiseControl.NET are used to initiate builds every time new code is checked in. These tools are used to produce binary files such as EXEs and DLLs, configuration files, and MSI installers for the CommSee client and Web service installation. These installations, as well as database scripts, are applied to the test environment every 1-2 days. To rapidly test interim builds, a developer can deploy code changes as required to the Dev-BA environment. The benefits of this multi-stage approach are continuous integration, quick feedback to the developer on build and logical issues, better quality of checked-in code, and the ability to test incremental changes without adversely affecting the large test environment.

Page 51 of 63

Test Approach As part of CommSee development, CBA has adopted an increased quality initiative, enforcing an end-to-end approach to testing and ensuring continual integration. Testing of CommSee is conducted throughout the solution lifecycle – from unit testing at development time to functional testing performed by more than 70 members of the Quality Assurance team. The following diagram summarises the testing approach adopted by CBA.

Developers

Use designers and code

generators to create initial

vertical slice

Unit tests incorporated from

the start by the automated

tools

Developers are encouraged

to write more complex unit

tests

Policy of not checking in code

that does not pass unit tests

Visual Source Safe

source control repository

Dev-BA interim

Testing environment

Business Analyst

Provide

immediate

feedback to

developer

Deploy interim changes

for immediate BA

feedback

Check in code that

builds, passes unit tests

and ready for testing

Build Server Farm

Uses CruiseControl

software to perform

builds every checkin and

also nightly

Reports build issues to

developer

Builds binaries and client

& server MSI installers

Testing Environment

Full solution builds

deployed every 1-2 days

QA Team

Perform regression,

functionality and QA

testing

TrackRecord bug

management site

Create Bug

Reports

Track bug assignments

Update bug status

Figure 17: Quality Assurance Approach

Page 52 of 63

The testing starts with developers incorporating unit test calls into the product code (For more information on the unit testing environment, see “Unit Testing” later in this section.) To assist with this task, designers are used to create the initial code from the generic vertical slice template incorporating base unit tests, and developers further customise them for the specific implementation. Developers also create new, more complex unit tests to ensure that all functions return correct values, expected by the caller. A policy of not checking in code that does not compile ensures that developers monitor and update unit tests as they update the code. After the vertical slice has reached a working state, it is submitted to the team‟s business analyst (BA) for interim feedback. In order for the BA to see the component without interfering with the development team, a separate Dev-BA environment is maintained. This environment is used to deploy interim changes for BA testing, without affecting the main testing environment. The BA provides immediate verbal feedback to the developers, ensuring faster turnaround of the initial development process. After the code has passed both the unit tests and the BA testing stage, it is considered ready for testing and is checked into the central Visual SourceSafe repository. CruiseControl.NET software then picks up the latest code to create a build of the binary files and MSI installers. If a build issue or a conflict occurs during the build process, it is reported back to the developer. Every 1-2 days, the latest build is taken from the Visual Source Safe repository and deployed to the test environment. The test team then performs regression, functionality, and QA testing on the test environment server. The bugs are recorded by using TrackRecord software on the test management intranet Web site and are assigned to the appropriate developers for resolution.

Unit Testing

Unit testing has been a critical factor in the overall quality of the solution. Investing time in setting up unit testing infrastructure and educating developers about the importance of unit testing has provided excellent return by creating a higher-quality solution, minimising time and the cost of additional testing by the testing team and bug convergence. The development team has incorporated basic unit tests into the code generated by their Visual Studio designers and automatic code generators. Developers then customise and create additional unit tests to ensure adequate coverage of the functionality they are implementing. This has helped ensure that unit testing is incorporated into source code from the start, and has overcome the issue of the developer‟s initial reluctance to write unit tests. The developers have been educated about the importance and benefits of unit testing and are actively encouraged to incorporate more complex unit tests into their code. The development team has also adopted the policy of not checking in code that does not pass associated unit tests.

Ensuring Continual Integration

Any solution on the scale of CommSee presents an integration challenge. Ensuring that components written by more than 200 developers integrate and work with existing and other new components is no easy task. This challenge was addressed in two different ways. First, the Continual Integration environment ensures that with every check-in, the entire code base is built and tested, thus allowing breaking changes to be detected quickly. Second, the modular architecture and vertical slice development process adopted helps reduce the likelihood of integration problems, as each team is operating largely independent of the others.

Page 53 of 63

The Dev-BA Testing Environment This special testing environment allows developers to quickly deploy early changes without affecting the build of the entire product. The code deployed to the Dev-BA environment is early code that allows the BA to provide immediate feedback to the developer. One of the key benefits of this approach is that it eliminates the “it only works on the developer‟s machine” phenomenon from the beginning.

Load Testing

CBA employs a Compuware solution to conduct load testing prior to each quarterly rollout. A team of three people is dedicated to load testing. A dedicated test environment is configured consisting of a farm of four Web service servers and a number of load clients. The load-testing plan is designed to test new features and critical parts of the solution.

Page 54 of 63

Deployment and Operations Approach New releases of CommSee are deployed quarterly. The following diagram summarises the deployment process.

Visual Source Safe

source control repository

Release Build:

* Client MSI

* Final Database Script

* Server MSIs

* IFW Installs

* BizTalk Assemblies

Release Services

Use SMS for Client and Altiris

for Server Deployment

Take down all systems

Install Server MSIs using Altiris

Run DB Scripts

Make manual config changes

Web Service Servers

Servers run current Version +

previous version for backwards

compatibility

Branch Site Clients

Deploy new version of

SOFA and web

services

Deploy client MSI

1000 clients initially deployed

After 2 week performance

monitoring period deploy to

remaining ~29000 clients

Figure 18: Deployment

Packaging

The release build contains client and server installation files in the form of MSI (Windows Installer) files. These installers are verified before hand-off to the release management team. As part of the hand-off, additional deployment components are created such as final database scripts, IFW installation files, and BizTalk assemblies, as well as release documentation.

Client Deployment

Previously, Radia was used as a solution to deploy client MSI installations. However, Radia has now been replaced with Microsoft Systems Management Server (SMS). The client MSI is rolled out to approximately 1,000 pilot users and is closely monitored for two weeks. After the monitoring period, the MSI is rolled out to the remaining users.

Page 55 of 63

Updates can be deployed via the CommSee client‟s built-in automatic updating feature which prevents the need for the entire package to be deployed for minor updates.

Server Deployment

Multiple MSI installers are created for installation of private and public services. The release of the SOFA framework and IFW object libraries is set to take place at the same time as a CommSee release. The installer includes required binaries, Web pages, and configuration files. The installation is pushed out to all SOFA servers by using Altiris Server Management Suite. Because many services are deployed on each server, automation is critical to a successful rollout.

Versioning

Versioning of the CommSee solution is designed for simplicity and optimal manageability. Therefore, no binary compatibility is required between different versions of client and service. (A version of a service supports only the corresponding version of the client, and vice versa.) In order to support older clients during transition periods, the servers host the current as well as the one previous version of the service running side by side. The version of the service is incorporated into the service URL, thereby ensuring that correct binding is always set. In order for the database stored procedures to be able to make decisions based on the client version, the version of the calling client is visible to the Web services and the database.

Page 56 of 63

Future Planning CommSee is a growing solution designed to last many decades through extensibility features. As new technologies emerge, the CommSee team will integrate them into the existing solution. The technical future of CommSee lies with technologies such as Windows Communication Foundation (WCF), Windows Presentation Foundation (WPF), BizTalk Server 2006, and .NET Framework 2.0. In order to make the solution “future-proof,” design decisions had to be made to ensure compliance with emerging technologies. The business direction of CommSee is to provide services to many other consumers, thus opening new business markets and providing better service across the board.

Visual Studio 2005 / .NET Framework 2.0

The current release of CommSee has already been successfully ported to .NET Framework 2.0 and is in the process of being rigorously tested before deployment. The new version of the .NET Framework offers performance and framework advantages to CommSee, while Visual Studio 2005 offers many new developer productivity enhancements.

Visual Studio Team System The CommSee team will implement Visual Studio Team System to provide a single tool that will facilitate unit testing, integrated bug reporting, and better collaboration between team members.

SOFA Layer – Preparing for WCF

Choosing the best abstractions for building software is an ongoing process and is very important to CBA‟s architecture team. Windows Communication Foundation (formerly code-named “Indigo”) is Microsoft‟s strategic technology choice for building easily scalable, high-performing, and interoperable service-oriented applications. The WCF architecture implements SOAP and other Web services technologies, allowing developers to create reliable, secure, and transactional services that can interoperate with software running on any system. CBA has focused on the following aspects of WCF:

Unification of Microsoft's Distributed Computing Technologies. The .NET Framework includes several different technologies for creating distributed applications, many of them used by CBA. These include:

o Interoperable Web services o Communication between multiple .NET Framework–based applications o Distributed transactions and Enterprise Services o The latest Web services specifications, such as WS-Addressing and WS-Security

(as in Web Services Enhancements)

Page 57 of 63

o Message Queuing Technologies (Microsoft Message Queuing [MSMQ])

WCF allows the integration of all these technologies into a single communications platform.

Support for service-oriented development. WCF places a clear focus on services as distinct from objects. The four key tenets in the design of WCF are:

o Shared schema. Unlike older distributed object technologies, services interact

with their clients only through a well-defined XML interface. Behaviours such as passing complete classes, methods and all, across service boundaries are not allowed.

o Autonomous services. A service and its clients agree on the interface between

them, but are otherwise independent. They may be written in different languages, use different run-time environments (such as the Common Language Runtime [CLR] and the Java Virtual Machine), execute on different operating systems, and may differ in other ways as well.

o Explicit boundaries. A goal of distributed object technologies such as

Distributed COM (DCOM) was to make remote objects look as much like local objects as possible. Although this approach simplified development in some ways by providing a common programming model, it also hid the inescapable differences between local objects and remote objects. Services avoid this problem by making interactions between services and their clients more explicit. Hiding distribution is not a goal.

o Policy-based compatibility. When possible, determining which options to use

between systems should rely on WS-Policy–based mechanisms.

As the figure below shows, every WCF service is constructed from three components:

A service class, implemented in C#, Visual Basic .NET, or another CLR-based language, that implements one or more methods

A host environment (an application domain and process) in which the service runs

One or more endpoints that allow clients to access the service

Page 58 of 63

Figure 19: Windows Communication Foundation All communication with a WCF service happens via the service‟s endpoints. Each endpoint specifies a contract that identifies which methods are accessible via that endpoint, a binding that determines how a client can communicate with the endpoint, and an address that indicates where the endpoint can be found. A WCF service class is a standard .NET class with a few additions. These additions allow the class‟s creator to define one or more contracts that this class implements. Each WCF service class implements at least one service contract, which defines the operations this service exposes. A service class might also explicitly implement a data contract, which defines the data those operations convey. The SOFA layer was built with WCF in mind. Many of the concepts such as a modular pipeline, separation of interface from implementation, service wrappers, and attribute-based development are shared; however, the implementation will be different. The architecture that is adopted means that the SOFA framework can be ported to WCF without too much effort.

CommSee UI – Preparing for WPF The Microsoft Windows Presentation Foundation (formerly code-named “Avalon”) provides the foundation for building applications and high-fidelity experiences in Windows Vista and Windows XP, blending together application UI, documents, and media content, while exploiting the full power of the PC. The functionality extends to support for Tablet and other forms of input, a more modern imaging and printing pipeline, accessibility and UI automation infrastructure, data-driven UI and visualisation, and the integration points for weaving the application experience into the Windows shell. The CommSee client is already using many paradigms that WPF makes significantly easier to implement, such as rich visualisation of business data. The key advantages of WPF to the design of CommSee are:

Hardware-accelerated rendering. The underlying WPF engine makes possible animated 3-D meshes with video materials composited on an animated image montage; sub-pixel clear type text; 2-D animation with vector-based shapes; and more. The engine is hardware-accelerated and integrates these different media into a common experience.

Unified programming model for all media types. WPF provides a common programming model that integrates 2-D vector shapes, 2-D animation, images, 3-D geometries, 3-D animation, video, and text into a single common platform.

3-D support. 3-D is used in subtle but powerful ways to enhance the user experience, employing a third dimension to provide intuitive UI metaphors. CommSee already uses many 3-D objects in its interface, and WPF streamlines the development of 3-D interface components.

Full access to the .NET Framework. WPF supports powerful features of the .NET platform, such as its XML de-serialisation capabilities.

Interoperability with Windows Forms. The investment CBA has already made with Windows Forms will be preserved with WPF. Windows Forms and WPF will provide seamless interoperability, meaning the CommSee harness could be rewritten in WPF and

Page 59 of 63

all of the existing WinParts could still be used as is, or various new WinParts could be implemented in WPF and integrate seamlessly in the existing Windows Forms infrastructure. This allows CBA to continue to leverage its previous investments while also taking advantage of the features and benefits of WPF where it makes sense for CBA to do so.

The highly graphical and interactive nature of the CommSee UI will benefit greatly from the new capabilities offered by WFP.

Page 60 of 63

Lessons Learned The following section summarises key lessons, success strategies, and best practices learned during the CommSee development life cycle:

Build instead of buy. Sometimes, investing in long-term, extensible, tailor-made solutions understood and maintained by an organisations own staff can be a much better solution than purchasing a monolithic packaged application. For CBA, the decision to build instead of buy has brought many benefits, including:

o Increasing agility – CBA has a very flexible system that it can easily modify to

support its changing business needs. o Delivery of a far superior user experience than is available with an off-the-shelf

product. o Ability to customise based on CBA‟s unique requirements – CBA does not need

to change its business practices to suit its software package, which is often the case when implementing a large off-the-shelf CRM system. Instead, CommSee is designed to suit CBA‟s specific business needs.

o Substantially lowered total cost of ownership (TCO) of the solution Choosing this approach has also ensured the longevity of the project, with in-house developers building more business functionality into the solution as business practices evolve.

Invest in Framework Development. Frameworks have proven to be key to the extensibility of the solution. The time spent during thorough investigation into what functionality the frameworks should provide, and during development of the frameworks, was recovered by streamlining and simplifying the process of business logic development.

Integrate existing technology assets into the new solution. Many organisations choose to rewrite their existing software applications as new technology becomes available. However, in a large enterprise such as CBA, rewriting existing applications is not a viable solution due to time and resource restrictions. Expending the effort to find out how these existing assets can be integrated into the new solution pays off because a solution can be delivered faster, and past investment in IT is preserved. CBA has integrated both back-end and front-end systems into CommSee, bringing the project in on time and achieving a better return on investment (ROI) from the past expenditure.

Standardise business vocabulary and implement the “Single Source of Truth” pattern. Large organisations often have many disparate systems holding and describing business information using different vocabularies. This results in duplication of effort and out-of-date data, leading to slower and less accurate business decisions. Agreeing on a standard business vocabulary and standardising data-access practices is no easy task; however, as the CommSee project has proven, it delivers substantial business benefits. Many large organisations who attempt to create their own standard vocabulary (or schema) often fail, and in the process waste many years and millions of dollars. The most challenging part of implementing a standard or schema throughout such a large enterprise is firstly getting all disparate parts of the organisation to work collaboratively to come up with the schema, and secondly, to agree on it. CBA chose a different approach, they chose to by-pass the usually costly exercise of creating their own schema and instead bought one (IBM's Information Framework) and enforced its use across the organisation. Purchasing or adopting a schema and enforcing it across the organisation is generally much easier way to achieve a common vocabulary

Page 61 of 63

than trying to create your own. The approach of purchasing a schema and enforcing it across the different divisions has been a key component of CBA‟s success.

Use Web Services as a standard for all communications. For CBA, standardising communication methods on Web Services built in .NET has meant that time spent developing communication patterns pays for itself when they are used many times throughout the organisation.

Leverage the power of the client. To build a highly scalable solution, minimise bandwidth, and implement a rich, visual interface, CBA chose to implement the front-end of CommSee as a Smart Client. This approach has proven to be highly extensible, thereby preserving investment in existing front-end systems. For more information on the Smart Client pattern, visit the Smart Client Developer Centre at http://msdn.microsoft.com/smartclient/ .

Find new ways to present business data. Presenting the user with a rich, highly visualised way to access standard business data has brought many technical and business benefits, from better usage of screen real estate to greater employee satisfaction. Although it is sometimes difficult to move away from standard business screens characterised by grids and text details, doing so has resulted in a fresh new way of looking at business processes.

Manage memory effectively. When developing a highly visual Smart Client interface, memory management becomes a top priority. Understanding memory management best practices has been critical in CBA‟s development of a stable, successful software solution. CBA has discovered that memory leaks, although not as prevalent in managed languages such as C#, do happen, and need to be addressed. Review the .NET memory management best practices by visiting http://msdn.microsoft.com/msdnmag/issues/05/01/MemoryOptimization/

Use caching for scalability and performance. The use of smart caching has meant better performance and scalability for the CommSee solution (for more information on how caching is implemented in CommSee, see the “Caching” section of the Solution Design chapter). One of the lessons learned by the CommSee team is that caching can be difficult to implement and test. To address these difficulties, Microsoft recommends leveraging something like the Enterprise Library Caching Application Block. You can find out more about this component by visiting http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnpag2/html/EntLibJan2006_CachingAppBlock.asp .

Leverage Visual Studio Designers. Visual Studio 2003 designers and custom design-time components are heavily used in CommSee development because they provide a standard way for developers to build their parts of the solution. These components greatly speed up creation of new content and standardise developer output; for example, by implementing a common naming convention and incorporating initial unit tests. However, the CommSee team has experienced some issues with the designers; for example, realising that the workings of a designer are hidden and that it is difficult to debug or rewrite the designer. To better understand Visual Studio designers, visit “Developing Windows Forms Controls at Design Time” at http://msdn2.microsoft.com/en-us/library/w29y3h59(VS.80).aspx ; “Walkthrough: Creating a Windows Forms Control That Takes Advantage of Visual Studio Design-Time Features” at http://msdn2.microsoft.com/en-us/library/35ea88wb.aspx ; “Extending Design-Time Support How-to and Walkthrough Topics” at http://msdn2.microsoft.com/en-us/library/ms171818(VS.80).aspx ; and “External Resources for Extending Design-Time Support” at http://msdn2.microsoft.com/en-us/library/ms171844(VS.80).aspx .

Page 62 of 63

Implement a disconnected development approach. Many organisations rely on shared software components to reduce the development time of a solution and implement better scalability and better performance. However, the CommSee development team has discovered that with a very large development team, relying too much on shared components leads to increased coupling and reduced agility. Communications between more than 200 developers are hard to manage, and knowing who will be affected by a proposed change to a shared component is virtually impossible. CBA has addressed these difficulties by eliminating any unnecessary communications between members of the development team and by using the Vertical Slice Methodology described in the Development Approach chapter of this case study.

Page 63 of 63

Conclusion CBA has successfully redesigned its core software solution to implement new business processes and standardise the way its employees access customer information. This effort involved making key business decisions, such as changing CBA‟s approach to conducting its core business from a transaction-based to customer-focused banking model, as well as implementing a standard business vocabulary. The CommSee solution was built on the .NET Framework, using SmartClient approach and Web services as a standard communications mechanism. The solution has delivered numerous business benefits, from reduced paperwork to streamlined decision-making to reduced costs and increased efficiency. The CommSee project has been a huge software development effort and has proven that .NET technology can deliver solutions for even the largest enterprises.

Special Thanks Many thanks go out to the Commonwealth Bank of Australia, the entire CommSee team and everyone involved in making CommSee a success. In addition, a special thanks to everyone who was instrumental in the creation of this case study, including:

Anthony Ash, Mark Bugeja, David Faulkner, Jeremy Forkgen, Dan Green, Clement Koh, Steven Lau, Darren Matson, Yuri Misnik (Microsoft), Peter O‟Dowd (Blade/Wadeware), Ramesh Vaikuntam (Microsoft), Tom Williams