15
Intelliflo – Cloud Environment Case Study. 18/02/2020
Intelliflo – Cloud Environment Case Study.
18/02/2020
Intelliflo – Cloud Environment
2
Introduction.
The project is characterised as follows:
• Client/Operating Unit – Intelliflo, Invesco
• Industry Vertical – Financial Services
• Products/Services – outline application for Financial Advisors, Mortgage Brokers,
and Financial Product Providers
• Case Study Title – Intelliflo, DevOps, Cloud Migration
Intelliflo, Invesco are a provider of one financial services products based in Kingston. Its
current market is the United Kingdom which is being extended to provide a global
reach.
The overall project goal was to provide a cloud environment to house the Intelliflo
application portfolio and customer data and thereby service a global customer base,
as opposed to just the United Kingdom.
The migration to a cloud platform was also influenced by a desire to support an
increased number of customer and to provide a resilient platform which does not exhibit
some of the constraints of the current Disaster Recovery provisions.
Work started in August 2019 which saw the environment being completed in January
2020, ready for the containerisation and relocation of applications to the environment
during 2020.
This case study has been compiled to showcase the project and demonstrate the
DevOps best practices created and implemented for Intelliflo to facilitate a rapid
growth in their business.
The Challenge.
During 2019 Intelliflo were acquired by Invesco, an investment management company.
The acquisition of Intelliflo has seen an increased focus on global markets and a desire
to offer access to applications in Australia, Germany and Canada.
To provide the Intelliflo application suite with a global reach requires the existing on-
premise applications and customer data be migrated to the cloud. This is being
Intelliflo – Cloud Environment
3
undertaken in several phases starting with the design and implementation of an
underpinning environment.
The current application portfolio includes online applications for Financial Advisors,
Mortgage Brokers, and Financial Product Providers based around a Microsoft
technology stack including Active Directory and SQL Server databases.
The cloud environment needs to provide a secure location for the development and
automated test of applications, and their staging to production through a secure
automated delivery pipeline with the associated audit controls.
Intelliflo staff were assigned to form an element of the Project team to assimilate best
practice, and the knowledge required to transition the existing service delivery model to
a DevOps model.
Key focus areas included:
• application deployments relying on Microsoft SQL server database and
credentials being created on a per app basis by the internal operations slowing
deployment.
• the storage of unrestricted amounts of Customer data with high levels of
availability and redundancy
• support for large amounts of application containerisation with scalable hosting
platforms
• central store for infrastructure and application logs, infrastructure monitoring on
containers and virtual machines.
The initial Project Discovery phase identified the following Key Performance Indicators
for the Intelliflo environment:
• Deployment of business applications to Australia, Germany and Canada within a
period of six months
• Product augmentation and release cycle of 1-hour catering for a 100% increase
in the volume of changes
• Centralise regulatory and compliance controls through a common technology
platform, security model
Intelliflo – Cloud Environment
4
• Platform for the existing applications portfolio and its augmentation for at least
the next five years
• Increased flexibility and future proofing through a solution landscape defined
using infrastructure as code
• Reduced overall IT operating costs through a DevOps model and shared Cloud
platform.
The remainder of this Case Study focuses on the challenges surrounding creation of the
Cloud environment.
Solution.
DevOps Transformation
The key objective of the DevOps transformation was to provide a Cloud based
environment into which Intelliflo could relocate its suite of on-premises business
applications.
The salient aspects of the DevOps transformation comprised the delivery of:
• Infrastructure Provisioning, scripted service catalogue for the relocation and
instantiation of Intelliflo applications with a global reach (Terraform Cloud)
• Build and Test – repositories and a three-stage delivery pipeline, including
Intelliflo automated testing, to support the delivery and release of changes,
including audit controls
• Release and Acceptance – an automated release process from Pre-production
to Production environments following successful automated tests, which includes
audit controls
• Configuration Management – driven by software development changes using
GitHub and Ansible as the entry point to the delivery pipeline Test environment
• Container Services – catered for by the environment to enable the
containerisation and relocation of Intelliflo application suite
• Monitoring and Tracing – monitoring of Amazon EC2instances, Amazon
S3buckets and application operational data for troubleshooting
Intelliflo – Cloud Environment
5
• Identity and Access Management – is used as an integral element of the security
model to control access to the S3 buckets storing Customer data.
Technology Transformation
Overview
The key objectives of the technology transformation were to create a cloud
environment to host Intelliflo applications and facilitate their access across the globe.
The initial project phase comprises the creation of an environment containing test, pre-
production and production areas, each of which is supported by a separate stage in
the delivery pipeline.
At a conceptual level the environment comprises the following components:
• Environment provisioning
• Delivery pipeline
• Monitoring and audit
• Communications infrastructure.
Subsequent Project phases are planned to perform the containerisation and relocation
of applications and Customer data to the environment created by ECS.
Cloud Environment
Environment provisioning is performed via a service catalogue that enacts a suite of
Terraform ‘runs’ managed through Terraform Cloud and stored in a GitHub repository.
By adopting a modular architecture changes can be readily accommodated to
portions of the environment.
The secure storage of unlimited volumes of Customer data across multiple zones with
high levels of availability is a key. Consequently, S3 buckets have been used which have
no size limit, and cross region replication which can be rapidly configured. Access to the
S3 buckets is restricted using IAM user policies and ACLs to implement the required
security permissions.
The Intelliflo applications are tied to a specific version of SQL Server database which
needs to be hosted as a cluster using EC2. Until the Intelliflo applications are approved
Intelliflo – Cloud Environment
6
to host data in the cloud a direct connect will be provided to there on premise
database instances.
A key consideration is Intelliflo’s desire to refactor their applications and reduce their
reliance on Active Directory. Consequently, AWS Directory Services are for SSO, AWS
Workspaces, and Active Directory groups are used by Vault to control access to
services.
Application and servers are accessed and managed via AWS Workspaces and Session
Manager and not directly from the outside world.
Applications are hosted via AWS EKS using Kubernetes controllers, such as EFS
provisioner, external DNS, ALB Ingress and monitored using AWS CloudWatch agents.
Harness will be used for all application/infrastructure pipeline deployments.
This provides an elegant mechanism for developers to deploy their applications, which
can be restricted per area, via the associated AWS account. SpotInst Ocean is used to
automatically scale EC2 workers based on load avoid AWS EKS maintenance outside of
version upgrades.
Figure 1 – Environment Overview
The following decisions are reflected in the environment:
• applications have a separate Route53 record and may be reached by load
balancers which route to the applications hosted in EC2 or to the EKS workers
Intelliflo – Cloud Environment
7
• Terraform runs are used to provision all AWS resources and store software in
separate GitHub repositories; state files are stored in workspaces in Terraform
cloud
• Hashicorp Vault is used to manage the credentials for applications and
deployment using clusters on EC2
• Consul is used for service delivery combined Kubernetes Service Mesh to support
clusters using EC2
• AWS Control Tower is used to manage a master AWS accounts for the test, pre-
production and production areas of the environment.
Delivery Pipeline
A three-stage pipeline has been used comprising test, pre-production and production
areas the entry point to which is software changes which are stored in GitHub; changes
enact an automated build and functional test, defined and maintained by Intelliflo.
Automated functional tests are performed at each stage of the pipeline, and if the tests
pass, the change is automatically progress through the pipeline.
GitHub repositories are used for configuration management and Jenkins, including its
Control Centre, is used to implement continuous integration.
A central shared services account hosts the master Jenkins instance, from which further
instantiations can be provisioned and configured for each area. It is also used to share
services such as Active Directory between accounts.
To maintain flexibility Jenkins scripts have been used to provide modular pipeline
elements that deploy to selected all parts of the environment.
Intelliflo – Cloud Environment
8
Figure 2 – Delivery Pipeline
Monitoring and Audit
Application logs, infrastructure logs, EC2 logs and container logs are published to
CloudWatch using FluentD. CloudWatch provides a central indexed location for
troubleshooting the logs, see below.
EKS nodes and application pods are monitored using AWS CloudWatch integration and
the AWS CloudWatch agents within EKS. AWS CloudWatch agents are also used to
monitor the EC2 instances, and to display alerts and availability centrally.
Figure 3 – Audit/Logging Infrastructure
Intelliflo – Cloud Environment
9
Communications Infrastructure
A new communications infrastructure was established to host the Intelliflo environment
which comprises subnets for public, management and application traffic, each of which
use a single VPC account, see below.
The management and application subnets only host internally reachable applications
and infrastructure, while the public subnet provides a DMZ to the other subnets to
control network connectivity.
VPC security groups and NACLs have also been used to restrict inbound and outbound
connections, complemented with VPC peering to provide connectivity between
accounts.
Figure 4 – Communications Infrastructure
Intelliflo – Cloud Environment
10
Project Execution
The Project comprised a Discovery phase followed by several Enablement phases. Each
of the phases was undertaken by a self-managed team of DevOps specialists,
undertaking business and technical transformation activities, referred to a POD.
The project team comprised five engineers including a Project Lead, Security Specialist
and Windows Specialist supported by an Agile Coach. The team was located at the
Intelliflo Offices in Kingston to work closely with their DevOps Manager and business
sponsors.
The project comprised the follow activities:
• Project Inception, a Discovery phase that identified a significant change in
Project scope, as requested by Intelliflo, which was addressed through a solution
Road Map
• Project Execution, the completion of fixed duration Sprints to deliver the required
environment through Proof Of Concept spikes
• Delivery Management, through the application of a Sprint Backlog populated
with agreed elements of the Road Map
• Scope Management, was performed using the Road Map against which changes
were raised
• Customer Acceptance, performed via demonstrations including the DevOps
Manager, Programme Manager and senior business sponsors and an
Acceptance Certificate
• Customer Management, conducted by an ECS Account Executive liaising with
the DevOps Manager, and a complete Customer Satisfaction Questionnaire.
Intelliflo – Cloud Environment
11
Services Used.
The following AWS services were used to deliver the DevOps solution:
Service Application
AWS Lambda Various copy, resizing and
Intelliflo AWS island
AWS CloudWatch Monitoring the infrastructure to
underpin dashboards
AWS Config Baseline change alerts and
auditing
Amazon EC2 Provisions virtual machines for
non-containerised applications
AWS X-Ray Ad-hoc container
troubleshooting
AWS Control Tower Provisions new AWS accounts
AWS API Gateway Implements API's to EKS/EC2
hosted applications
AWS S3 (Simple Storage) Stores Intelliflo and Customer
documents
AWS IAM (Identity Access
Management)
Manages user permissions to
AWS resources
AWS GuardDuty Performs continuous threat
detection for provisioned AWS
accounts
AWS VPC Deploys the underlying network
infrastructure and layer 3
firewalls
Intelliflo – Cloud Environment
12
Service Application
AWS SNS (Simple Notification
Service)
Inter application
communication using
microservices
Amazon EKS (Elastic Kubernetes
Service)
Hosts the majority applications
in Kubernetes pods
AWS ECR (Elastic Container
Registry)
Stores container images
AWS Direct Connect Connects AWS eco-system to
On-premises applications
(migration)
AWS Route53 Performs external DNS routing
and domain purchase via the
registrar
AWS SSO (Single Sign On) Centrally manages access to all
AWS accounts
AWS KMS (Key Management
Service)
Stores encryption keys for
secrets and customer data
AWS EFS (Elastic File System) Hosts NFS volume for Jenkins
backend storage (initially)
AWS Backup Performs backups of EFS
(fileshares) and EBS (Volumes)
Amazon ElastiCache Host Redis caches for
applications to store data for
rapid access
AWS Secrets Manager Stores Hashicorp Vault recovery
keys and root tokens
AWS Certificate Manager Manages certificates for
domains purchased in Route53
Intelliflo – Cloud Environment
13
Third Party Solutions.
The following 3rd party packages augment the application ecosystem.
Solution Application Use Cases
Terraform Infrastructure-as-Code Provision AWS Resources,
Deploy Kebernetes Helm
Chart
Terraform Cloud Backend State file storage Provision AWS Resources,
Deploy Kebernetes Helm
Chart
Artifactory Image/Nuget package
store
Store Container Images
Store Nuget Packages
Vault Secret Manager Store Active Directory
Secrets, Generate Active
Directory Secrets
Consul Service Discovery Provide Service Mesh,
merge Legacy Services
Helm Kubernetes package
manager
Manage Kebernetes
Application
SQL Server Customer Database Manage Application
Database
Cloudbees Jenkins Continuous Integration Orchestrate Application
Life-cycle, Promote
Application Software
Intelliflo – Cloud Environment
14
Metrics for Success.
The deployment of business applications was extended from the United Kingdom to
Asia Pacific (Australia) within six months, with the goal of reaching the remainder of
Europe and North America as quickly as possible.
The application availability has been increased through the increased resilience
provided by the Cloud as previously in a disaster recovery scenario the user population
supported by a fall-back data centre was reduced significantly.
The product change cycle has been reduced from 24 hours to 1 hour and can easily
cater for a 100% increase in the volume of changes (currently fourteen per day), with
improved quality through automated testing.
Regulatory and compliance controls have been augmented through a common
technology platform, security model and centrally managed security controls, which can
also be readily changed to accommodate new security threats.
The technology transformation and the increased flexibility enabled by using
infrastructure as code will provide a platform for the
The Outcome.
The skilling of Intelliflo staff in DevOps practices was constrained by their initial staffing
levels and the level of business as usual activity. This has now been addressed and a
further two staff are being assigned to the project.
Customer changes of scope and in-flight learnings impacted the delivery dates which
was managed through prioritisation and re-scoping. Ideally more technical spikes
should have been included to allow for subtleties in the Intelliflo Environment.
ECS recommended the replacement of Bastion Hosts by Amazon Workspaces. This was
agreed by the Intelliflo once it had been confirmed that the required functionality was
provided. This decision resulted in reduced effort to deliver and maintain the
environment.
The use of Terraform Cloud to provision the test, pre-production and production
environments has significantly reduced the manual effort required to create
environments. These can now be provisioned within minutes through a Service
Catalogue.
Intelliflo – Cloud Environment
15
The extra work required for the Intelliflo development staff to host their applications
Kubernetes as opposed to EC2 was significant. This included ALB Ingress, Route 53, and
NGINX controllers, all of which required significant levels of testing.
London Floor 7, 2 More London Riverside, London, SE1 2AP +44 (0) 207 403 0477 Edinburgh Apex 3, 95 Haymarket Terrace, Edinburgh, EH12 5HD +44 (0) 131 543 3215 Glasgow Parkview House, 6 Woodside Place, Glasgow, G3 7QF +44 (0) 141 572 3040 Manchester Bartle House, Oxford Court, Manchester, M2 3WQ +44 (0) 161 407 0069 India Office 203, 2nd Floor Buidling 2, Commerzone, Survey 144/145 Yerwada Samrat Ashok Path Pune 411006 +91 (0) 206 726 0700 Singapore 8 Wilkie Road, 3-01 Wilkie Edge, Singapore 228095 +65 96830785
