Life Outside of TFS

transcript

Life Outside of TFSAndrew Ochsner

DISCLAIMER

About CSG

Business Support System (BSS) and Operational Support System (OSS) software• Product, Order, Billing, and Customer management• Inventory, Provisioning, and Job management• Primarily North America cable, satellite, and telecom companies• Expanding internationally• Hosted solution

10s of Millions of Lines of Code• Mainframe – Assembler, COBOL, JCL• Linux/Solaris – C/C++• Linux – Java• Windows – .NET/C#, C/C++, Java

Databases• SQL Server• Oracle• DB2 on Mainframe

10s of Millions of Lines of Code• Mainframe – Assembler, COBOL, JCL• Linux/Solaris – C/C++• Linux – Java• Windows – .NET/C#, C/C++, Java

Databases• SQL Server• Oracle• DB2 on Mainframe

Agile Transformation

Waterfall

Scaled Agile Framework (SAFe)

http://scaledagileframework.com/

CSG’s ALM Approach

Best in breed approach• Mostly an evolution – no big bang• Integrated what we had• Upgrade to better components as we find them

Solidifying around Atlassian Suite • Not Open Source but customers are allowed to view and modify

source• Integrates with many third party tools• Web based• Hosted or On Premise• Rich plugin ecosystem/marketplace• Open web-based APIs and SDKs

Core Principles

Have No Fear Trunk Based Development Continuous Integration

• Commit early• Commit often

Collective Code Ownership Demand ownership of failing builds Standardized project layout Dedicated System Team (ESM)

• Staffed with good developers

Development Lifecycle

Issue Tracking – Atlassian JIRA Source Code Control – VisualSVN (Subversion) Package Repository –Artifactory/NuGet Testing – NUnit & SpecFlow Database Migrations – CSG Tools Continuous Builds – Jenkins/NAnt/MSBuild Code Reviews – Atlassian Crucible Code Search – Atlassian Fisheye Installers - WiX Automated Deployments – BMC BladeLogic

Issue Tracker

Jira – Issue Tracker

Web based issue tracker created by Atlassian On-Premise or Hosted Very flexible custom fields and workflows

• Lots of third party plugins• Mobile app• Plugin SDK to build your own

Scrum and Kanban story boards EpicsFeaturesStoriesTasks Development and Production Defects Production Defects synchronize with Remedy IDE Integration Reports and Charts are lacking a bit

Issue Tracker

Developer Tools

Visual Studio Resharper VisualSVN – Visual Studio Subversion integration NUnit & SpecFlow CSG Developer Tools

• CSGCopyDeps – Copy dependencies – LibraryManifest.xml• CSGVerifySolution – Enforce Standard Project Layout• SqlServer Management Studio Addins for Database Migrations

Developer Tools

Visual Studio Resharper VisualSVN – Visual Studio Subversion integration NUnit & SpecFlow CSG Developer Tools

• CSGCopyDeps – Copy dependencies – LibraryManifest.xml• CSGVerifySolution – Enforce Standard Project Layout• SqlServer Management Studio Addins for Database Migrations

Standard Project Layout

Built Assemblies (not checked in)

Database Schema & Static Data

Dependencies (not checked in)Deploy configs and build script hooksSource code

Module specific build option settings

Module Solution File

Module Installer Solution File

Generated WSDL

Documentation

CSGCopyDeps – SCM\LibraryManifest.xml

CSGCopyDeps

CSGVerifySolution

Developer Tools

Package Repositories

Artifactory - Package Repositories

Need the ability to grab the build artifacts of any given version

Third party dependencies too! Don’t check binaries into Subversion Non .NET teams have same needs Eventually could replace CSGCopyDeps with NuGet

Artifactory - Package Repositories

Artifactory Pro by JFrog• Supports NuGet (Pro), Ivy, Maven

Web Based & On-Premise or Hosted Stores CSG built packages along with version Acts as a remote proxy between build tool and external

galleries• Pin a package to a particular version• Governance around approved Third Party Software and

Licenses (Pro w/ BlackDuck integration)

Package Repositories

Source Code Control

Subversion - Source Control

Subversion – Open Source Centralized Version Control Copy-Modify-Merge rather than Lock-Modify-Unlock Directories are first class objects, just like files Branching and Tagging are “cheap”

• Copy on Write Widespread adoption & integration with other tools No Stash/Shelve features

• We utilize a Branch

VisualSVN Server – On Premise Subversion installation for Windows• ActiveDirectory integration

VisualSVN – Visual Studio Subversion Integration TortoiseSVN – Windows Explorer Integration

VisualSVN Server – On Premise Subversion installation for Windows• ActiveDirectory integration

VisualSVN – Visual Studio Subversion Integration TortoiseSVN – Windows Explorer Integration

Source Code Control

Continuous Integration

Jenkins – Continuous Integration

Web-based, open source continuous integration server On-Premise or Hosted by Cloudbees Fork of Hudson project Tons of plugins to integrate with just about anything

• Write your own as needed Open APIs and command line tools to automate as

needed• JenkinsGenerator

One Master (Controller), Many Agents

Polls Repository for changes Triggers builds Generate Reports off of build artifacts

• Emails• Test Results

Polls Repository for changes Triggers builds Generate Reports off of build artifacts

• Emails• Test Results

Common build - NAnt

Checkout source (done by Jenkins slave) Get dependencies via CSGCopyDeps (varies by build

type) Run CSGVerifySolution Build solution - MSBuild Run Tests

• Build system only runs tests that don’t hit external systems• Smoke test builds run all tests

Run Code Analysis (varies by build type) Build Installers Publish Artifacts

Build Types

Every module has at least 3 builds• Daily Build (DSM)• Continuous Stable (CS)• Continuous Integration (CI)

Other build types• Smoke Test (SMOKETEST)• Continuous Stable Database (CSDB)• Database Restore (QRDB/PRDB)• Documentation (DOC)• Sonar (SONAR)

Build Types

Every module has at least 3 builds• Daily Build (DSM)• Continuous Stable (CS)• Continuous Integration (CI)

Other build types• Smoke Test (SMOKETEST)• Continuous Stable Database (CSDB)• Database Restore (QRDB/PRDB)• Documentation (DOC)• Sonar (SONAR)

Build Types – Daily Build

“The official build” Builds latest from SVN Uses dependencies officially staged from Package

Repository Results are staged into Package Repository CREATE and MIGRATION scripts also generated Triggered nightly

Build Types – Continuous Stable

“Current build” Builds latest from SVN Uses dependencies from Package Repository

(dependency DSMs) Runs extra checks such as Code Analysis Results are not staged into Package Repository Triggered off of SVN commit or dependency DSM

Build Types – Continuous Integration

“Tomorrow’s DSM” Builds latest from SVN Uses dependencies from latest CI of each module

dependency Results are not staged into Package Repository

• Go into a “sandbox” area to be used by other CI builds Triggered off of SVN commit or dependency CI

Build Types

Build Type Description

Daily Build (DSM) Builds the latest of a module's branch/trunk, including database CREATE and MIGRATION SCRIPTs, using other modules officially staged assets as dependencies, results are officially staged. Triggered Nightly.

Continuous Stable (CS) Builds the latest of that module's branch/trunk, using other modules' officially staged assets as dependencies, results are not officially staged. Triggered by an SVN change.

Continuous Integration (CI) Builds module's trunk and uses the artifacts of the latest successful CI build for each consumed module. Triggered by an SVN change. Causes all down stream modules' CI builds to run.

Smoke Tests Owned by module teams. Takes built artifacts from CS build and runs entire test suite.

Build Types

Build Type Description

Continuous Stable Database (CSDB)

Builds CREATE and MIGRATION script based off the latest of that modules branch/trunk

Database Restore Utilities (QRDB/PRDB)

Stands up a Production/QA database backup to a database server. Triggered by forcing the build and selecting various parameters.

Documentation (DOC) Runs like a DSM except it does not publish built artifacts. Builds documentation using Sandcastle and publishes the documentation as defined in the various projects.

Sonar (SONAR) A suite of tests are run under the SONAR framework and results are published for subsequent analysis. Triggered on a nightly schedule but can also be forced.

Database Migrations

Built by CSG years ago• Similar in functionality to Database Projects or RoundhousE

Databases are versioned alongside code Standard folder layout (DbGhost) SQL Server Management Studio plugins Build to Build migrations (CSDB)

• Generates CREATE.sql and MIGRATION.sql• Validate consistency (RedGate SQL Compare)

CSGExecuteMigration• Inspects Version table• Applies subsequent migrations

Database Migrations

Change Scripts go hereCreate Scripts go here

SQL Server Management Studio

Database Migrations

Continuous Integration

Code Reviews

Crucible & Fisheye - Code Reviews

Web-based built by Atlassian On Premise or Hosted Source code browser - Fisheye Source code search engine – Fisheye Code review tool – Crucible Integrates with SVN and a few other source code control

systems

Code Reviews

We made it!

Questions?

Extras

BMC BladeLogic – Automated Deployments Atlassian Confluence – wiki

• Designs WiX - Installers Configuration Migrations – CSG built (XSLT) Build History website – CSG built Sonar – Build Analysis Git/Stash/SourceTree – Distributed Version Control

• SubGit syncs with Subversion

Thanks!

Trunk Based Development

http://paulhammant.com/2013/04/05/what-is-trunk-based-development/

Life Outside of TFS

Technology