Date post: | 18-May-2015 |
Category: |
Technology |
Upload: | aleksandr-tsertkov |
View: | 3,704 times |
Download: | 2 times |
Continuous Integration and Engineering Environment approach
Aleksandr Tsertkov
CONTINUOUS INTEGRATIONContinuous Integration provides quick feedback on recent code changes
Continuous Integration
• Software engineering practice, where project is build frequently
• Emerged in the Extreme Programming (XP) community
Continuous Integration
• Build is made by automated build tools like: make, Ant, Maven, etc.
• Automated tests results and software metrics are collected for each build
• Build process is executed and results published on CI server
Build
CI server executes build scripts regularly:
– Predefined delay is used to execute build scripts. Every one hour for example
– CI server regularly checks SCM system and in case of changes made since last build time a new build is made
Build status
• Each build has a status: successful or failed
• Build is considered as successful if all build tasks were executed without errors and failed otherwise
• Automated tests are executed as build tasks
Build status
• Build status may be reported by email, IM, SMS, etc.
Build artifacts
• Each build receives unique number. Build artifacts are published with this number.
Build artifacts
• Build artifacts are different from project to project but usually they include:
– Binaries: installation package, application files, …– Generated documentation: API, end-user
documentation, …– Unit test results: time taken, code coverage, …– Software metrics: LOC changes, PMD, CRAP index,
coding standards violations, …
Binary packages
Binary packages produced by build process may include:
– Installation packages– Executable binaries– Etc.
Documentation
• Usually API documentation is compiled during build and is published within other artifacts
• End-user documentation also may be compiled and published accordingly to project needs
API Documentation
Code coverage
• Code coverage used to show which lines of source code has been tested with unit tests
• 100% code coverage is a required but not a sufficient criteria for measuring test quality
Code coverage
Code coverage
Coding standards violations
• Strict coding standards in a project help to improve reading and understanding of code and raise maintainability
• With help of specialized tools coding standards may be checked as part of build process
Software metrics
A software metric is a measure of some property of a piece of software or its specifications.
– LOC - Lines Of Code– PMD – Project Mess Detector– C.R.A.P. – Change Risk Analysis and Predictions– Code coverage
Advantages
• Early notification about integration errors• Early warning about broken code• Immediate unit testing of all changes• Constant availability of build packages
including: testing, demo, beta, etc.• Collected software metrics focus developers
on producing high quality code
Software
• A wide range of CI software is available providing different notification medias, SCM systems, build tools support and IDE integration:
– CruiseControl– Bamboo– Hudson– BuildBot– Etc.
CruiseControl
CruiseControl is an extensible framework for creating continuous build process:
– It includes dozens of plugins for a variety of source controls, build technologies, and notifications schemes including email and instant messaging
– A web interface to view details and access artifacts of builds
CruiseControl
CruiseControl is written in Java but is used on a wide variety of projects thanks to different build tools supported:
– Ant, NAnt, Maven, Phing, Rake, Xcode– “exec” builder which can be used with any
command-line tool or script
CruiseControl
• May be used with C/C++ since build tasks can be wrapped into Ant tasks
• CppUnit or CxxTest can be used as unit testing framework
Generic Visual C++.NET Ant target
CruiseControl
phpUnderControl
phpUnderControl is customization of CruiseControl that brings functionality needed for PHP projects:
– PHPUnit– PHPDocumentor– PHP_CodeSniffer– …
phpUnderControl
phpUnderControl
phpUnderControl
phpUnderControl
phpUnderControl
PEER CODE REVIEWImproves code quality and aids software developers professional growth
Code review
• Code review is a process of checking programming code for common mistakes, vulnerabilities, bugs and errors.
• Code might be reviewed by author, team members or 3rd party company providing code review service.
Code review
Several common types of code review:– Formal inspection – code is reviewed on a
projector by a team of reviewers– Over-the-shoulder review where author walks the
reviewer though a set of code changes– E-mail pass-around – whole files or changes are
packed up by the author and sent to reviewers via email
– Tool assisted – specialized code review tools are used
Peer code review
• Systematic examination of code individually by each reviewer.
• Usually developers are able to review each other code in a small team, but it’s better to set rules for handling review process roles.
Peer code review best practices
• Review fewer than 200-400 lines of code at a time
• Take enough time for a proper, slow review, but not more than 60-90 minutes
• Verify that defects are actually fixed
Advantages
• Helps to identify bugs early• Encourages collaboration and builds a team• Helps in keeping code maintainable• Improves code quality• Improves cross team know-how• Shares experience within a team• Improves developers professional skills
Peer code review software
• Reviewboardhttp://code.google.com/p/reviewboard/
• Cruciblehttp://www.atlassian.com/software/crucible/
• CodeCollaborator http://smartbear.com/
Reviewboard
CodeCollaborator
Crucible
• Web based peer code review tool• Simplified workflow• Integrates with Fisheye and JIRA since it is
ATLASSIAN’s product• Pre and post commit reviews are supported• Integration into Eclipse & IntelliJ IDEA
Crucible
To start a new review Crucible allows:
– Selecting concrete revision, for example the most recent commit in Subversion or revision identified by number or date
– Selecting individual files and folders (even from different revisions)
Crucible
Supports email notification of:
– Review requests– Review comments– Review actions (close, etc.)
Crucible
Crucible
ENGINEERING ENVIRONMENT
Engineering environment
Our engineering environment approach was specially designed for web development though some subsystems may be utilized for non-web development as well.
Environment is based on dedicated server where project code is developed and tested – remote environment.
Environment components
• CI: CruiseControl + phpUnderControl• Peer code review: Crucible• SCM system: Subversion• Development web servers• Database, memcached, etc.• Development tools
PhpMyAdmin, PHPUnit, PHPDocumentor
Environment components
• Each component is highly tuned to simplify it’s use and speed up development
• Subversion hooks are used to apply validation rules on committed code and send on-commit notifications to the team
• Web server components are configured accordingly to project requirements
Remote environment
• Code is developed, tested and stored on a remote dedicated server
• Each developer has isolated environment where he has good control over his vhosts configuration
Key points
• Environment similar to production (server OS, cron jobs, tools and software)
• Abstracts developers from occasional need in environment reconfiguration since this is made only once by server admin
• Possibility to send links to development/current/demonstration version of project
Key points
• Build automation, continuous integration
• Engineering tools suite: debugging tools, common frameworks and libraries, unit testing suite, build tools, database management tools
• Centralized backup
Subversion
Subversion is used as Version Control System and provides following benefits:
– IDE integration– Access over WebDav (web browser)– Hooks system
Subversion over WebDav
Subversion Hooks
• Subversion hooks system allows to execute custom program on some repository event.
• With post-commit hooks we send on-commit emails notifying team about changed made in the repository.
• Pre-commit hooks are used to validate coding standards and apply other rules.
Subversion on-commit email
Subversion pre-commit script
Remote development
• Every developer has a separate account on development server with access over SSH
• Remote files might be accessed via:– SFTP/SCP– FTP/FTPS– Samba (Windows network share)
Samba share mounted
Private namespace (subdomains)
• Every developer has full control over subdomains of USERNAMENAME.dev.gface.de
• Subfolders in ~/vhosts/are automatically mapped to domain names:
Document root Domain name~USERNAME/vhosts/PREFIX.htdocs/~smith/vhosts/htdocs/ ~smith/vhosts/cms.htdocs/~smith/vhosts/v2.cms.htdocs/
PREFIX.USERNAME.dev.example.comsmith.dev.example.com cms.smith.dev.example.com v2.cms.smith.dev.example.com
Development web servers
• Several development web servers with different configurations and- or modules may run on dev server.
• With help of reverse proxy it is possible to switch between servers by simply prefixing domain name with server identificator.
Development web servers
Development web servers
Three web servers are running on one machine with different versions of PHP installed
Hostname Proxied tosmith.dev.example.comphp52.smith.dev.example.comphp53.smith.dev.example.comphp6.smith.dev.example.com
smith.dev.example.com:8081smith.dev.example.com:8081smith.dev.example.com:8082smith.dev.example.com:8083
Web server Port Domain name prefixPHP-5.2 (default)PHP-5.3PHP-6
808180828083
php52.php53.php6.
Questions?
Thank You!