Software Development Best Practices

Kian-Tat LimLSST DM System Architect

LSST All Hands MeetingAugust 13 - 17, 2012

Session Goals

– Collect information to standardize processes and tools across LSST development

– Provide practical information on processes and tools to developers

2

Session Outline

– How Development Works at Google (Rob Pike, 10 min)

– Software Standards Working Group Progress (KTL, 10 min)

– Common Libraries (AJC, 10 min)– Using git for Version Control (MJ, 15 min)– Code Development Best Practices (KTL, 15 min)– Using gdb for Debugging C++/Python Code (RHL,

15 min)– Using valgrind for Profiling (and Memory Checking)

(KTL, 15 min)

3

Software Standards Working Group Progress

Kian-Tat LimLSST DM System Architect

LSST All Hands MeetingAugust 13 - 17, 2012

5

Membership

Subsystems unified under System Engineering:– K-T Lim (lead)– Andy Connolly– Francisco Delgado– Gregory Dubois-Felsmann– Stuart Marshall– Jon Thaler

Charter

– Evaluate and prioritize topics for standardization– Distinguish between “design” and “process”

standards– Prepare a plan to implement the highest priority

standards

6

General Progress

– Identified trade-offs, including:✓Increased sharing✓Economies of scale×Additional support, training costs= Slows work?

– Decided some topics should have “standards for standards”• Subsystem-wide standard must meet

specifications– “Best practices repository”

7

Specific Topics

Version Control System– Currently git, svn, cvs– Highly desirable to share a system– DAQ may have different requirementsBuild Tools– Must have documentation on how to buildRelease Packaging– Common standard for release namingData Retention and Archiving– Standards for project-wide data– Metadata, migration, maintaining access, formats

8

Languages and Coding

Languages– Standardize something like current list• C, C++, Java, Python, sh, LabView, PHP, Lua

– Resist addition of new languages for main development

– Stricter standard for web development– Stricter standard for control scriptingCoding Standards– Design (not coding) standards for shared code

9

Development Best Practices

Kian-Tat LimLSST DM System Architect

LSST All Hands MeetingAugust 13 - 17, 2012

Best Practices Repository

– Standards Working Group will collect and disseminate best practices

– These are just some examples from DM– Your contributions are needed!

11

Version Control

– Check in early and often– Use branches to isolate development

12

Continuous Integration

– Keep at least one buildable version at all times– Automated build and test system– Unit tests– Integration tests– Test coverage

13

Design Reviews

Goals:– Ensure design is appropriate, implementable,

testable, and documentedPrepare:– Use cases and requirements– Component and interface descriptions– Technologies/existing code (re-)used– How function and performance will be tested

14

Code Reviews

Primary Goals:– Ensure code implements design– Ensure code is maintainableSecondary Goal:– Spread knowledge of code, algorithms, best

practices throughout team

15

Release Process

– Branch for release-specific bug fixes– Tag in version control system for easy reference– Version numbering

16

Package Management

– Packages at clean interfaces– Packages for developer independence– Record all package versions used during execution

17

Introduction to valgrind

Kian-Tat LimLSST DM System Architect

LSST All Hands MeetingAugust 13 - 17, 2012

valgrind

General:– Pronounced with short “i” (entrance to Valhalla)– http://valgrind.org/Principle of Operation:– Virtual machine plus just-in-time compiler– Essentially simulates running code with

instrumentation

19

valgrind Tools

Memcheck– Default tool– Checks for memory leaks and inappropriate usageCallgrind– Profiler

20