The common Software The common Software Repository for SL and STRepository for SL and ST
(The SCaM Project)(The SCaM Project)
A. Bragg (SL/CO), E. Hatziangeli (SL/CO), J. Patino (ST/MO)
PlanPlan The project The solution Razor The elements of SCM The Functionality
The system The development environment
The Work Documentation and Training The Benefits
Provide the means for users to develop and exchange software, using a common repository, based on available industrial products
Why the SCaM project?Why the SCaM project?
Replace and extend the functionality of SL/CO software management system (SLAPS)
Give control to the users over their software development process Parallel development Quick fixes by Operators without the need of administrator Allow version and release management for in-house software, as well as external
software integration Improve the maintenance of the Operational Software Unify all diverse versioning systems efforts (SLAPS, SL/BI cvs, ST/MO
sccs, PS/CO rcs and sccs, ...) Create a software management system that is easy to extend and maintain
The people involvedThe people involved
Advisors SL/BI Jannes de Vries ST/MO Roberto Bartolome LHC/VAC Isabelle Laugier PSSL Convergence Project Alessandro Risso
Primary Team SL/CO Andrew Bragg & Eugenia Hatziangeli [pr. leader] ST/MO Jose Patino
Project Sponsors SL/CO/AP Pierre Charrue ST/MO Pierre Ninin
Observers PS/CO Alessandro Risso IT/IPT Arash Khodabandeh (LHC experiments: SPIDER SCM sub-project)
Major Project MilestonesMajor Project Milestones Identify problem and create project SCaM [SL-Note 97-44 (CO), Jun. 97]
Collect User Requirements [SL-Note 97-59 (CO), Oct. 97]
Market survey for industrial and public domain tools=> Evaluation report and recommendations for tool and procedures to SL-CO and ST-MC management [SL-Note 98-022(CO) Mar 98]
Re-commit SL-CO & ST-MC management to solution chosen [SL-Note 98-34 (CO), Jun. 98]
Purchase and installation of the chosen tool (Nov. 98)
Software management system implementation (end Aug. 99)
The solution packageThe solution package
1) Tool => Razor
2) Software Management Procedures => User and administrator
3) Administration and User support for the:Software Management tool and the Repository
Software Management Procedures
New User Training
The team will ensure that the Software Management activities for Operational Software development are carried out as planned.
Why Razor?Why Razor? It demonstrates a good all around functionality
good management of software versions, supports baseline and release management provides problem reporting management
Very adaptable, with GUI and command line interface
It is simple to learn, to use and to maintain
Good level of technical support and constant product evolution
It is commercial and well priced
It is based on floating or site license scheme 14 floating licenses available (28 concurrent developers)
More on Razor ...More on Razor ... Architecture
License Manager
Repository Server
Licenses
Repository Files
Archive Engine
Razor User
Interface
Platforms
• HPUX• Linux• Windows NT/95• AIX (IBM)
• SCO• Solaris 2.4 (SunOS 5.4) or higher• Solaris x86• SunOS 4.1.x
• IRIX (SGI)• OSF (DEC)
UNIX UNIX Win95/NT
Win95/NT UNIX
Win95/NT
Win95/NTUNIX
LAN Connection
WA
N C
onne
c tio
n
License Manager Repository Server
Network
Software Configuration Software Configuration Management ??Management ??
Version Management
Problem Management
Release Management
Software Repository
It is about “what changes” in our development environment, while we are working and programming
Control and track “what changes” during development
Trilogy of RazorTrilogy of Razor
Versions
Issues
Threads
Archives changes and coordinates efforts amongst developers
Manages all files related to
a released product
Records and coordinates things yet to be accomplished
Release Management
Version Management
Repository
Problem Management
Version ManagementVersion Management“….what the hell, let’s call him Ramses, too….”
Ramses the 1st, inventor of the versioning, at the birth of his son, 1784 B.C.
File: hello.c Version 1.0#include <stdio.h>main(void){
printf(“hello world\n”);}
File: hello.c Version 1.1#include <stdio.h>main(void){
printf(“Hello World\n”);}
File: hello.c Version 1.2#include <stdio.h>main(void){ printf(“Hello New World\n”);}
Version 1.0 Version 1.1 Version 1.2
Version 1.2.1.1
Version 1.3
Version 1.2.1.2
Version 1.4
Version 1.5
Release ManagementRelease Management
1.3 1.4 1.5 1.6 1.7
1.2 1.3 1.4 1.5
2.4 2.5 2.6 2.7 2.8
calories.c
meal.data
food.h
get_thin 1.2 get_thin 1.3
• Grouping (threading) software as a collection of specific versions of files is done for:
• release or/and build management
• try out certain combination of files for testing or prototyping
• The exact state of each file, as they relate to a specific named release, can be reproduced
• Each release is given a unique name and evolves over time.
• The meaning and the utility of a release is up to the user
Problem ManagementProblem Management
File: hello.c
Version 1.0
#include <stdio.h>main(void){ printf(“hello world\n”);}
File: hello.c
Version 1.1
#include <stdio.h>main(void){ printf(“Hello World\n”);}
File: hello.c
Version 1.2
#include <stdio.h>main(void){ printf(“Hello New World\n”);}
File VersionsRelease Versions
calories.c 1.4
Release: get_thin
meal.data 2.4
food.h 1.2
Version: 1.2
calories.c 1.6
Release: get_thin
meal.data 2.7
food.h 1.5
Version: 1.3
Change Requests
Bug Reports Software changes, arising from Bug Fixes or Change Request are managed through the software lifecycle
Track the changes by relating the versions of the files/releases with the Problem Reports or the Change Requests
RepositoryRepository“…. you know, I preferred the way you had the pyramid yesterday….”
Ramses the 5th, inventor of the vault, shortly before his entombment, 1637 B.C.
All versions of files
Software Releases
Software Change Requests,Problem Reports
RCS SCCS
All previous development investment in RCS or SCCS can be injected in the new repository
We are independent from the tool
Located on REPSRV
• HPUX 10.20
• 8GB of space
• on regular backup schedule
• 365 days/year available
• on power fail safe
Repository ContentsRepository Contents
• Source files
• Header files
• Data files
• Scripts
• Documentation files
• Help files
• Icons
• Bitmaps
• Other graphics
• Configuration files
• Spreadsheets
• Off the self software
All versions of files
Reproducible code (binaries, output data files,…)
ASCII Files RCS : The last
version is kept along with backward deltas for the older versions
SCCS: The first version is kept along with forward deltas for the newer versions
Binary Files
Stored in compressed format
The set up of the repositoriesThe set up of the repositories
ST-MO projects
SL-CO projects
SL-OP projects
SL-BI projects
SPS2001
PS-SL-conv
…. . . .
Alarm Software LHC/VAC Software
Minimize user inconvenience
User should not need to traverse an enormous tree to have access to his software
Possibility to customize according to the local development philosophy
Repository Login Interface
User Several repositories with software related by a
common theme, set up using standard defaults
ST LHCALARMSSL …...
SL
The set up of each repositoryThe set up of each repository
A software group is a “narrower” collection, contained within a repository
SL-COSL-BI
SL-OP
SPS2001 PSSL-conv
groups
folders
The folders are the simplest collection. They contain files and are a way of grouping related files
The plan…The plan…
The philosophy
Access Control
PEOPLE files
The structure of a project in the repository
File name/extension, Symbolic-Link Control Accessing the right Repository and starting
the GUI’s
The philosophyThe philosophy
Underlying philosophy: Who do I want to have write access to my software?
(individual files and projects)
Myself and my project team Others in my group in case of my absence
(holidays, sickness etc etc…) There is a complete history of repository
activity linking actions to users
“Do unto others as you would have them do unto you” Someone a long time ago
Access ControlAccess Control Applies to both (Unix) Command Line and GUI
PC Windows95/NT Client Transparent to user - No additional passwords Possible to configure, per Repository
per Project
Any action which can affect a file or the attributes associated with a file are under access control
Two levels of validation 1. PEOPLE file1. PEOPLE file 2. User’s CERN Division/Group2. User’s CERN Division/Group
PEOPLE file?PEOPLE file?
Reflects the project team / collaborations
A PEOPLE file is a list of valid UNIX user id’s
Separated by ‘white space’
Allow write access to the structure below
Used to obtain e-mail information for group
validation
Maintained by the project team
Scenario.3.
Scenario.2.
Scenario.1.
The file PEOPLE contains the group of people that they have write access under that structure. The lowest level PEOPLE file has precedence, over the upper one, for the directory structure below.
Any connection to an existing or proposed Division/Project is entirely co-incidental.The names have been changed to protect the innocent.
SPS2001ST-MO
cryo tds
PEOPLE inc lib src
PEOPLE inc lib src
PEOPLE
SL-CO
APWS
Repository
bctmeaslib
PEOPLE inc lib src
File/Extension, File/Extension, Symbolic-Link ControlSymbolic-Link Control
Only valid file extensions may be introduced, e.g. *.c, *.java, *.doc
Certain file’s without extensions may be introduced, e.g. PEOPLE, Makefile, readme
Acceptable extensions are defined per software group Why? - Avoid unwanted files e.g. core
New file(s) / extension(s) are entered by the developers
Symbolic links are NOT allowed!
Accessing the right Repository Accessing the right Repository and starting the GUI’sand starting the GUI’s
From a Unix environment - GUI’s
Logging into the repository
Starting the Razor Suit
Accessing the right Repository Accessing the right Repository and starting the GUI’sand starting the GUI’s
(Continued…)(Continued…) From a PC windows95/NT client environment
26
Problem ReportingProblem Reporting(Issues)(Issues)
PlanPlan
What is the ISSUES program
Operations on a Problem Report
Problem Report lifecycle
Attributes of a Problem Report
What is the ISSUES What is the ISSUES program?program?
Main list: Displays each problem report or change request made within a software domain
The main display:
It is the Problem Tracking System part of the Razor trilogy
Operations on a Problem Operations on a Problem Report Report
View/modify problems reports Delete problem reports Print problem reports Relate problems reports to an activity in either
the file version control or release management aspect of Razor
Make filtered selections on all the archived reports
Generate reports
GUIEmail templatesWeb
Create new problems reports
Problem Report lifecycleProblem Report lifecycle
The possible STATES and TRANSITIONS of an issue are:
(e-mail notifications on state changes)
Submitted
Closed
Assigned
Feedback Suspended
RejectedAccepted
Attributes of a Problem Attributes of a Problem Report (I)Report (I)
Common Attributes
Attributes of a Problem Attributes of a Problem Report (II)Report (II)
Category
Assigned person
State
Attributes of a Problem Attributes of a Problem Report (III)Report (III)
Two text panes Problem Description (Originator) Actions Taken (Assigned Person)
User Development Area
Che
ck o
ut
Ch
eck
in
Cre
ate/
build
/test
a r
elea
se
Repository
Public Software Area
(For compilation and linking of user software)Libraries: SL_EQUIP, SL_RPC, SL_MEAS,SL_UMMI,
…
Includes: nc.h, Mequip.h, sl_measlib.h,
binaries:
Configuration:.Xdefaults, etc.
Installation procedure
Operational Software Area
(Used to operate the machines)Products:tz_drive, logbook, shiftlog,
…
Complete releases with sources
Inst
alla
tion
pro
ced
ure
Operational Software InstallationOperational Software Installation
Install software release
Install public software
Public SW Area
lib include bin
Build Support EnvironmentBuild Support Environment
Support for csh environment Support the existing SLAPS environment
$SL_INCLUDE, $SL_LIB, $SL_BIN,... $SL_RPC, $SL_EQUIP,…
Support for sh like environment
General Makefile (new projects)
Nex
t
Cur
rent
Pre
viou
s
……
…..
Previous versions of public softwarePrevious versions of public software
Current (default) public softwareCurrent (default) public software
Newly released public softwareNewly released public software
The Operational SoftwareThe Operational Software Operational software will be released in and run from a designated
distribution area
Each release will be complete with sources to allow quick bug fixes
The Operator will be able to quickly revert to previous working version of a product (global for all consoles)
The Operator will be able to access any available version (previous or future) of a product (local to a console)
/user/ops/software/releases/product1 product2 product3 ….. productx
Under each product:
/user/ops/software/releases/product1/v1.0 v1.1 v1.2 PREVIOUS (Previous working) v2.0 v2.2 CURRENT (In operation)
v2.3 NEXT
Roles and CapabilitiesRoles and Capabilities
Developers
introduce new files in his software project
check out their software
check in their software
create/modify releases of their software projects
extract releases of software products from repository
build releases of their software projects
install software releases in the Operational Software Area
receive/create/modify Problem Reports and Change Requests
Librarians
install public software (libraries, includes,…) in Public Software Area
Administrator
manage the tool, the repositories, the software management procedures and new user training
Defined per software group according to group requirementsDefined per software group according to group requirements
Some of the Policies and Procedures
PEOPLE file (Access control to a product in the repository)
Control over the type of files in the repository (File Extension control)
Public software installation via an Automatic Installation Procedure
Operational software installation via an Automatic Installation Procedure
Policies and RecommendationsPolicies and Recommendations
“Write your code for somebody who does not understand it, because by the time you come back to it, that somebody else will be you….”
Some of Recommendations
Files self-identification (RCS keywords)
Relation of file/release versions to PRs or SCRs
Naming scheme of an Operational release
Who does the work?Who does the work? Project team
Tool installation Set up of the software repositories Set up of user development environment Set up of Operation Software distribution area Implementation of automatic installation procedures User training and documentation
Project team + Developers Porting of software project (SLAPS, SL/BI, ST/MO, LHC/VAC, ALARM, …) inside
the repository
Administrator(s) Administration of the tool and management of software repository User support User training
DocumentationDocumentation User Documentation
Administration Documentation
SCaM project web page: http://venice.cern.ch/~slaps/hot/SCAM_II/index.html
Razor documentation available: http://venice.cern.ch/~slaps/hot/razorDocs/rz_web/razortoc.htm
The official Razor internet site:http://www.tower.com
TrainingTraining
We will provide training Users Librarians
Small groups of users - Targeting their specific needs
Lasting no more the 1½ to 2 hours maximum
There will be several training sessions
Future training sessions will be arranged as required
The situation todayThe situation todayOperational Software unable to identify the versions of the majority of the operational software unable to identify/control all elements that influence the operational software unable to control changes of the operational software and identify the initiators not easy to run another version of the operational software
Development Environment public software change often and without warning difficult to debug, since reproduction of the exact version of the software
concerned is often not possible non unique software and hardware configuration of the development platforms
External Software software written by external contractors cannot be properly synchronised with
software written in CERN, because rules to integrate software are not well specified
What will it change?What will it change?
All software from different groups will reside in a single repository
Good working practices and procedures Version management (at least) for existing project in
maintenance phase Software improvements
knowledge of where each software component is and what state is in stability, because changes affecting operational software will be under control useful reports available about the evolution and changes of the software
Centralization of repository system management
Free some resources
Benefits for the Operation teamBenefits for the Operation team
Delivery of consistent operational software
Being able to trace and identify any component of an operational system
Being able to incorporate quick bug fixes in the operational environment
Minimisation of uncontrolled changes of the operational software
Able to run other (future or past) versions of the operational software
Able to “revert to previous working” version of the operational software
Platform independent repository access (HPUX, Win95/NT, AIX, Linux,..)
Benefits for developersBenefits for developers
Version management
Software is kept safely in the repository
“Care free” parallel development
Manage the process used to develop your software
Project lifecycle management
Projects in maintenance New Projects
All Projects
Guidelines and procedures concerning the integration and installation of operational software and the introduction of software modifications
A single software configuration management environment
Transparency of software and documentation
Important for subcontracted software development
Why this presentation?Why this presentation?
Make you aware of this project Receive feedback on our proposed solution Encourage you to try the new system
We would like to produce a Software Management system that works
Iterative process
Help and feedback from the software developers
Thank you for comingThank you for coming
Any questions?