WP3 - Software Configuration Tools and Methodologies

www.eu-etics.org

INFSOM-RI-026753

Elisabetta RonchieriElisabetta RonchieriINFN CNAFINFN CNAF

ETICS 2ETICS 2ndnd EU Review (CERN) EU Review (CERN)15 February 200815 February 2008

WP3 - Software Configuration WP3 - Software Configuration Tools and MethodologiesTools and Methodologies

INFSOM-RI-026753 EGEE and gLite are registered trademarks

Contents

ETICS 2nd EU Review - CERN - 15 February 2008 2

• GoalsGoals

• AchievementsAchievements

• ChallengesChallenges

• WP3 in NumbersWP3 in Numbers

• ConclusionsConclusions


Contents


• GoalsGoals






Goals


G1. Provide a configuration, build and integration system able to describe software tasks to be executed by the ETICS facility.

G2. Provide a configuration information schema able to capture the configuration information of software developed by software project and a coherent storage of such information in the ETICS database


Goals


G3. Web Server: the heart of the ETICS system – Provide a logic for the entire ETICS system– Handle access to the ETICS system in secure mode– Handle metadata, artefacts, and user requests– Perform complex queries on behalf of the clients– Handle build/test remotely– Support build/test locally

G4. Command Line Interfaces (CLIs): one of the ETICS clients– Allow users to register their software, specify configuration

information (i.e., edit metadata)– Handle access to Web Service in secure mode– Checkout code and metadata– Build/Test code locally and remotely– Tag code and metadata– Produce reports and different format of artefacts


ETICS CLIs, Build/Test Web Serivce and metadata

ETICS Portal (myETICS, Build/Test, Repository, Administration)

ReportDB

Metronome Execution Engine

Metronome ClientWrapper

WNs ETICS Infrastructure

Repository Service

Build/TestArtefacts

ProjectDB

Clients

Web Service

Via browser


Contents


• GoalsGoals






Achievements

8ETICS 2nd EU Review - CERN - 15 February 2008

A1. Authentication and Authorization• Performed authentication using standard x.509 certificates in Web

Service and CLIs• Registered information of users (e.g., name and distinguish name)

in the ETICS data model • Based authorization on role definition

– i.e., defined roles to descriminate the operations users can perform– e.g., clients connecting without certificate are mapped as Guest

• Allowed or denied operations to the users by Web Service

Role Description

Administrator Super user

Module Administrator Handles individual projects

Developer runs remote builds

Integrator runs remote builds, registers artefacts in the repository

Tester runs remote tests, stores results in the report repository

Release Manager Defines configurations at project level, and project releases

Guest Read-only access


Achievements


A2. Software Metadata definition• Based schema version on Common Information Model (CIM) • Defined formal entities to express project structure,

build/test/checkout configurations, security information, dependencies, environment variables, properties, users, roles– E.g., software project structure and configuration structure

– Project, Subsystem and Component referred as Modules– More than one Configurations is allowed for each Module – More than one Platforms is allowed for each Configuration– Set Hierarchy in Project and Subsystem Configurations

Project Subsystem Component Component ... Subsystem ... Component .....

Configuration Platform 1 VCS Command Build Command Test Command Dependency1 Property1 Environment1 Platform n VCS Command Hierarchy1


Achievements


A3. Build/Test• Handled the downloading of code and metadata information• Supported the build and test locally and remotely• Generated platform specific artefacts (e.g., RPMS, MSI, Debian,

tarballs) during the build of software• Generated build and test reports • Supported the browsing of software information• Supported the editing of software metadata

– e.g., the possibility to create metadata, modify/remove them and copy existing metadata

• Supported co-scheduling by the editing functionality– DTDT {node1, node2, …, nodeN} {node1, node2, …, nodeN}

– Set hierarchy in DT configurationSet hierarchy in DT configuration

– NodeNode {service1, service2, …, serviceN} {service1, service2, …, serviceN}– Set services in Node configurationSet services in Node configuration

• Handled code and metadata tag

Project DestrebutedTest Node Node ... Subsystem ... Component .....


Achievements


A4. Interoperability and Compliance to Standards • Metrics

– Identified four interoperability metrics1. Number of 3rd Party Components (No3C)

2. Number of Generic Components (NoGC)

3. Number of Architectural Revisions (NoAR)

4. Number of Versions (NoV)

– Used configuration information stored in the ETICS database

• Collectors– Provided information collectors for the reporting engine

– Compliance to standards:

Ipv6

WS-compliance– Interoperability

Metrics


Contents


• GoalsGoals






Challenges


C1. C1. Improve the data model in order to guarantee a better Improve the data model in order to guarantee a better response time from the backend response time from the backend– CIM does not scale to large software applicationsCIM does not scale to large software applications

– Started a revision of the data modelStarted a revision of the data model

C2. C2. Improve the CLI performancesImprove the CLI performances– Current version based on CIM and SOAP Current version based on CIM and SOAP – As the size of a project increases, the performance of the client As the size of a project increases, the performance of the client

decreases decreases to handle large XML documentsto handle large XML documents– SubstantiallySubstantially performance improvements by using newer version of the performance improvements by using newer version of the

Python SOAP enginePython SOAP engine (called ZSI) (called ZSI) in handling large XML documentsin handling large XML documents– Started tests in the ETICS clientStarted tests in the ETICS client

C3. C3. Improve the calculation of the configuration propertiesImprove the calculation of the configuration properties– SubstantiallySubstantially performance improvements performance improvements by making better use of by making better use of

cachingcaching


Contents


• GoalsGoals






WP3 in Numbers


DeliverablesDeliverables::

Milestones:Milestones:

Del. N. Del. Name Delivery Date

D3.1 Configuration, build and integration specifications PM03

D3.2 Interoperability reports specifications, configuration, build and integration system PM12

D3.3 Final evaluation report and configuration, build and integration system (final release) PM20

Mil. N. Mil. Description Delivery Date

M3.1 Configuration, build and integration system deployed in the ETICS resource pools (prototype) PM12

M3.2 Configuration, build and integration system deployed in the ETICS resource pools (final release) PM22


WP3 in Numbers


OS Version Architecture Compiler

Fedora Core 3 ia32 gcc 3.4.4

5 ia32 gcc 4.1.1

6 ia32, x86_64 gcc 4.1.1

Scientific Linux 3 ia32, x86_64 gcc 3.2.3

5 Ia32 gcc 4.1.1

Scientific Linux Cern 3 ia32, x86_64 gcc 3.2.3, gcc 3.4.5

4 ia32, ia64, x86_64 gcc 3.4.6

Red Hat Enterprise Linux 3 ia32 gcc 3.2.3

4 ia32, x86_64 gcc [3.4.3, 3.4.6], gcc [3.4.5, 3.4.6]

SUSE 9 ia32, x86_64 gcc [3.3.1, 3.3.5], gcc 3.3.3

10 ia32 gcc [4.0.2, 4.1.0]

Solaris 10 ia32 gcc 3.4.3

Debian 3 ia32, x86_64 gcc 3.3.5

4 ia32, x86_64 gcc 4.1.2

centOS 4 ia32, x86_64 gcc [3.4.4, 3.4.6], gcc 3.4.6

5 ia32, x86_64 gcc 4.1.2, gcc 4.1.1

Darwin/Mac OS 772 powerpc gcc 3.3.3

880 powerpc gcc 4.0.1

891 ia32 gcc 4.0.1

8100 powerpc gcc 4.0.0

ubuntu 5 Ia32 gcc 4.0.2

6 Ia32 gcc 4.0.3

7 ia32 gcc 4.1.2

Win32 71

aix 52 powerpc gcc 3.3.3

MeasurementsMeasurements• Supported Platforms Supported Platforms 4141

– Fedora Core 4Fedora Core 4

– Scientific Linux 3Scientific Linux 3

– Scientific Linux Scientific Linux Cern 5Cern 5

– Red Had Enterprise Red Had Enterprise Linux 5Linux 5

– SUSE 5SUSE 5

– Solaris 1Solaris 1

– Debian 4Debian 4

– centOS 5centOS 5

– Darwin/Mac OS 4Darwin/Mac OS 4

– win32 1win32 1

– aix 1aix 1

– ubuntu 3ubuntu 3


WP3 in Numbers


Measurements:Measurements:– Supported languagesSupported languages 7 7

– C, C++, Python, Java, Perl, sh, csh

– Implemented RequirementsImplemented Requirements 97.56% 97.56% D3.1– Total number of open and closed bugs over timeTotal number of open and closed bugs over time

– Ratio of closed/open bug over timeRatio of closed/open bug over time

020406080

100120140160

nu

mb

er o

f b

ug

s

2006 2007 years

open

closed

Ratio Year

0.90 2006

0.67 2007


Contents


• GoalsGoals






Conclusions


• Most important achievements A1. Authorization and Authentication

A2. Software Metadata Definition

A3. Build/Test

A4. Interoperability and compliance to standards

• Three outstanding challengesC1. Data Model Structure

C2. CLI Performances

C3. CLI Property Calculation

• Planned in ETICS II activities to address the challenges

INFSOM-RI-026753 EGEE and gLite are registered trademarksETICS 2nd EU Review - CERN - 15 February 2008 20

Thanks

http://www.eu-etics.org

Date post:	14-Jan-2016
Category:	Documents
Upload:	vanya
View:	39 times
Download:	0 times

WP3 - Software Configuration Tools and Methodologies

Documents