Date post: | 18-Jan-2018 |
Category: |
Documents |
Upload: | diana-harrison |
View: | 218 times |
Download: | 0 times |
D4Science and ETICS
Building and Testing gCube and gCore
Pedro AndradeCERN
EGEE’08 Conference
25 September 2008Istanbul (Turkey)
www.d4science.eu
2
www.d4science.eu
D4Science
D4Science is a FP7 project establishing a knowledge infrastructure of data, services and processing resources through the provision of Virtual Research Environments
D4Science simplifies the discovery, access and production of existing and new data through a collaborative environment
D4Science continues and consolidates the work started in the DILIGENT project opening it to new communities
D4Science provides a production environment for ESA & FAO
D4Science and ETICS25 September 2008, Istanbul (Turkey)
3
www.d4science.eu
gCube
gCube is the software implementing the D4Science resources sharing vision via Virtual Research Environments
gCube software: Provides a multi-domain Service Oriented Infrastructure Gathers data, services and hardware as system resources Supports on-demand virtual collaborations Reduces the cost of managing research infrastructures Tailored for e-Science (content oriented, data intensive) Is complaint with consolidated and emerging standards
D4Science and ETICS25 September 2008, Istanbul (Turkey)
4
www.d4science.eu
gCube
gCube resources: Computing and storage VRE services
content management search and index annotations
Collections raw data content metadata
Processes to manage other resources
D4Science and ETICS25 September 2008, Istanbul (Turkey)
5
www.d4science.eu
gCore
gCore is a framework developed to reduce the complexity of designing and implementing gCube SOI complaint servicesgCore is composed by: gCore Framework: An application framework to consolidate
and develop existing/new gCube services gCore Container: A self-contained software container
distributed to facilitate the deployment of gCube services
D4Science and ETICS25 September 2008, Istanbul (Turkey)
6
www.d4science.eu
D4Science in ETICS
Two projects: org.gcube: 24 subsystems org.gcore: 2 subsystems
225 components: 160 software components
51 services, 84 libraries, and 25 portlets Including services samples, external libs, test libs (18 components)
65 distribution components 64 gCube Software Archives gCore Hosting Node
Around 500 configurations
D4Science and ETICS25 September 2008, Istanbul (Turkey)
7
www.d4science.eu
D4Science Release Cycle
D4Science and ETICS25 September 2008, Istanbul (Turkey)
etics component configurations
etics subsystem configurations
etics project configurations
etics-build
etics-test
8
www.d4science.eu
Build
The objective of the build activity is to ensure api-level compliance of components and the creation of components’ artifacts.
ETICS is exploited to: Model build scripts and manage build dependencies Execute on-demand remote builds (via WA and CLI) Execute nightly local builds for development configurations Produce distribution artifacts (via dedicated components) Feed the D4Science reporting application with xml raw
reports
D4Science and ETICS25 September 2008, Istanbul (Turkey)
9
www.d4science.eu
Deployment Testing
The objective of the deployment testing activity is to ensure that the gCube software archives: Can be remotely and automatically deployed Once deployed, that it starts and is activated correctly
ETICS is exploited to: Manage the executing the deployment tests Define the deployment test tool dependencies Generate deployment test reports
D4Science and ETICS25 September 2008, Istanbul (Turkey)
10
www.d4science.eu
Deployment Test
D4Science and ETICS25 September 2008, Istanbul (Turkey)
ETICS Server
Condor Node
Condor Node
Condor Node
Condor Node
Condor Node
Condor Node
Service 2 Service 1Service A
Service 2 Service 1Service A
11
www.d4science.eu
Static Analyzers
The objective of the adoption of static analyzers is to improve the quality of gCore and gCube software
ETICS is planned to be exploited to: Run jFindbugs and jDepend (under adoption)
Other analyzers (not exploited via ETICS): Check coverage of javadoc documentation (checkstyle) Monitor the content of code repository
e.g. pre-compiled artifacts, etc.. Certification of distribution components
check structure, correctness, etc..
D4Science and ETICS25 September 2008, Istanbul (Turkey)
12
www.d4science.eu
Current Releases
D4Science and ETICS25 September 2008, Istanbul (Turkey)
13
www.d4science.eu
Current Releases
D4Science and ETICS25 September 2008, Istanbul (Turkey)
14
www.d4science.eu
Improvements
Parametric tests on remote infrastructure Single remote test with multiple parameterized tests
Co-scheduling mechanism to be further exploited Condor nodes firewall limitations
Notification mechanism ETICS is designed to support distributed teams No notifications between different ETICS roles
components availability successful build/test execution
D4Science and ETICS25 September 2008, Istanbul (Turkey)
15
www.d4science.eu
Improvements
ETICS configurations contains many information Can an ETICS API be useful ?
Reports summary information Summary page with built, test, and plug-ins execution Organized by configuration
Plug-ins availability Info about existing ones? How to add external plug-ins?
D4Science and ETICS25 September 2008, Istanbul (Turkey)
16
www.d4science.eu
Conclusions
Solid exploitation: DILIGENT/D4Science using ETICS since the early days ETICS is the default tool in D4Science build an test tasks ETICS completely adopted by all D4Science members
Solid collaboration: D4Science and ETICS plans kept aligned Efficient support from ETICS
D4Science and ETICS25 September 2008, Istanbul (Turkey)