Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
CERN’s Project Management PlatformCERN’s Project Management Platform
Jurgen De Jonghe CERN / IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Projects, Timeline
LHC 3.3 BCHF expenditure12 years
ATLAS Detector 500 MCHF CORE Cost,150 institutes in 35 countries
CERN wide strategic planning
1999
2002
2004
2005
2003
CNGS 75 MCHF expenditureCERN & Gran Sasso
EGEE 50 MCHF,70 partners 800 project members
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Concepts
• WBS: Work Breakdown Structure, hierarchical list of all the activities to be performed to complete the project.
• Workunits: The “Leaves” of the WBS tree. Activity small in size/duration, under responsibility of one person.– use Resources (money, people) – to produce Deliverables
WorkUnits
ResourcesDeliverables* *
WBS
*
*
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Agenda
• Intro: CERN’s Project Management Platform
• In-Depth 1: DB infrastructure to track project evolution
• In-Depth 2: Excel Upload in a JEE environment
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Change Management
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Tracking: what is the aim ?
• Answer questions– What was changed by whom and when?– What are the emerging project trends?– Who is currently using the system?– How did the application’s data look in the past?
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Tracking: requirements
• Easy to use, transparent• Robust• Performant
• Support Workflow: approval before acceptance of new data
• Collaborative: inform project stakeholders of changes
• Storage overhead (less important)
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
<?xml version="1.0"?><!DOCTYPE database SYSTEM "http://jakarta.apache.org/turbine/dtd/database.dtd"><database name="PPT_EVM" tableTableSpace="tabppt" indexTableSpace="idxppt">
<table name="DOMAIN_LISTS" javaName="DomainLists“ tracking=“true” audit=“true”>
<storage initial="50K" next="100K" maxExtents="20"/>
<column name="ID" javaName="Id" primaryKey="true" required="true" size="6" type="DECIMAL" sequenceName="S_DOMLIST_ID“/> <column name="CODE" javaName="Code" required="true" size="12" type="VARCHAR“/> <column name="DESCR" javaName="Description" size="60" type="VARCHAR“/>
… <foreign-key foreignTable="DOMAIN_LISTS"> <reference foreign="ID" local="PARENT_ID"/> </foreign-key>
</table></database>
XML File for Database Description
Triggers & Stored Procedures
Tables Views
• Jakarta ‘torque’ + ‘velocity’ templates• Scripts for tracking are generated as pl/sql
packages• ANT is used to build
Insert into shop(id, descr) values (1, ‘Shop 1’);
Update shop set descr = ‘Produkt shop’ where id = 1;
Update shop set descr = ‘Product shop’ where id = 1;
10:00
11:30
12:00
User1
User2
User3
10:00 11:30 12:00
Administrative Information ServicesTracking: timelineTracking: timeline
Jurgen De Jonghe CERN/IT-AIS
id descr t_workid t_start t_end t_authstart t_authend t_status t_delete
1 Shop 1 1 10:00 11:30 10:30 12:30 A N
1 Produkt Shop 2 11:30 12:00 R N
1 Product Shop 3 12:00 13:30 12:30 14:00 A N
1 Product Shop 4 13:30 future A Y
Insert into shop(id, descr) values (1, ‘Shop 1’);
Update shop set descr = ‘Produkt shop’ where id = 1;
Update shop set descr = ‘Product shop’ where id = 1;
10:00
11:30
12:00
User1
User2
User3
10:00 11:30 12:00
Delete shop where id = 1;13:30 User4
Authorise;10:30 PowerUser
Reject;11:45 PowerUser
Authorise;12:30 PowerUser
Authorise;14:00 PowerUser
10:30 12:30 13:30 14:00
Administrative Information Services
Tracking: explicit Tracking: explicit authorizationauthorization
Jurgen De Jonghe CERN/IT-AIS
• Each insert, update or delete on tracked object creates a new version (= row) in the database
• Tracking can be with or without authorization– Each version can be in status: new, authorized or rejected
• Two timelines: – First timeline shows time when each version was really created and
shows all user actions, including rejected actions and delete attempts.
– Second timeline shows only authorized versions (as user will normally see it).
Administrative Information ServicesTracking: summary so farTracking: summary so far
Jurgen De Jonghe CERN/IT-AIS
Source Table
SHOP_SRC
Mirror Table
SHOP_MIRIDDESCR
IDDESCR
SHOP
View for last authorised version
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
SELECT
INSERT
UPDATE
DELETE
‘instead of’ trigger
Administrative Information ServicesTracking: how does it work?Tracking: how does it work?
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Transaction tables
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Tracking: Usage Statistics
APT (since 2005)
EGEE (since 2004)
LHC (since 2003)
TRANS_ELEMENTS(thousands)
377 185 1’159
TRANSACTIONS (thousands)
142 109 723
About 1 million transactions for slightly less than 2 million versions
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Agenda
• Intro: CERN’s Project Management Platform
• In-Depth 1: DB infrastructure to track project evolution
• In-Depth 2: Excel Upload in a JEE environment
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel Import/Export
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel upload: Diagram
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel Upload: Why Spring Webflow
• Definition of the flow in one XML file – the decision-states, action-states (executed code) and the view-states (JSPs shown to the end user)
• Possibility to conduct our flow according to the results of the action’s execution (not only in case of error or success)
• Simple integration into our Spring Application Framework
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel Upload: Webflow definition
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel Upload: JAXB config
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
JAXB: Java Architecture for XML Binding
• Easy to use ( SAX, DOM)
• Customizable• Standard• Validation & Round-
tripping
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Q & A
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Backup Slides to follow
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Source Table
Versions View
IDID_PREVID_CHANGED…VERSIONIS_CURRENT_VERSION
SHOP_CHANGES SHOP_SRC
Mirror Table
SHOP_MIRIDDESCR
IDDESCR
SHOP
View for last authorised version
IDDESCR
SHOP_ALL
View for last version
SHOP_PAST
‘Back to the past’ view
IDDESCR
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
Administrative Information Services
Tracking: DB Implementation
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
10:00 11:3012:0010:30 13:3014:00
User1 10:00 10:30
Insert into shop(id, descr) values (1, ‘Shop 1’);
User2 11:30 11:45
Update shop set descr = ‘Produkt shop’ where id = 1;
User3 12:00 12:30
Update shop set descr = ‘Product shop’ where id = 1;
User4 13:30 14:00
Delete shop where id = 1;
SELECT * FROM SHOP_ALL WHERE ID = 1;
id
descr
1 Product Shop
12:30
Administrative Information Services
Tracking: All Versions
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Source Table
Versions View
IDID_PREVID_CHANGED…VERSIONIS_CURRENT_VERSION
SHOP_CHANGES SHOP_SRC
Mirror Table
SHOP_MIRIDDESCR
IDDESCR
SHOP
View for last authorised version
IDDESCR
SHOP_ALL
View for last version
SHOP_PAST
‘Back to the past’ view
IDDESCR
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
Administrative Information Services
Tracking: DB Implementation
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
10:00 11:3012:0010:30 13:3014:00
User1 10:00 10:30
Insert into shop(id, descr) values (1, ‘Shop 1’);
User2 11:30 11:45
Update shop set descr = ‘Produkt shop’ where id = 1;
User3 12:00 12:30
Update shop set descr = ‘Product shop’ where id = 1;
User4 13:30 14:00
Delete shop where id = 1;
SELECT * FROM SHOP_CHANGES WHERE ID = 1;
id
id_prev
id_changed
descr descr_prev descr_changed
version
…
1 Y Shop 1 Y 1 …
1 1 N Produkt Shop
Shop 1 Y 2 …
1 1 N Product Shop
Produkt Shop
Y 3 …
1 1 N Product Shop
Product Shop
N 4 …
12:30
Administrative Information Services
Tracking: Changes
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Insert into shop(id, descr) values (1, ‘Shop 1’);
Update shop set descr = ‘Produkt shop’ where id = 1;
Update shop set descr = ‘Product shop’ where id = 1;
10:00
11:30
12:00
User1
User2
User3
10:00 11:3012:00
Delete shop where id = 1;13:30 User4
13:30
Administrative Information Services
Tracking: implicit authorization
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
• Transactions group user actions – E.g., Changes to 5 workunits and their resources
& deliverables
• Transaction can have different transaction types.• Transaction can have comments.
• Each change of tracked table belongs to some transaction.
• Authorised transaction can be rejected later.
Administrative Information Services
Tracking: Transactions
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
• Application is responsible for:– starting and ending transactions– setting the comments – setting type of transaction
• Apart from specific screens (trends, history) no other app.code is aware of tracking.
Administrative Information Services
Tracking: Application
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
• How did the data look one year ago?• There is a possibility to ‘Go back to the past’.
– Just call one function in TRANSACTION package and all _PAST views will return data for the desired date.
Administrative Information Services
Tracking: Going back to the Past
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Similar generated infrastructure:Auditing, Trees, Searching
• Auditing: automatically adds fields– creation date/user– last modification date/user
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Trees: Concepts
• WBS: Work Breakdown Structure, hierarchical list of all the activities to be performed to complete the project.
• Workunits: The “Leaves” of the WBS tree. Activity small in size/duration, under responsibility of one person.
• Workunits use Resources (money, people) to produce Deliverables
• Other secondary breakdown structures will be used in large projects:– People and monetary resources are also
organized in OBS (Organization)– Deliverables are part of the final
product, described in PBS (Product)– Individual parts are assembled
according to an ABS (Assembly)…
WorkUnits
ResourcesDeliverables
* *
WBS
*
*
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Trees: Why
• How to retrieve trees quickly from the database– Oracle “CONNECT BY” not ANSI
standard, slow, sub-query restrictions
– Iterate in Java: slow, not possible as sub-queries
– We want a simple/quick query like:
select code, description from wbs where b_path like ‘21.1.0.0.%’ order by b_orderintree;
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Trees: example WBS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Searching: Simple example
Id First Last Tel Dep Group Building
2777923
Jurgen De Jonghe
71234 IT AIS 513
Search
Jurgen De Jonghe IT-AIS, Tel: 712234, Address:513
Auto-generation of an additional search field:
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Searching: more advanced
WorkUnits
ResourcesDeliverables
* *
19418: Supply & deliver series of thyristor converter : 1st batchHolder: A. BEURET Planned Start: 16-Nov-2004 End:16-Nov-2004Actual Start: 30-Nov-2004 End:30-Dec-2004Supplies & procurement (under F contracts only) Amount: 286,765 CHF BudgetCode: 96637 ContractF511/00, FF1008091 (IT-3003) Duty travel Amount: 2,000 CHF BudgetCode: 96637 Travel expenses for visiting the firm All other procurement (under M budget) Amount: 20,000 CHF BudgetCode: 96637 Free issues items
F511 810A,450V] converters - Batch 1prov. Accepted 4 / 4 Unit Eq. Code: RPTF 16-Nov-2004 until 30-Dec-2004
WorkUnit extended with an extra Search Field:
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Source Table
Versions View
IDID_PREVID_CHANGED…VERSIONIS_CURRENT_VERSION
SHOP_CHANGES SHOP_SRC
Mirror Table
SHOP_MIRIDDESCR
IDDESCR
SHOP
View for last authorised version
IDDESCR
SHOP_ALL
View for last version
SHOP_PAST
‘Back to the past’ view
IDDESCR
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
Administrative Information Services
Tracking: DB Implementation
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
10:00 11:3012:0010:30 13:3014:00
User1 10:00 10:30
Insert into shop(id, descr) values (1, ‘Shop 1’);
User2 11:30 11:45
Update shop set descr = ‘Produkt shop’ where id = 1;
User3 12:00 12:30
Update shop set descr = ‘Product shop’ where id = 1;
User4 13:30 14:00
Delete shop where id = 1;
SELECT * FROM SHOP_SRC WHERE ID = 1;
id
descr t_workid
t_start
t_end
t_authstart
t_authend
t_status
t_delete
1 Shop 1 1 10:00
11:30
10:30 12:30 A N
1 Produkt Shop
2 11:30
12:00
R N
1 Product Shop
3 12:00
13:30
12:30 14:00 A N
1 Product Shop
4 13:30
future
A Y
12:30
Administrative Information Services
Tracking: Tab & View Example
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Source Table
Versions View
IDID_PREVID_CHANGED…VERSIONIS_CURRENT_VERSION
SHOP_CHANGES SHOP_SRC
Mirror Table
SHOP_MIRIDDESCR
IDDESCR
SHOP
View for last authorised version
IDDESCR
SHOP_ALL
View for last version
SHOP_PAST
‘Back to the past’ view
IDDESCR
IDDESCRT_PKT_STARTT_ENDT_AUTHSTARTT_AUTHENDT_WORKID
Administrative Information Services
Tracking: DB Implementation
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
10:00 11:3012:0010:30 13:3014:00
User1 10:00 10:30
Insert into shop(id, descr) values (1, ‘Shop 1’);
User2 11:30 11:45
Update shop set descr = ‘Produkt shop’ where id = 1;
User3 12:00 12:30
Update shop set descr = ‘Product shop’ where id = 1;
User4 13:30 14:00
Delete shop where id = 1;
SELECT * FROM SHOP WHERE ID = 1;
id
descr
12:30
Administrative Information Services
Tracking: Normal User View
Jurgen De Jonghe CERN/IT-AIS
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel Upload: Spring intro
• Spring is essentially a technology enabling you to build applications using POJOs.
• Through its bean factory concept, Spring is an Inversion of Control (Dependency Injection) container .
• Hollywood Principle: "Don't call me, I'll call you.“
• Benefits:– components don't need to look up collaborators at runtime,
they're simpler to write and maintain– for the same reasons, application code is much easier to test– most business objects don't depend on IoC container APIs.
This makes it easy to use legacy code... an IoC container isn't invasive
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
Excel Upload: Spring WebFlow
• For building web application modules that guide the user through controlled navigations that drive business processes spanning HTTP requests.
• A generic flow engine based on a finite-state machine for the execution of a web conversation.
• Goals:– Integrates with existing frameworks like Struts, JSF...– Flow definitions should be declarative and understandable by
business analysts (easy like a visual diagram, such as a UML state diagram)
– Flow definitions should be self-contained.• Reusable modules usable standalone or combined in larger, more complex
flows. – Easily testable in isolation.
Administrative Information Services
Jurgen De Jonghe CERN/IT-AIS
PPT: a Project Management Platform
Relational Database
Change ManagementSummary Layer
OLAP Engine
Analysis reports
Ad hoc crosstabs
Collaborative Events & Alerts Bus
Project Management Domain*
*
Configurable Business Rules
User Interface Components
Web Components