31
1v2 LHC Availability Tracking: Past and Future B. Todd, A. Apollonio, L. Ponce on behalf of the LHC AWG.
| Date post: | 19-Jan-2018 |
| Category: |
Documents |
| Upload: | brent-lawson |
| View: | 216 times |
| Download: | 0 times |
1v2
LHC Availability Tracking:Past and Future
B. Todd, A. Apollonio, L. Ponce on behalf of the LHC AWG.
CERN
[email protected] Dependability Workshop 2013 2
1. LHC Physics in Context of Availability
Key Performance IndicatorsAvailability vs Operation
2. Tracking Availability: 2010-2012
2012 experience, limitationsSystem survey results
3. Tracking Availability: post LS1
Outline
LHC System Availability Tracker (LSAT) RequirementsImplementation
4. The Cardiogram: August 2012
Reverse engineering
5. The future
Milestones and TimelineIntegration with Maintenance Management
CERN
[email protected] Dependability Workshop 2013
Context of Availability
4
more time in physics
less
more
more time in physics
higher physics performance
higher
lower
more time in physics
higher physics performance
shorterturnaround between
physics fills
longer
shorter
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
less
more
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
integrated luminosity is a function of…
1. time colliding physics beams2. turnaround between successive physics beams
3. time to clear faults4. physics performance during colliding beams
machine understanding& operator skill
availability!
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
Integrated luminosity
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
Integrated luminosity
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
Integrated luminosity
more time in physics
higher physics performance
less time clearing faults
shorterturnaround between
physics fills
machine understanding& operator skill
availability
Integrated luminosity
These are not independent variables
CERN
[email protected] Dependability Workshop 2013
Context of Availability
5
1. time colliding physics beams2. turnaround between successive physics beams
3. time to clear faults4. physics performance during colliding beams
availability!
the Availability Working Group has been concentrating on understanding these relationships
Studied 2010-2012 availabilityStudied protection system dependabilityAnalysed methods currently employed
Proposes a future direction
CERN
[email protected] Dependability Workshop 2013
The Survey
7
Q1: How systems find out that something is faulty?
Q2: How do systems keep track of what is faulty and needs repairing?
Q3: root cause / failure mode of every fault / event identified?
Three key questions:
CERN
[email protected] Dependability Workshop 2013
The Survey – Typical Results
8
… there are more rows
CERN
[email protected] Dependability Workshop 2013 9
Fault / Event Information Flow
Operations
operators
Equipment
piquet &
experts
Operations
operators
Equipment
piquet &
experts
Operations
operators
Equipment
piquet &
experts
interface eLogbook
Operations
operators
Equipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
Equipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
fault informationEquipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
fault information
JIRA issueseLogbook
Webpagesconverter event logOneNoteexcel
Equipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
fault informationEquipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
fault informationspreadsheet
Equipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
fault information
Post-mortemTIMBER
spreadsheet
Equipment
equipment tool
piquet &
experts
interface eLogbook
Operations
operators
fault information
Post-mortemTIMBER
reports & views
spreadsheet
for each equipment group
CERN
[email protected] Dependability Workshop 2013 12
Fault / Event Information Flow
Operations
operators
Equipment
Operations
piquet &
experts
operators
Equipment
Operations
piquet &
experts
operators
Equipment
Operations
piquet &
experts
operators
eLogbookinterface
fault / event entry
Equipment
Operations
piquet &
experts
operators
operationinterface
fault information common format
eLogbook
CERN
[email protected] Dependability Workshop 2013 13
fault / event entry
Equipment
Operations
piquet &
experts
operators
operationinterface
fault information common format
eLogbook
Fault / Event Information Flow
CERN
[email protected] Dependability Workshop 2013 14
Fault / Event Information Flow
fault / event entry
Equipment
Operations
piquet &
experts
operators
operationinterface
fault information common format
eLogbook
fault / event entry
Equipment
Operations
piquet &
experts
operators
eLogbookinterface
fault information common format
eLogbook
fault / event entry
Equipment
Operations
interpretationpiquet &
experts
operators
eLogbookinterface
fault information common format
eLogbook
fault / event entry
Equipment
Operations
interpretationpiquet &
experts
operators
eLogbookinterface
fault information common format
eLogbook
fault / event entry
Equipment
Operations
interpretationpiquet &
experts
operators
eLogbookinterface
LSAT tool
fault information common format
eLogbook
fault / event entry
Equipment
Operations
interpretationpiquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
fault information common format
eLogbook
fault / event entry
Equipment
Operations
interpretationpiquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
fault information common format
eLogbook
CERN
[email protected] Dependability Workshop 2013 15
fault / event entry
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
eLogbook
Fault / Event Information Flow
2. Information Interpretation
1. Common Data Format
3. Viewing and Reporting
CERN
[email protected] Dependability Workshop 2013 16
1. Common Data Format
a common set of fields to describe faults / events:
CERN
[email protected] Dependability Workshop 2013 17
2. Information Interpretation
Interpreters to convert information from equipment logs to the common format
see how we go…
eLogbook – change is minimalSmall – moved by hand
Logged Variables– use them as a starting point
JIRA, Power Converter Logsto be investigated…
CERN
[email protected] Dependability Workshop 2013 18
3. Viewing and Reporting • Outstanding Faults ListAll faults which are active at a requested moment in time
• Availability Trackingsystem / sub-system availability shown across a defined time frame
• Availability MatricesFor systems completing “failure-mode” the availability matrices drawn on the fly
• CardiogramThe most interesting view…
CERN
[email protected] Dependability Workshop 2013 20
Cardiogram – August 2012
energy
beam-1 intensitybeam-2 intensity
stable beams = producing physicspost-mortem = beam dump
faults recorded
CERN
[email protected] Dependability Workshop 2013 21
Cardiogram – August 2012
energy
beam-1 intensitybeam-2 intensity
stable beams = producing physicspost-mortem = beam dump
faults recorded
CERN
[email protected] Dependability Workshop 2013 22
Cardiogram – August 2012
The whole raw cardiogram for August 2012 is in the indico folder with this presentationA modified cardiogram showing some consolidated information is also found there
CERN
[email protected] Dependability Workshop 2013
Esystem
event
Basic Event
systema
systemb
child
parent
Parent / ChildDependencies
system
sub-system System à Sub-System
system
fault postponed
Postponed Repairs
systema
systemb
blocker
Blocking Dependencies
Information to be Displayed in the Fault Area:
system
start timeend time
Fault
Basic Fault / Event why physics stopped
CERN
[email protected] Dependability Workshop 2013
Q. What will a “Maintenance Tracker” cover?
585 dumps
A. machine events that need maintenance / repair to continue operation…
≈50%
CERN
[email protected] Dependability Workshop 2013
Q. What will a “Maintenance Tracker” miss?
A. machine events that do not need maintenance / repair to continue operation…
Operator = End of Fill
Feedback events
Beam Loss / UFO events
Pre-cycles
Transfer Line Steering
Waiting for handshakes
The maintenance / asset management tool alone will not provide the information needed.
CERN
[email protected] Dependability Workshop 2013 27
2014: eLogbook – additional fieldsInterpreter feasibility and first implementationViewer – first implementation
Reverse engineering more data from 2012…
Milestones and Timeline
v1 Prototype ready for LHC restart
interaction with MMP to identify common areas
CERN
[email protected] Dependability Workshop 2013 28
Milestones and Timeline
2015:Every week – manual corroboration of data = operators + expertsEvery Technical Stop – production of availability snapshot…Evian 2015 – review of whole year’s data
Online tweaking of v1 prototype = v2… v3…Determine “usefulness”
CERN
[email protected] Dependability Workshop 2013 29
Milestones and TimelineLonger term:
Regular interaction with MMP whilst we learn
fault / event entry
Equipment
Operations
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
fault / event entry
Equipment
Operations
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
fault / event entry
Asset and Maintenance Management
Operational Issues Management
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
Availability Working Group
fault / event entry
Asset and Maintenance Management
Operational Issues Management
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
Availability Working Group
fault / event entry
Asset and Maintenance Management
Operational Issues Management
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
Availability Working Group
fault / event entry
Asset and Maintenance Management
Operational Issues Management
piquet &
experts
operators
eLogbookinterface
LSAT tool
Post-mortem TIMBER
reports & views
Availability Working Group
LHC System Availability Tracker
CERN
[email protected] Dependability Workshop 2013 30
to optimise time in physics and faults we must track faults coherently
inverse femtobarn is a performance metric of LHC…
… a function of time in physics, faults, beam performance and turnaround time& machine understanding
Conclusions
• LHC System Availability Tracker based on the eLogbook… developed in 2014
• prototype with manual interaction… used in 2015
• If LSAT worthwhile integrate into other asset / maintenance tracker tools… 2015+
AWG aims at providing a tool, gathering and generating required information for availability tracking. Information gathered will be used to generate reports and views to assess LHC availability in the post LS1 era.
AWG will collaborate with MMP on defining and exploiting tools and needs required for effective tracking of availability and measuring effects on physics performance of the LHC.
