Date post: | 25-Dec-2015 |
Category: |
Documents |
Upload: | wesley-stevens |
View: | 234 times |
Download: | 5 times |
Archiving Workshop (Soleil, May 2010) 1
Archiving System Status
Archiving Workshop (Soleil, May 2010) 2
Plan
Archiving Systems Overview HDB / TDB / SNAP / ADB
Soleil Deployment Status Oracle DB infrastructure description Latest modifications Production Missing features
Archiving Workshop (Soleil, May 2010) 3
Archiving Systems Overview
Archiving Workshop (Soleil, May 2010) 4
HDB/TDB Archiving: General Description
Operators Requirements Storage of a large number of data produced by the Control System Data exploitation at anytime
• Watch the vacuum temperature and the pressure evolution• Look for an explanation of a beam loss
The tools Historical DataBase (HDB)
• Data are kept for lifetime• The minimal archiving period is 10 seconds• On-line extraction available during x years (hardware capacity limitation)
Temporary DataBase (TDB)• Data are kept x days (hardware capacity limitation)• The minimal archiving period is 100 milliseconds
The two are managed via the graphical tool named Mambo
Archiving Workshop (Soleil, May 2010) 5
HDB/TDB Archiving: Mambo
MAMBO : Configuration and Extraction application
Web access also available via JBOSS
Archiving Workshop (Soleil, May 2010) 6
HDB/TDB Archiving: Devices
Devices Hdb/TdbArchiver : Collect of the attributes values and storage in
the DB
ArchivingManager : Upper HDB/TDBArchiver, contains Start/Stop archiving commands
HDB/TDBExtractor : various extraction commands as getAttDataBetweenDate, getArchivingMode …
HDB/TDBArchivingWatcher : Supervision Device
• Diagnosis tool : detecting abnormal archiving interruption
• Recovery : reactivate archiving on the failed attributes
Archiving Workshop (Soleil, May 2010) 7
SNAP Archiving: General Description
Operators Requirements Capability to take a picture of a set of attributes at t time
• Motors positions before a planned electric halt Compare quickly and easily the attributes values
• Before and after an experience to analyse the beamline parameters evolution
Send instructions easily to several equipments • Set the beamline in a configuration reference
The tool Snap DataBase (SNAP)
• Allows to take a snapshot of a set of attributes• Allows to apply values on a set of attributes• Allow to compare attributes values to each other
It is managed via the graphical tool named Bensikin
Archiving Workshop (Soleil, May 2010) 8
SNAP Archiving: Bensikin
BENSIKIN : Configuration and Exploitation
Archiving Workshop (Soleil, May 2010) 9
SNAP Archiving: Devices
Devices SnapArchiver : takes the snapshot and stores it in the DB
(Hibernate java technology) SnapManager : triggers the launch snapshot to an available
archiver SnapExtractor : various extraction commands as getSnap
Archiving Workshop (Soleil, May 2010) 10
ADB Archiving: General Description
Operators Requirements Supervise attribute quality and device state changes
The tool Alarms DataBase (ADB)
• Monitors the attributes alarm state
It is managed via the graphical tool named Macarena
Archiving Workshop (Soleil, May 2010) 11
ADB Archiving: MacarenaMacarena : Current and Deleted Alarms
Archiving Workshop (Soleil, May 2010) 12
ADB Archiving: Devices
Devices AdbArchiver : monitors the attributes quality and stores the
values in the DB
AdbManager :
• dispatches alarm supervision between the existing AdbArchiver devices
• Used by the GUI for each database transaction.
AlarmDiary : contains the available alarms list, access by the GUI only
Archiving Workshop (Soleil, May 2010) 13
Common Points
SGAD: GUI tool generates a MySql DB creation script for Unix or Windows for HDB, TDB or/and SNAP DB
Common point : Archiving systems are provided in Java 1.6
Archiving Workshop (Soleil, May 2010) 14
Soleil Infrastructure
Archiving Workshop (Soleil, May 2010) 15
Machine Deployment status
All the archiving systems use an Oracle Database
SNAP Oracle engine updated Following some Bensikin updates and a little ICA pressure the
operators plan to work more with it during the next weeks• Currently important data are stored in Matlab files• Still now, the Vacuum group makes some manual comparison at
every run startup
ADB Before the data were stored into files System recently migrated to be built as HDB/TDB system GUI consolidation is ongoing and strongly awaiting by the operators to
help them in the control system monitoring
Archiving Workshop (Soleil, May 2010) 16
Machine Deployment status
HDB : 8900 attributes 8750 scalars + 150 spectrums
• 1800 at 10 s• 4000 at [30 – 60] s• 1800 at [200 – 900] s• 1000 at [1000 – 3600] s• The rest is upper 3600 s
25 DServers * 5 devices, so 125 devices hold by One Linux Server• Each device supports about 70/75 attributes (initial estimation was 50
attributes / device) 5 years of online data
TDB : 6500 attributes 6470 scalars + 30 spectrums
• 10 at 100 ms• 1900 at 500 ms• 3100 at [1 – 5] s• 650 at [10 – 60]• The rest is upper 60 s
60 DServers * 5 devices, so 300 devices hold by Three Linux Servers• Each device supports about 20/25 attributes (estimation is about 30
attributes / device) Data are available 30 days
Archiving Workshop (Soleil, May 2010) 17
Beamline Deployment status
All the archiving systems use a MySql Database
SNAP Really often used to store motor positions, to set beamline in reference
position ...
TDB: at most 250 attributes Mainly scalars 1 DServers * 5 devices Data are available 20 days (easily configurable per beamline) Minimal archiving period equals to 1 s due to a MySql limitation
HDB: Not available Oracle installation is in progress All the hardware infrastructure is available Will be deployed progressively beamline per beamline
Archiving Workshop (Soleil, May 2010) 18
Oracle DB Solution
Archiving Workshop (Soleil, May 2010) 19
Oracle Solution
Refer to the Jérémy’s presentation
Archiving Workshop (Soleil, May 2010) 20
Latest Modifications
Archiving Workshop (Soleil, May 2010) 21
Latest Modifications: Mambo
Mambo
Better .lock file management
• At Mambo startup this file is stored in the directory linked to the connexion profile
• This file is still available when the application is brutally interrupted
The profile is no more usable
• Now .lock can be easily suppressed after confirmation
Archiving Workshop (Soleil, May 2010) 22
Latest Modifications: Mambo
DB connection parameters location
Either in command line with • … -DHDB_HOST=srv1 -DHDB_NAME=hdb
-DTDB_HOST=srv1 -DTDB_NAME=tdb fr.soleil.mambo.Mambo …
• Useful to start 2 Mambo in front of 2 differents DB
Or in Hdb/Tdb Archiver class properties
• DbHost, DbName
• Properties also used by the archivers devices
Archiving Workshop (Soleil, May 2010) 23
Latest Modifications: Mambo
Mambo
Display Multiple VC Some parts has been transformed in java bean :
• Table Mode• Connection Window• VC (not completly done)
Start archiving error description in the log part
Archiving Workshop (Soleil, May 2010) 24
Archivage HDB/TDB : Mambo
Archiving modes: something more in the Absolute and Relative modes Definition
• Attribute value is archived when the variation between two successive values (in absolute value or in percentage) is upper configured bounds.
Default behavior • the variation is computed with the attribute value previously read on the device and the current
value Slow drift behavior
• the variation is computed with the attribute value previously stored in the DB and the current value
Archiving Workshop (Soleil, May 2010) 25
Latest Modifications: Mambo Mambo
New Graphical tool usage named Comete various impacts :
• Spectrum viewer • X Axis configuration
management
Indice
Time
Time Stack
Archiving Workshop (Soleil, May 2010) 26
Latest Modifications: Mambo
Mambo
Set All Colors in the New/Modify VC
Archiving Workshop (Soleil, May 2010) 27
Latest Modifications: HDB/TDB Devices
HdbArchiver Some attributes can be archived under 10 s by configuration (not
fully tested)
Hdb/TdbArchivingWatcher Device state modification Number of KO attributes ≥ MinNumberOfKoForAlarm (new
property) then State = ALARM otherwise State = ON
Database Api refactoring Done but Not completed
Archiving Workshop (Soleil, May 2010) 28
Latest modifications: Bensikin
Context description Number of attributes
SnapShot description Contexte number Number of attributes
SetEquipment With Command Send a command with the
Read or Write value as argin
At startup: last open snapshot is loaded (quickly opened)
Archiving Workshop (Soleil, May 2010) 29
Latest modifications: Bensikin
SnapShots comparaison String and boolean attributs types can be also compared
Table bean and Connection window bean usage
Archiving Workshop (Soleil, May 2010) 30
Latest modifications: Bensikin
Application options (proper to the profile) : Date Filter criteria Snapshot expanded
Prefilled Time field
Archiving Workshop (Soleil, May 2010) 31
Latest modifications: Bensikin
Columns order definition in the comparison window
Configuration of the Data included in the Snapshot export
Application options (proper to the profile)
Archiving Workshop (Soleil, May 2010) 32
Latest modifications: SNAP Devices
SnapArchiver / Manager LaunchSnapshot method has been moved to its right place: it
concerns the SnapManager and call the TriggerLaunchSnapshot command of the SnapArchiver
Archiving Workshop (Soleil, May 2010) 33
Production
Archiving Workshop (Soleil, May 2010) 34
Maven production
Now .classpath project is empty pom.xml usage
Pom.xml contains the main jars files see Vincent Hardion presentation on
Thursday
Archiving packaging ongoing Either one zip file with the complete archiving systems distribution Or one zip file per archiving system Zip file will be stored in the Tango Pink site
<dependency> <groupId>fr.soleil.lib</groupId> <artifactId>hdbtdbArchivingApi</artifactId></dependency>
<dependency> <groupId>fr.soleil.lib</groupId><artifactId>commonArchivingApi</artifactId><version>1.4.4</version></dependency>
<dependency><groupId>fr.esrf.tango</groupId><artifactId>TangORB</artifactId></dependency>
Archiving Workshop (Soleil, May 2010) 35
To be improved
Archiving Workshop (Soleil, May 2010) 36
Missing features
No archiving system manage the Tango::Image data_format Archiving system monitoring must be improved for computer
team and operators Long term TDB: some data mustn’t be deleted Logging information improvement with log 4 j usage Database api code refactoring not fully finalized MySql doesn’t support the milliseconds Tdb/Mysql DB upgrade (versus Oracle) to ensure a heavy load
(eq. SqlLoader, jobs, partitions, backup mechanism …)
Archiving Workshop (Soleil, May 2010) 37
Summary
Archiving Workshop (Soleil, May 2010) 38
To sum-up
Archiving Systems are functionally fully operational many interesting features are missing
Deployment is a big part of the archiving system activity we are lacking tools to ease this task
Oracle DB infrastructure management is a huge task How to share the know-how for MySql management ?
Archiving Systems are always in evolution Sharing modifications, with the community, is crucial to avoid
forking
Last but not least : the Archiving system philosophy Give Users the ability to be autonomous