NICA control system, beam diagnostics
V.Andreev, E.Gorbachev, A.Kirichenko, D. Monakhov, S. Romanov, G.Sedykh, T. Rukoyatkina, V.Volkov
VBLHEP, JINR, Dubna
MAC'2015 NICA Control System 2
Control System development goals
• Centralized control of distributed objects• Reliable operation and fast recovery• Provide safe operation, access restriction• Easy support, modification and scalability• Easy and fast development and deployment• Integration of third-party and existing systems
MAC'2015 NICA Control System 3
Plan• Control system layout– Middleware– Front-end layer– Client layer– Service layer
• Control System services• Nuclotron and NICA subsystems development• Several information about beam diagnostics• Conclusion.
MAC'2015 NICA Control System 4
Control System layout
PXI, VME, PC
Devices, PLCs
Front-end layer.PXI, VME, Industrial PCs.
PC
Client layerPC workstations.Windows, Linux
Service layer.Processing apps, services.Windows, Linux
VMsArchivingLinux
Tango DBLinux.
Storage
EthernetCS services
Ethernet
Field bus
Mid
dlew
are
MAC'2015 NICA Control System 5
MiddlewareBased on :• CORBA based distributed object-oriented control
system framework• Multiplatform• Hides network location and technical details• Provides unified interface for hardware equipment or
service:– Commands– Attributes– Properties
• Has services and tools for control implementation.
MAC'2015 NICA Control System 6
Front-end layerNational Instruments PXI – high performance modular platform for
measurements and automation systems
Digital signal processing modules FlexRIO High performance embedded systems
Compact RIO
MAC'2015 NICA Control System 7
Front-end layerTango drivers with runtime configurable properties for NI equipment were developed:• Digitizers and scopes• Analog and digital I/O• Timers and counters• Digital multimeters• RTD input modules• TANGO interface for FlexRIO and CompactRIO.Combination of tango devices can be used to quickly deploy a solution based on almost any NI acquisition equipment.
MAC'2015 NICA Control System 8
Client layer: Desktop & Web appsThe client layer is responsible for presentation of the accelerator complex state to the operator, visualize measurements and provide user interface to fulfill some control tasks.
MAC'2015 NICA Control System 9
Service layer
Contain high-level TANGO device servers, representing whole subsystem:• Collects and process data from low-level
devices• Performs control loop• Provides standard interface to client
applications: commands, attributes
MAC'2015 NICA Control System 10
Service layer: VirtualizationApplications running on service layer are hardware independent and can be virtualized (run on VM).Advantages of Virtualization: easier management, better tasks isolation, resources tuning and utilization, high availability.
Proxmox VE - complete open source virtualization management solution for servers. It manages virtual machines, storage, virtua-lized networks, HA clustering
Proxmox VE features:• Web GUI• Role-based administration• Integrated backup tool• Flexible Storage
MAC'2015 NICA Control System 11
Service layer: Shared storage• Crucial component of virtualization to provide high
availability of VMs• Need good performance , both transfer rate and IOPS
to manage read/write operations of number of VM images
• Have to be redundant to provide data consistency in case of hardware or network failure
• Need to be scalable – should be able to add more storage space without loosing performance
MAC'2015 NICA Control System 12
Service layer: CEPH storage is a distributed object store and file system. Ceph is highly reliable, easy to manage and open-source. It provides redundancy, excellent performance and scalability.We currently use 3 ceph storage nodes based on Supermicro servers:
• Dual socket E5-2600 Intel Xeon• 32Gb RAM• 5 x 600G SAS2 15k RPM drives• 1 enterprise class SSD Intel S3710• LSI 2128 RAID controller with 512Mb cache• 2x1Gb Ethernet card for public network• 2x10Gb Ethernet card for CEPH networks
MAC'2015 NICA Control System 13
Service layer: Scalability
The scalability achieved by adding nodes to cluster. CEPH retains most characteristics at scale or even improves. Adding more nodes to run more VMs:• Capacity increases• Throughput increases• IOPS increase• CPU cores and RAM increaseTemporary impact during re-balancing.
MAC'2015 NICA Control System 14
Service layer: AdministrationAstor/Starter –tools for remote control and monitoring of TANGO applications
MAC'2015 NICA Control System 15
Service layer: Monitoring TANGOSpecial TANGO device is running on every F-End computer. It presents information about computer metrics. Another device monitoring states of every TANGO device of certain subsystem.
MAC'2015 NICA Control System 16
Service layer: monitoringDesktop and web clients to represent states and statuses of TANGO
MAC'2015 NICA Control System 17
Service layer: Access controlAccess control and security are vital to safely run a distributed control system.
Tango provides AccessControl service that allows restricting tango connections to some specific devices only for certain users from certain locations.
Additional authorization checks are necessary on the server side for more secure and flexible access restrictions. We developed our own authorization system which use additional authorization device server.
MAC'2015 NICA Control System 18
NicaControls database
MAC'2015 NICA Control System 19
NicaControls databaseDatabase WEB-manager written on Java (JSF, JPA, PrimeFaces):• Tree view to display
subsystems hierarchy.• Plain tables output. Sorting,
filtration, navigation between tables are provided.
MAC'2015 NICA Control System 20
Nuclotron CS development• Injection control and diagnostics
MAC'2015 NICA Control System 21
Nuclotron CS development• Slow extraction control and diagnostics
MAC'2015 NICA Control System 22
Nuclotron CS development• Betatron tune measurement
MAC'2015 NICA Control System 23
NICA CS components prototypes• Booster thermo diagnostics – 80 channels PXIe based
measurements.• Booster magnetic cyсle control – FlexRIO + custom IO
modules.• Booster RF – integration of third-party CS. • Booster vacuum – integration of third-party Zenon
SCADA control.• NICA injection/extraction control – CompactRIO +
custom IO modules.
MAC'2015 NICA Control System 24
Booster BPM assembly drawing
MAC'2015 NICA Control System 25
Booster BPM prototype
MAC'2015 NICA Control System 26
ConclusionsDistributed, scalable control system infrastructure based on Tango has been designed to provide :• rapid development by using code generation and
drivers for NI equipment and other;• reliable operation by deployment on Proxmox cluster;• easy administration by using hardware and software
monitoring, server-based access control; • easy management of equipment and software by using
NicaControls database.Works on beam diagnostics are in progress now
MAC'2015 NICA Control System 27
Thank you for your attention!