The Detector The Detector Control System – Control System –

FERO related FERO related issuesissuesPeter ChochulaPeter Chochula

On behalf of DCS teamOn behalf of DCS team

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

The DCS sub-systemsThe DCS sub-systems Main role of the DCS is to assure safe operation of ALICE. Main role of the DCS is to assure safe operation of ALICE.

To fulfill this task, it has to communicate with a large To fulfill this task, it has to communicate with a large number of devicesnumber of devices

The DCS devices are logically grouped into several sub-The DCS devices are logically grouped into several sub-systems which are operated separately and synchronized systems which are operated separately and synchronized via FSM tools implemented in the SMI++ packagevia FSM tools implemented in the SMI++ package

Main DCS sub-systems are:Main DCS sub-systems are: High Voltage (HV)High Voltage (HV) Low voltage (LV)Low voltage (LV) Front-End and Readout Electronics (FERO)Front-End and Readout Electronics (FERO) GasGas CoolingCooling

Some detectors implement additional sub-systems such Some detectors implement additional sub-systems such as laser control, etc.as laser control, etc.

On top of this, the DCS communicates with LHC, Detector On top of this, the DCS communicates with LHC, Detector Safety System, external services …Safety System, external services …

In this talk only issues related directly to FEE will be In this talk only issues related directly to FEE will be discusseddiscussed

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

The DCS Architecture and The DCS Architecture and its Place in ALICE Online its Place in ALICE Online

HierarchyHierarchy The DCS is a strictly The DCS is a strictly

hierarchical online hierarchical online system, its system, its synchronization with synchronization with DAQ, TRG and HLT is DAQ, TRG and HLT is achieved via the ECSachieved via the ECS

There are no horizontal There are no horizontal connections between connections between online systemsonline systems

Efficient operation of Efficient operation of ALICE sub-detectors ALICE sub-detectors depends on depends on communication between communication between online systems and this online systems and this is strongly influenced by is strongly influenced by the FERO designthe FERO design

DCS DAQ/RC

TPC

SPD

TRG HLT

ECS

…TPC

SPD

…TPC

SPD

.

.

.

FERO

LV

HV

Gas

LV

HV

FERO

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

The (simplified) DCS The (simplified) DCS operation of a deviceoperation of a device

DeviceConfiguration

ConfigurationDatabase

DeviceControl

(Regulation, switching, handling of alarms and exceptions…)

DeviceMonitoring Archive

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Two main groups of questions Two main groups of questions related to FERO need to be solved:related to FERO need to be solved:

How to access the electronics?How to access the electronics? How to operate the electronics? How to operate the electronics?

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

How to access the How to access the devicesdevices

Unlike the other online Unlike the other online systems, the DCS systems, the DCS communicates with communicates with wide range of different wide range of different devicesdevices

The nature as well as The nature as well as handling of acquired handling of acquired data differs from sub-data differs from sub-system to sub-system system to sub-system

There is no unique and There is no unique and standardized standardized mechanism for mechanism for accessing the DCS accessing the DCS devicesdevices

Ether

net Profibus

CANbus

JTAGRS 232

VME

Easy

Net

Custom solutions…

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Access to the DCS Access to the DCS hardwarehardware

The main operation tool of the DCS is a commercial The main operation tool of the DCS is a commercial SCADA system : PVSS-IISCADA system : PVSS-II

The PVSS-II offers a limited set of tools for accessing The PVSS-II offers a limited set of tools for accessing the hardware.the hardware. Native PVSS-II drivers are used where appropriate (e.g. for Native PVSS-II drivers are used where appropriate (e.g. for

RS-232)RS-232) The aim is to use a set of software interfaces, which The aim is to use a set of software interfaces, which

will hide the complexity of the underlying hardware will hide the complexity of the underlying hardware The ALICE DCS profits from (and contributes to) The ALICE DCS profits from (and contributes to)

common developments between four LHC common developments between four LHC experiments – the JCOP project. The JCOP experiments – the JCOP project. The JCOP framework software package provides solutions for framework software package provides solutions for most standard devices (power supplies, ELMB based most standard devices (power supplies, ELMB based monitors, etc.)monitors, etc.)

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

The hardware access The hardware access standards in the DCS – the standards in the DCS – the

OPCOPC

Most commercial devices are shipped Most commercial devices are shipped with software based on the OPC with software based on the OPC industrial standardindustrial standard OPC client is integrated in PVSSOPC client is integrated in PVSS The DCS team is testing OPC servers and The DCS team is testing OPC servers and

provides feedback to manufacturers if neededprovides feedback to manufacturers if needed Reports are regularly given at DCS Reports are regularly given at DCS

workshops organized during the ALICE weeksworkshops organized during the ALICE weeks

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

The main problem related to the FERO control The main problem related to the FERO control in ALICE is caused by the fact, that different in ALICE is caused by the fact, that different detectors are using different access strategies.detectors are using different access strategies. The FERO of some sub-detectors is accessed The FERO of some sub-detectors is accessed

exclusively via the DDL which belongs to the DAQ exclusively via the DDL which belongs to the DAQ domain and is handled by the DAQ/RCdomain and is handled by the DAQ/RC

Other architectures use a wide variety of access Other architectures use a wide variety of access paths, which are handled by the DCSpaths, which are handled by the DCS

In principle we can group all ALICE sub-In principle we can group all ALICE sub-detectors into four architectural classes shown detectors into four architectural classes shown on the next slideon the next slide

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

FERO Access ArchitecturesFERO Access Architectures

FERO

DDL

Control Monitoring

Class A

Class C Class DControl

Control MonitoringMonitoring

FEROFERO

Non-DDLNon-DDL

Class B

Control Monitoring

FERO

DDL

Non-DDL

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

The hardware access The hardware access standards in the DCS – the standards in the DCS – the

FEDFED The ALICE FERO (and some additional The ALICE FERO (and some additional

devices like Arem-Pro power supplies) are devices like Arem-Pro power supplies) are not covered by JCOP developmentsnot covered by JCOP developments

Unfortunately the operation of this group of Unfortunately the operation of this group of devices proves to be a very complicated onedevices proves to be a very complicated one

The Front-End Device (FED) concept for The Front-End Device (FED) concept for accessing this group of devices has been accessing this group of devices has been elaborated (see presentations on DCS elaborated (see presentations on DCS workshops and TB in May 2004)workshops and TB in May 2004)

The corresponding software architecture The corresponding software architecture (called the FED Server) is based on DIM (called the FED Server) is based on DIM protocol, which is also integrated in the protocol, which is also integrated in the PVSS-IIPVSS-II

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

FED ArchitectureFED Architecture

Class B,C,D Control

Class A+B Control

ECSECS

DAQ/RCDAQ/RC DCSDCS

Control CPU

FERO Hardware Layer

FED Server

FED Client

Profibus, JTAG, etc.

Control CPU

DDL SW

DDL SW

FEDFED

DDL

Monitoring of all classes

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Architecture of the FED Architecture of the FED ServerServer

(PVSS) DIM Client

CA1 CAi MA1 MAi

Hardware

Device driver(s)

Database

DIM server

Serv

ices

DIM Interface layer

allows for communication

with higher levels of software

Hardware access layer contains device drivers

FE

DSe

rver

Cli

ent

Soft

war

e

Com

mands

& D

ata

Application layer contains detector

control and monitoring code

(agents)

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

How to operate the FEDHow to operate the FED

The architecture of the FED server covers The architecture of the FED server covers also some basic operational aspectsalso some basic operational aspects Implemented commands allow for integration Implemented commands allow for integration

with the DCS with the DCS The DCS handles cross-dependencies The DCS handles cross-dependencies

between sub-systems between sub-systems e.g. FERO configuration is pending until the e.g. FERO configuration is pending until the

LV becomes operational, this is dependent on LV becomes operational, this is dependent on cooling status etc.cooling status etc.

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Standard FED OperationStandard FED Operation

OFF

Configured

Running

Configuring

Configure

Re - Configure

RunStop

Switch-Off

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Additional (detector-Additional (detector-specific) FED commandsspecific) FED commands

A number of FED Server commands is unique for sub-A number of FED Server commands is unique for sub-detectorsdetectors E.g. JTAG chain verification in SPDE.g. JTAG chain verification in SPD Autocalibration…Autocalibration…

In close collaboration with DCS contact persons we try In close collaboration with DCS contact persons we try to identify these commands and propose solutions. to identify these commands and propose solutions. In In some cases only a very limited feedback is provided – some cases only a very limited feedback is provided – this is mostly given by the fact that the software design this is mostly given by the fact that the software design has not reached the implementation phase yet. has not reached the implementation phase yet.

FED Server prototyping is very well advanced for some FED Server prototyping is very well advanced for some detectors (SPD, TPC, TRD, PHOS) and we are profiting detectors (SPD, TPC, TRD, PHOS) and we are profiting very much from a very good collaboration with detector very much from a very good collaboration with detector groupsgroups

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

FERO Device Monitoring FERO Device Monitoring Principle in DCSPrinciple in DCS

Pu

bli

shin

g

dea

db

and

Publishedvalue

Acquiredvalues

Samplinginterval

Value recordedIn DCS

PVSS Alarm Limit

PVSS Alarm Limit

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

FERO operationFERO operation A number of questions still needs to be answered:A number of questions still needs to be answered:

What detector specific commands need to be What detector specific commands need to be implemented? implemented?

How do we monitor and treat SEU ?How do we monitor and treat SEU ? What are the sub-system and system dependencies? What are the sub-system and system dependencies?

(switching order….)(switching order….) What parameters need to be monitored and at what What parameters need to be monitored and at what

frequencies? frequencies? What are the expected actions if some parameters are What are the expected actions if some parameters are

out of range? (sometimes is sufficient to record the out of range? (sometimes is sufficient to record the anomaly in the archive, sometimes we can recover the anomaly in the archive, sometimes we can recover the settings, in some cases the run must be stopped…)settings, in some cases the run must be stopped…)

In some cases the DCS is expected to monitor for example In some cases the DCS is expected to monitor for example local trigger counters – what happens if these are out of local trigger counters – what happens if these are out of range ?range ?

WHO and WHEN will starts the software developments WHO and WHEN will starts the software developments on detector side, HOW and WHERE do we test the on detector side, HOW and WHERE do we test the prototypes?prototypes?

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Typical problem requiring Typical problem requiring synchronization between online synchronization between online

systems (Example – Class A device)systems (Example – Class A device)

DCS DAQTRG

FEROVR

ECSVR Failure

(e.g. due to SEU)

Recovery Action by DCS

As a consequencethe FERO gets mis-configured

DCS informs the DAQ and TRG via ECS

DAQ reloads the FERO

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Device Configuration and Device Configuration and ArchivingArchiving

All configuration data is stored in online databaseAll configuration data is stored in online database Present implementation is based on OraclePresent implementation is based on Oracle

Monitoring and alarm limits are read by PVSS and Monitoring and alarm limits are read by PVSS and sent to the FED Serverssent to the FED Servers

FEE parameters (thresholds, mask matrices etc.) FEE parameters (thresholds, mask matrices etc.) are read directly by FED Servers in order to are read directly by FED Servers in order to minimize the amount of data passed through the minimize the amount of data passed through the PVSS-IPVSS-I

All acquired data, alarms and errors are stored in All acquired data, alarms and errors are stored in DCS archive (Oracle) and displayed in PVSS DCS archive (Oracle) and displayed in PVSS panelspanels

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

ALICE FERO Configuration will be shared between TRG,DAQ, DCS and ECS

DCS Configuration DCS Configuration DatabaseDatabase

System Static Configuration

Device Static Configuration

Device Dynamic Configuration

FERO Static Configuration

Configuration DB

PVSS-II & underlying software

Hardware

FERO Dynamic Configuration

Common Solutions(FW devices only)

Alice Specific

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

DCS ArchivingDCS Archiving

Archive

PVSS

ArchiveArchive

ArchiveCondition DB

PVSS

Condition DBArchive

ORACLE Database

New archiving model will be introduced in PVSS-II v. 3 - release is scheduled for September 2004. Depending on its performance, the CondDB model could be dropped

Present scheme

New scheme

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

Configuration and Configuration and Archiving Related Archiving Related

QuestionsQuestions Again, there are questions to be answered:Again, there are questions to be answered:

What are the configuration parameters and how What are the configuration parameters and how are they used? (data size, loading sequence…)are they used? (data size, loading sequence…)

What are the requirements for data archival?What are the requirements for data archival? Interface to offline, analysis procedures …?Interface to offline, analysis procedures …?

How are the configuration parameters How are the configuration parameters obtained?obtained?

Calibration proceduresCalibration procedures Data analysisData analysis Configuration database updatesConfiguration database updates

Who, when, where, how?Who, when, where, how?The DCS team is presently testing the prototypes The DCS team is presently testing the prototypes

based on ORACLE. First results will be based on ORACLE. First results will be presented at coming DCS workshoppresented at coming DCS workshop

Peter Chochula, ALICE workshop on radiation hard electronics, August 30, 2004

ConclusionsConclusions FERO access strategy has been developedFERO access strategy has been developed

Prototyping is proceeding well for SPD, HPMID, Prototyping is proceeding well for SPD, HPMID, TPC, TRD and PHOSTPC, TRD and PHOS

Operational requirements are being studiedOperational requirements are being studied Feedback is essential: please make sure that the Feedback is essential: please make sure that the

DCS-URD’s are updatedDCS-URD’s are updated DCS team is happy to help youDCS team is happy to help you

DCS lab and its infrastructure are ready for DCS lab and its infrastructure are ready for testing the prototypestesting the prototypes

Many questions are still open, please inform Many questions are still open, please inform us about the progress of developments and us about the progress of developments and your plansyour plans