Data & Storage Services

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

DSS

New tape server softwareStatus and plans

CASTOR face-to-face workshop

9-10 December 2013

Eric Cano

on behalf of CERN IT-DSS group

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

InternetServices

DSS

2New tape server software Castor Workshop Dec 2013

What’s being done?

• Finish of the push to replace legacy tape software– Started with creation of tape gateway and bridge

• Consolidation into a single daemon– Merge stack of bridge, rtcpd, taped (and children processes)– Drop abandoned features:

• Leftovers interfaces from Shift/Castor1• Fortran format translation• Multi-tape files• User file concatenation

• Full C++, maintainable software• Interfaces will remain the same (w/exception)

– Drop-in replacement, interfacing to gateway, VDQM, VMGR, and SCSI– Logs will use the same interfaces as the stager daemon

• Will keep one process per tape mount by forking– Option to kill a session for operators

• There will be only one daemon instead of the current stack• This will imply changes on the management & monitoring

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

InternetServices

DSS

3New tape server software Castor Workshop Dec 2013

What is achieved?

• Tape SCSI (st driver + generic SCSI)– Access to tape drives has been reverse engineered– All accesses now make up the interface of “drive” objects, gathered in a single

place– All in readable C++ structures, matching the SCSI specs– Simpler device access for generic SCSI ioctl

• File structures (AUL format)– Structures have been defined, API not fully defined yet

• The threading model has been completely revisited and a proof of concept has been validated– Used to have worker thread running onto a single linked list– Now fully FIFOs based “pipelined architecture”. Each stage has its own FIFO

on input. Easy to chop the work into simple steps

• All unit tested– Still to be integrated on the master branch

• Cmake build (brought to master branch)• Documentation consolidated in a single document (tape server

handbook), and completed in parallel

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

InternetServices

DSS

4New tape server software Castor Workshop Dec 2013

What remains to be done?

• Unit testing not yet brought into Castor main branch

• SCSI and AUL support still to be merged into Castor master branch

• Threading model prototype to be made prod level (error handling, unit testing, etc…)

• Session forking has to be tested• VDQM replier to be ported from tape bridge• Link with client libraries for the rest (VMGR,

VDQM, tape gateway)• Still a lot to be done, but big subjects have been

covered

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

InternetServices

DSS

5New tape server software Castor Workshop Dec 2013

Going further (post-release 1)

• This new tape server software will allow much easier and predictable maintenance

• Some idea already floating for follow-up version:– Performance monitoring (on which FIFOs are we

waiting?)– Usage of data integrity features of the drives

CERN IT Department

CH-1211 Genève 23

Switzerlandwww.cern.ch/

it

InternetServices

DSS

6New tape server software Castor Workshop Dec 2013

Timeline

• Development started last summer• Targeting delivery for end H1 ’14• First release will be double release

– Like tape gateway

• Early testing welcome (to assess impact of changes on other sites)