A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 1

CERN Cloud 2nd Price Inquiry

Andrea Valassi (IT-SDC)

Thanks to Andrew, Stefan, Vladimir, Federico, Chris and the IT-SDC cloud team for help and discussions!

LHCb brainstorming – 7th October 2015

A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 2

CERN Cloud Procurement Overview• Timeline for commercial procurement of clouds

– 2nd Price Inquiry (~MC simulation workflows)• 1000 cores for LHCb from ~1st Oct to 18th Dec 2015

– 3rd Price Inquiry (“Data-intensive” workflows)• Higher budget, preparation ongoing, resources from ~March 2016

– Pre-Commercial Procurement• Project starting Jan 2016 (details in May GDB talk by Bob Jones)

• In parallel, tests with various providers– MS Azure

• Limited budget and resources for technology tests and stress tests

• Using this in IT-SDC also to prepare for Price Inquiry resources

A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 3

Azure tests• Successfully executed the first few MC jobs

– Single-core VMs, 2GB memory limit– Vcycle infrastructure (now merged to LHCb code base)

• Submitted ~10 jobs, more than half failed– Not 100% clear why, but seems correlated to “upgrade” MC

• Submitting now ~100 jobs as a new test– Special MC production excluding “upgrade” MC (Vladimir)– Also solved ASM/ARM issues on IT-SDC side (3k VMs ok)

• Next steps if this test is successful– Ramp up to ~1000 jobs– Also test Andrew’s multi-core aware pilots

A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 4

Azure tests

• On second look: only mildly correlated to “upgrade”– 4 success: 2 upgrade, 2 not– 9 failures: 7 upgrade, 2 not– Need more statistics... will try more jobs and see

A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 5

2nd Price Inquiry – Overview

A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 6

2nd Price Inquiry – Status• Status

– Provider identified: consortium with 5 sites– Network connectivity to Geant being prepared (1 week?)– Minimal resources available and under tests in IT-SDC

• LHCb preparation– Plan to use this for MC simulation– Multi-core-aware pilots by Andrew are now ready– Agreed to set up 5 different sites– LHCb-GEOC will be added to communication list– VO sharing [resources will be paid for even if not used!]

A. Valassi – CERN Clouds 2nd Price Inquiry LHCb brainstorming – 7th Oct 2015 7

2nd Price Inquiry – Questions