OSM DEVOPSCOMBINING CI/CD AND INTEROP TESTING BEST PRACTICES
Presented by Silvia Almagia, ETSI
© All rights reserved
Talk overview
• A bit of background on ETSI CTI
• About NFV & Open Source MANO
• Setting up the OSM CI/CD
• Takeaways
© All rights reserved
ETSI CTI, Centre for Testing and Interoperability
We are specialists in testing and validating standards, ensuring interoperability, conformance and implementability by supporting ETSI groups with different :
Activities:
• Development of conformance and interoperability
test methodologies, languages and specifications
• PlugtestsTM, interop events
Tools:
• HIVE, Hub for Interoperability and Validation and ETSI
But also..
• PoCs, Proof of Concept Frameworks
• Open Source project hosting, development, testing
• OpenAPI development, hosting and validation
3 © All rights reserved
HIVE – Hub for Interoperability and Validation at ETSI
4 © All rights reserved
• Allows to securely interconnect remote sites to ETSI
• Seamless access among connected sites and local infrastructure at ETSI
• Provides a neutral and coordinated framework for collaborative testing and validation activities among different organizations: continuous and ubiquitous environment
Many uses at ETSI CTI:
• Support remote participation to Plugtests on many different technologies (NFV, MCPTT, Small Cells…)
• Support pre-testing phases on complex Plugtests
• Support TTCN-3 test suites validation campaigns with remote implementations
• Support multi-party Proofs of Concept and demonstrations
.. and lately also:
• Support OSM Remote Labs network … and CI/CD pipeline
HIVE example: 1st NFV Plugtests, January 2017
5 © All rights reserved
ETSI NFV & ETSI OSM (Open Source MANO)
6 © All rights reserved
….
VNFsVNFs
VNFsVNFs
ETSI NFV: Industry Specification Group working on Network Functions Virtualisation
ETSI OSM: ETSI hosted Open Source project developing a Management and Orchestration (MANO) stack aligned with ETSI NFV Information Model
Why ETSI Open Source MANO ?
7 © All rights reserved
Layering Abstraction
Modularity Simplicity
ETSI NFV
OSM: Large and diverse community, +80 members
8 © All rights reserved
• 9 Global Service Providers
• Leading IT/Cloud players
• VNF providers
MeadowCom
OSM Modules
9 © All rights reserved
Monitoring
plugins
OST
Plugin
OV
Plugin
VMW
Plugin
ODL
Plugin
ONOS
Plugin
MonitoringSO-VCA
Interface
Test
Framework
& scripts
VIM-EMU
SO-RO Interface
GUI CLI Client
9
IM NBI (Models)*
VCA
SO
UI-CLI
IM-NBI
MONN2VC
RO
VNFD &
Charms
Installers
Dev
Ops
OSM CI/CD Pipeline
10 © All rights reserved
Trigger per-
module pipelines
Scan U Test Build
Scan U Test Build Archive
ArtifactStorage
System Install
Smoke Tests
SystemTests
Stage 1 Stage 2
Stage 3Stage 4
Archive
Code Commit
VIM&NFVI
VNFsVNFsVNFsVNF
VIM&NFVINFVI & VIM
OSM Release Server
Stage 1: Launch
• Triggered by Gerrit upon developer’s code commit
• Launch multi-branch pipelines
• Call stage 2 and build required modules
11 © All rights reserved
Trigger per-
module pipelines
Stage 1
Code Commit
Stage 2
Stage 2: Per-module pipeline
• Triggered by Stage 1, on specific modules
• Operates within a Docker container
• Per-module call backs:1. License Scan (FOSSology)
2. Unit Test
3. Package build
4. Artifact creation & storage (Artifactory)
12 © All rights reserved
Scan U Test Build Archive
Stage 2
ArtifactStorage
Stage 1
Stage 3
ArtifactStorage
Stage 3: System Integration
• System Install from binaries
• From stage 2 & Artifact Storage
• Smoke Tests
• Based on pytest fixtures
• No NFV Infrastructure, Virtual Infrastructure Manager (VIM), SDN Controller ..
• Leveraging OSM Client library and VIM Emulator
• API checks, VNFD, NSD upload, …
13 © All rights reserved
System InstallSmoke Tests
Stage 3
Stage 4
Stage 2
OSM Client
VIM-EMU
sVNFD
Stage 4: System Tests
• With real VNF Infrastructure, VIMs, SDN Controllers, VNFs
14 © All rights reserved
OSM
System Tests
Stage 4
OSM Client
VIM&NFVI
VNFsVNFsVNFsVNF
VIM&NFVINFVI & VIM
Stage3
sVNFD
Stage 4: Challenges
• Requires NFV Infrastructure
• Many possible combinations of:
• NFV Infrastructure
• VIMs and SDN Controllers
• VNFs
15 © All rights reserved
Leverage OSM Community assets and HIVE(Hub for Interoperability and Validation at ETSI)
… and opportunities
Stage 4 with OSM Remote Labs
16 © All rights reserved
SystemTests
Stage 4
OSM Client
OSM Remote Labs
….
Takeaways
• “classic” open source ci/cd pipelines can benefit from interoperability testing approaches to achieve automated system and interop testing in complex environments
• requires to define and follow some best practices when writing system and interop test cases to ensure test environment(s) are stable and reliable any time (pre-/post-testing checks and clean-ups)
• the community can help to prioritize targeted test environments by directly making them available and providing test resources
17 © All rights reserved
OSM.etsi.org
18 © All rights reserved