#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Test your cloud as code

Pierre-Luc Dion (pdion@cloudops.com)

#CCCEU14 | #CloudStackWorks

Overview

• introduction

• Why?

• cloudstack_spec: how it work

#CCCEU14 | #CloudStackWorks

About Me

• Cloud Architect @ CloudOps

• Work full time with Apache CloudStack– As user

– As contributor

• Apache CloudStack PMC member

• Ruby and Chef Enthusiast

#CCCEU14 | #CloudStackWorks

About CloudOps

• CloudOps builds and operates CloudStack based clouds of all shapes and sizes

• Develops cloud infrastructure solutions and operational models

• 24x7x365 managed service for CloudStack based cloud infrastructures

• Customers are global

• Based in Montreal, Canada

#CCCEU14 | #CloudStackWorks

Current state of the project

• Proof of concept

• Support– VPC

– Port Forwarding

– Create snapshot

– Create template from snapshot

– Various zone features validation.

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Why ?

#CCCEU14 | #CloudStackWorks

What I’m trying to solve

• We have multiple LAB CloudStack installations running multiple versions and configurations. Need to validate their usability.

• CloudStack Release Upgrade tests on real hardware.

#CCCEU14 | #CloudStackWorks

Monitoring is not enough

• Monitoring can tell if there is enough Secondary storage. But cannot tell if it’s usable in CloudStack.

• Monitoring cannot tell if Featured templates are usable.

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

But what about Marvin?

#CCCEU14 | #CloudStackWorks

I did not tried Marvin

1. Look like it provision CloudStack configs and tests, right?

2. Defining tests require coding (from user perspective)

3. Could hardly benefit from ServerSpec, specinfra or test-kitchen

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Marvin+-+Testing+with+Python

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

cloudstack_spec

#CCCEU14 | #CloudStackWorks

The purposes

• Test existing CloudStack installation

• Validate functionality of a Cloud

• Advanced tests:– Port-forwarding rules into VPC

– Create Instance from Snapshot

– Create Template from Snapshot

#CCCEU14 | #CloudStackWorks

About cloudstack_spec

• Ruby Rspec based

• Based on ServerSpec

• Test case easy to define

• Use: – cloudstack_ruby_client

– serverspec

– specinfra

• Code maturity: under heavy dev

#CCCEU14 | #CloudStackWorks

It is not

• Deployment tool

• Replacement to Marvin

• Monitoring plugin/tool

• Reporting tool

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

In Action

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Define tests

#CCCEU14 | #CloudStackWorks

Zone

require 'spec_helper'

describe zone do

it { should exist }

it { should be_allocated }

its(:local_storage) { should be_set }

its(:security_group) { should_not be_set }

its(:network_type) { should match("Advanced") }

end

#CCCEU14 | #CloudStackWorks

Zone

require 'spec_helper'

describe zone do

it { should exist }

it { should be_allocated }

its(:local_storage) { should be_set }

its(:security_group) { should_not be_set }

its(:network_type) { should match("Advanced") }

end

Do I have a zone?

#CCCEU14 | #CloudStackWorks

Zone

require 'spec_helper'

describe zone do

it { should exist }

it { should be_allocated }

its(:local_storage) { should be_set }

its(:security_group) { should_not be_set }

its(:network_type) { should match("Advanced") }

end

The zone is enabled

(allocated)?

#CCCEU14 | #CloudStackWorks

Zone

require 'spec_helper'

describe zone do

it { should exist }

it { should be_allocated }

its(:local_storage) { should be_set }

its(:security_group) { should_not be_set }

its(:network_type) { should match("Advanced") }

end

rspec_its: eval method attributes

#CCCEU14 | #CloudStackWorks

Zone

require 'spec_helper'

describe zone do

it { should exist }

it { should be_allocated }

its(:local_storage) { should be_set }

its(:security_group) { should_not be_set }

its(:network_type) { should match("Advanced") }

end

Easy valid on false

#CCCEU14 | #CloudStackWorks

Zone

require 'spec_helper'

describe zone do

it { should exist }

it { should be_allocated }

its(:local_storage) { should be_set }

its(:security_group) { should_not be_set }

its(:network_type) { should match("Advanced") }

end

Rspec string validation

#CCCEU14 | #CloudStackWorks

Output results

#CCCEU14 | #CloudStackWorks

Output results

#CCCEU14 | #CloudStackWorks

Output results

#CCCEU14 | #CloudStackWorks

Output results

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Does my zone have system VMs?

#CCCEU14 | #CloudStackWorks

System vm tests

%w(consoleproxy secondarystoragevm).each do |svm|

describe system_vm(svm) do

it { should exist }

it { should be_running }

it { should be_reachable }

end

end

#CCCEU14 | #CloudStackWorks

SVMs output

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Create VM inside VPC and test the network

#CCCEU14 | #CloudStackWorks

VPC test Scope

• Create VPC

• Create VPC network Tier

• Create Instance

• Enable Port-Forwarding for SSH

#CCCEU14 | #CloudStackWorks

vpc_spec.rb

describe vpc('spec-vpc1') do

it { should be_created }

it { should exist }

it { should be_ready }

it { should be_reachable }

describe vpc_tier('tier11') do

it { should be_created }

it { should exist }

describe virtual_machine('vm-test1') do

it { should be_created }

it { should_not be_reachable }

it { should exist }

it { should be_running }

its(:open_pf_ssh) { should be_set }

context 'With port forwarding' do

it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }

end

end

end

end

#CCCEU14 | #CloudStackWorks

vpc_spec.rb

describe vpc('spec-vpc1') do

it { should be_created }

it { should exist }

it { should be_ready }

it { should be_reachable }

describe vpc_tier('tier11') do

it { should be_created }

it { should exist }

describe virtual_machine('vm-test1') do

it { should be_created }

it { should_not be_reachable }

it { should exist }

it { should be_running }

its(:open_pf_ssh) { should be_set }

context 'With port forwarding' do

it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }

end

end

end

end

be_createdCreate the object

#CCCEU14 | #CloudStackWorks

vpc_spec.rb

describe vpc('spec-vpc1') do

it { should be_created }

it { should exist }

it { should be_ready }

it { should be_reachable }

describe vpc_tier('tier11') do

it { should be_created }

it { should exist }

describe virtual_machine('vm-test1') do

it { should be_created }

it { should_not be_reachable }

it { should exist }

it { should be_running }

its(:open_pf_ssh) { should be_set }

context 'With port forwarding' do

it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }

end

end

end

end

be_reachablePing test

#CCCEU14 | #CloudStackWorks

vpc_spec.rb

describe vpc('spec-vpc1') do

it { should be_created }

it { should exist }

it { should be_ready }

it { should be_reachable }

describe vpc_tier('tier11') do

it { should be_created }

it { should exist }

describe virtual_machine('vm-test1') do

it { should be_created }

it { should_not be_reachable }

it { should exist }

it { should be_running }

its(:open_pf_ssh) { should be_set }

context 'With port forwarding' do

it { should be_reachable.with( :port => 22, :proto => 'tcp' ) }

end

end

end

end

be_reachableTcp connection test

Using Public IP

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Results of VPC tests…

#CCCEU14 | #CloudStackWorks

test execution (1/4)

#CCCEU14 | #CloudStackWorks

test execution (2/4)

#CCCEU14 | #CloudStackWorks

test execution (3/4)

#CCCEU14 | #CloudStackWorks

test execution (4/4)

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Working with Rspec

#CCCEU14 | #CloudStackWorks

Custom Rspec matcher

# If the zone is allocated (Enabled)

RSpec::Matchers.define :be_allocated do |expected|

match do |actual|

actual.allocated? == true

end

failure_message do |actual|

"status: #{actual.allocated?}"

end

description do

"be allocated (Enabled)"

end

failure_message_when_negated do |actual|

"Status: #{actual}"

end

end

#CCCEU14 | #CloudStackWorks

resources (Test cases)

• Test cases are:– Objects to test are Ruby Class:

CloudstackSpec::Resource::Snapshot

– Unit tests are Ruby Methods inside Class:

self.exist?

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

What’s next?

#CCCEU14 | #CloudStackWorks

Future improvements

• Provide as Gem

• Auto config, same as Serverspec

• More tests

• Better error message

• Add compatibility with ServerSpec

• test-kitchen compatibility?

#CCCEU14 | #CloudStackWorks

Help needed

• ServerSpec integration– Test from within an instance

• More test scenarios

#CCCEU14 | #CloudStackWorks

Future usage

• Validate new CloudStack release

• Cloud heath check

• Post automation tests

• Continuous Integration Tests (CI)

#CCCEU14 | #CloudStackWorks

Conclusion

• Test phase in the automated DevOps world

• Tests definition made easy.

• More tests case in progress.

#CCCEU14 | #CloudStackWorks

More Information

• Pierre-Luc Dion– pdion@cloudops.com

– www.cloudops.com

–@CloudOps_

• Github– https://github.com/pdion891/

cloudstack_spec

#CCCEU14 | #CloudStackWorks

#CCCEU14 | #CloudStackWorks

Thank you!