Normation – CC-BY-SAnormation.com
ncf
A powerful and structuredCFEngine framework
Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)
Normation – CC-BY-SAnormation.com 2
ncfwww.ncf.io
Who am I?
● Jonathan Clarke
● Title: Co-founder & Product lead at Normation
● Origins: Sysadmin, infrastructure management
● Now: Automation + “running a company”-stuff
● Contributor to free software:
– Co-creator of Rudder
– Contributor to CFEngine, OpenLDAP
● Co-organizer of events:
Normation – CC-BY-SAnormation.com 3
ncfwww.ncf.io
Intro
This presentationis about cakes
Photo CC BY 2.0 from http://www.flickr.com/photos/misscreativecakes/
Normation – CC-BY-SAnormation.com 4
ncfwww.ncf.io
Intro
Reminder: how are cakes made?
1. Ingredients
+2. Recipes
Photo CC BY-NC 2.0 from http://www.flickr.com/photos/tnemily/
Photo CC BY 2.0 from http://www.flickr.com/photos/misscreativecakes/
Normation – CC-BY-SAnormation.com 5
ncfwww.ncf.io
Background
A bunch ofCFEngine consultants
● > 4 years● Multiple companies: small, large & huge● Various uses: security, provisioning, DR...
We always got the same feedback
Normation – CC-BY-SAnormation.com 6
ncfwww.ncf.io
Feedback #1: CFEngine rocks!
CFEngine rocksMulti-platformLinux, Android, BSD, AIX, HP-UX, Solaris, Windows...
Open SourceGPLv3
Small footprint, scalableA few MB of RAM,just seconds to run...
Continuous checkingAgent based approach,no push
Resilient to errorsNetwork outages, failures,unavailable resources...
Normation – CC-BY-SAnormation.com 7
ncfwww.ncf.io
Feedback #2: CFEngine is hard!
CFEngine is hard
Too much “do it yourself”Building your own policy structure from blank slate
Steep learning curveSyntax is unusual, hardto learn and understand
Lack of feedbackOutput is hard to read,“what is going on?”
Workarounds for bugsNo way of systematically using a workaround
“Flour, eggs, milk and butter”As in: https://digitalelf.net/2013/04/a-case-study-in-cfengine-layout/
Normation – CC-BY-SAnormation.com 8
ncfwww.ncf.io
Feedback #2: CFEngine is hard!
Learning to bake cakes on your own is frustrating.
Photo CC BY-NC-SA 2.0 from http://www.flickr.com/photos/penguincakes/
Normation – CC-BY-SAnormation.com 9
ncfwww.ncf.io
Brief dilemma...
Can we fix this?
We've worked around this for customers, let's make it reusable!
CHALLENGE ACCEPTED.
Normation – CC-BY-SAnormation.com 10
ncfwww.ncf.io
Approach
Too much do it yourselfBuilding your own policy structure from blank slate
1) Provide a structured layout to start from
2) Provide single-purpose, reusable “methods” to get the basics done
Normation – CC-BY-SAnormation.com 11
ncfwww.ncf.io
Steep learning curveSyntax is unusual, hardto learn and understand
Approach
1) Hide the weirder syntaxinside these reusable “methods”
2) Only require a subset of syntaxto write everyday policies (method calls)
Normation – CC-BY-SAnormation.com 12
ncfwww.ncf.io
Approach
1) Implement workarounds into those reusable “methods” (and use them)
2) Automated tests to make sure the bugs don't “come back”
Workarounds for bugsNo way of systematically using a workaround
Normation – CC-BY-SAnormation.com 13
ncfwww.ncf.io
Lack of feedbackOutput is hard to read,“what is going on?”
Approach
1) Now everything goes through reusable “methods”, build automatic feedback in
2) Make the feedback format customisable and extensible
Normation – CC-BY-SAnormation.com 14
ncfwww.ncf.io
Result
“Forget baking,I'm gonna get some
cakes from the shop”Photo CC BY-NC-SA 2.0 from http://www.flickr.com/photos/omarsc/
Normation – CC-BY-SAnormation.com 15
ncfwww.ncf.io
Result
We created ncf
ncf is a frameworkthat runs in pure CFEngine language,to help structure CFEngine policy and
provide reusable, single purpose components distributed under the GPLv3 license.
Normation – CC-BY-SAnormation.com 16
ncfwww.ncf.io
Result
Example === 1000 words
With ncf:
Normation – CC-BY-SAnormation.com 17
ncfwww.ncf.io
Result
Example === 1000 wordsWithout ncf:
This is actuallyover-simplified:- No feedback- No exceptions for different OSes- No advanced options
Normation – CC-BY-SAnormation.com 18
ncfwww.ncf.io
Result
Example === 1000 wordsWith ncf, automatic feedback:R: [DEBUG] Promise kept, not doing anything: Install package ntp in version latestR: [INFO] Promise repaired, made a change: Build file /etc/ntp.conf from template ntp.confR: [INFO] Promise repaired, made a change: Restart service ntpR: [DEBUG] Promise kept, not doing anything: Ensure that service ntp is running
Normation – CC-BY-SAnormation.com 19
ncfwww.ncf.io
A layered approach
A layered approachServices
Techniques
IT services:“Corporate web site”
Components ofservices: “Apache”
IT Ops Knowledge
Shared information:“httpd” package name
Generic methods
Unit tasks: “Copy file”,“Install package”, ...
Normation – CC-BY-SAnormation.com 20
ncfwww.ncf.io
How does it work?
How does it work?CFEngine 3 can have self-contained “bundles” that you can call with parameters.
ntp
logger
package_install file_from_template service_restart
Outputs structured messages
Normation – CC-BY-SAnormation.com 21
ncfwww.ncf.io
Each generic_method does one thing and one thing only
Philosophy
Core principles
Objective not subjective
KISSDRY
ExtensibleOpen source
Normation – CC-BY-SAnormation.com 22
ncfwww.ncf.io
Overview of available generic_methods
Normation – CC-BY-SAnormation.com 23
ncfwww.ncf.io
Online documentation
http://www.ncf.io/pages/reference.html
Normation – CC-BY-SAnormation.com 24
ncfwww.ncf.io
Current status
Project is young, but robustNeed more generic methods
Ohloh statistics:
Source: http://www.ohloh.net/p/ncf-project
Actually CFEngine
Normation – CC-BY-SAnormation.com
Demo!
(aka “show me the code!”)
Check it out on:http://www.ncf.io/
Normation – CC-BY-SAnormation.com 26
ncfwww.ncf.io
The cherry on the cake
The cherryon the cake
Photo CC BY-NC-ND 2.0 from https://www.flickr.com/photos/creativeabubot
Normation – CC-BY-SAnormation.com 27
ncfwww.ncf.io
The cherry on the cake
Web interface“ncf builder”
Browse available generic methodsBuild Techniques
Lightweight app (client side HTML+JS)Uses a simple REST API to interact with ncf
Normation – CC-BY-SAnormation.com 28
ncfwww.ncf.io
The cherry on the cake
Normation – CC-BY-SAnormation.com
Questions?
Check it out on:http://www.ncf.io/
Jonathan CLARKE – [email protected]@jooooooon42 (that's 7 'o's)