It’s not an embedded Linux distribution –It creates a custom one for you.
Elizabeth FlanaganIntel Corporation
April 11, 2011
Delivering Predictability: The Yocto Project Autobuilder, Automated Sanity Testing, License Collection, and Build Statistics Tracking
2/52
Being proactive about code quality
● Reproducible builds● Identify bugs and fix early and often● Reduce time needed for code stabilization● Avoid integration headaches● Build performance history● Manage the chaos● License compliance● Deep QA Testing
3/52
Being proactive about code quality
Maximizing your ability to respond to changes in a complex embedded
ecosystem.
4/52
Being proactive about code quality
Reduce Software Development Lifecycle Churn.
5/52
Complexities
Our Recipe
6/52
Complexities
Our Recipe
qemuarm
qemumips
qemuppc qemux86_64
qemux86
7/52
Complexities
Our Recipe
qemuarm
qemumips
qemuppc qemux86_64
qemux86
Our Recipe
qemuarm
qemumips
qemuppc qemux86_64
qemux86
mpc8315e atom-pc
beagleboard routerstation
8/52
Complexities
Our Recipe
qemuarm
qemumips
qemuppc qemux86_64
qemux86
mpc8315e atom-pc
beagleboard routerstation
emenlow
Jasperforestn450 sandybridge
crownbayCrownbay-noemgd
9/52
Complexities
● One recipe● 5 architectures● 4 core BSP● 6 non-core BSPs in meta-intel● 15 x-compiles● But....
10/52
Complexities
● One recipe● 16 theoretical different image types
per arch * 15 architectures– Not all arches support all image
types● 240 total theoretically possible
images– sato, lsb, sdk....
11/52
Complexities
If we're not proactive about code quality, lots of things can go wrong.....
12/52
Complexities
It's only going to get more complex
13/52
What we need
– Reproducible builds– Basic QA– License tracking– Finding the pain points
14/52
Delivering Predictability
Sanity Testing
License Wrangling
Build Statistics
Autobuilder Poky
Yocto Project
15/52
Delivering Predictability
Sanity Testing
License Wrangling
Build Statistics
Autobuilder Poky
Yocto Project
16/52
Autobuilders
● Production Autobuilders● Quickly respond to a fast changing
code base● Avoid “Works on my machine”-itis● Find race conditions● Find host dependent issues● Find bad commits● Help bisect build failures● Help find dependency chain
breakage
17/52
Autobuilders
● Developer Autobuilders● Test cross-compilation before
commit● Production style builds● Small OS footprint● Build what you want to build
18/52
Autobuilders
● Yocto autobuilder● buildbot based● git://git.yoctoproject.org/poky-
autobuilder.git● Setup in under 5 minutes!
19/52
Autobuilders
● Prerequisites:● Python 2.6● python-twisted● python-jinja2● python-twisted-mail● sqllite
20/52
Autobuilders
● Comes with● Basic pokyABConfig.py● Helper scripts● Easy Installer
21/52
Set up your own!
cd ~git clone git://git.yoctoproject.org/poky-autobuilder.gitcd poky-autobuilder./scripts/poky-setup-autobuilder bothsource ~/.profile;cd ../poky-master; make startcd ../poky-slave; make start
22/52
Live Demo
23/52
Delivering Predictability
Sanity Testing
License Wrangling
Build Statistics
Autobuilder Poky
Yocto Project
24/52
Sanity Testing
● Extensible● Frees up QA resources● Reproducible smoke testing● Multiarch/multiimage scenarios● Can run automatically post build via local.conf● Or via an autobuilder
25/52
Sanity Testing
Sanity Test Bitbake Class
Architecture/image based test scenario
Test library/runner
Test helper scripts
26/52
Sanity Testing
meta/classes/imagetest-qemu.bbclass
scripts/qemuimage-tests/scenario/${ARCH}/*
scripts/qemuimage-testlib and runners
scripts/qemuimage-tests/{sanity|tools}/*
27/52
Sanity Testing
● Test suite● Architecture and image based scenarios ● Very easy to add already existing tests
28/52
Sanity Testing
sanity sshsanity scpsanity dmesgsanity zypper_helpsanity zypper_searchsanity rpm_querysanity connmansanity shutdown
29/52
Sanity Testing
● Tests● bash/expect based test scripts● called via test runners in scripts/sanity ● Tests stored in qemuimage-testlib
●More secure to create tap devs with poky-gen-tapdevs.
30/52
Sanity Testing
● Gotchas:● QEMU user NOPASSWD● More secure to create tap devs with poky-gen-tapdevs.
● For headless, see wiki docs: https://wiki.pokylinux.org/wiki/Enabling_Automation_Test_in_Poky
31/52
Delivering Predictability
Sanity Testing
License Wrangling
Build Statistics
Autobuilder Poky
Yocto Project
32/52
License Wrangling
● Verify image compatibility to required license type● non-GPLv3
● Provides an entire package directory tree● Actual licenses ● Generics.
● Helps maintain license compliance
33/52
License Wrangling
Recipe contains:● LICENSE
● Tells license.bbclass the common license type● Symlink from license wrangling output to a generic
● LIC_FILES_CHECKSUM● License file URI● Checksum● Where we get the specific license
34/52
License Wrangling
License are found in:
${POKYBASE}/build/tmp/deploy/images/licenses
35/52
Delivering Predictability
Sanity Testing
License Wrangling
Build Statistics
Autobuilder Poky
Yocto Project
36/52
Build Statistics
Build level:● Host info● Elapsed build time● CPU usage● Build failure information
37/52
Build Statistics
Package level:●List of events triggered●Elapsed event time●CPU usage●Event failure information
38/52
Build Statistics
PackageEvent
Build
EventEvent
PackageEventEventEvent
build/tmp/deploy/images/licenses/:poky-image-minimal-qemux86`-- 201103251310 |-- build_stats |-- autoconf-native-2.65-r2 | |-- do_compile | |-- do_configure | |-- do_fetch | |-- do_install | |-- do_patch | |-- do_populate_sysroot | |-- do_setscene | `-- do_unpack |-- automake-native-1.11.1-r1 | |-- do_compile | |-- do_configure | |-- do_fetch | |-- do_install | |-- do_patch | |-- do_populate_sysroot | |-- do_setscene | `-- do_unpack
Image Type
39/52
Build Statistics
Gives us:● Performance indicators● Track down issues
● CPU/Dependancy/IO bound ●Visualize your build performance
● Patch to pybootchart
40/52
Build Statistics Visualization
Time
41/52
Build Statistics Visualization
42/52
Build Statistics Visualization
43/52
Build Statistics Visualization
44/52
Build Statistics Visualization
45/52
Where do we go from here?
46/52
Where do we go from here?
Autobuilder:●Meta-targets●Helper script integration into main config●Continuous integration
47/52
Where do we go from here?
License tracking:●More generic license files●Better LICENSE field parsing
48/52
Where do we go from here?
Build Statistics:●Collect even more data.
● Image size w/o free space.●Better data visualization.
● Web based
49/52
Production autobuilder
50/52
Resources● http://www.yoctoproject.org
● http://git.yoctoproject.org/cgit/cgit.cgi/poky-autobuilder/
● http://autobuilder.yoctoproject.org
● pybootchartgui patch for build statistics● https://wiki.pokylinux.org/wiki/Enabling_Automation_Test_in_Poky
51/52
Legal InformationINFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL® PRODUCTS. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER, AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY RELATING TO SALE AND/OR USE OF INTEL PRODUCTS, INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT, OR OTHER INTELLECTUAL PROPERTY RIGHT.
Intel may make changes to specifications, product descriptions, and plans at any time, without notice.
All dates provided are subject to change without notice.
Intel is a trademark of Intel Corporation in the U.S. and other countries.
*Other names and brands may be claimed as the property of others.
Copyright © 2009, Intel Corporation. All rights are protected.
52/52
