Ci for i-os-codemash-01.2013

transcript

CI for iOS:do more while you sleep

Kevin Munc – @muncmanMethod Up LLC

For iOS?

Pain & Suffering

It can be done

• Start with the basics

• Xcode and the command line

• Version control and CI server

• Script it, run it, schedule it

Tip #1

Jenkins

• http://jenkins-ci.org/

• Mac installer or via homebrew

• Many alternatives

• Many plugins

Tip #1.1

How many Simulators does it take to break a build?

A CI server gives you:

Automation

Feedback

Automation

FeedbackWonderful!

Automation

FeedbackWonderful!

Peace o

f mind

(for bette

r naps)

Requirement: OS X

Requirement:

• macminicolo.net

• hosted-ci.com

• cisimple.com

• macincloud.com

• hostmyapple.com

• macminivault.com

• xcloud.me

Requirement:

• macminicolo.net

• hosted-ci.com

• cisimple.com

• macincloud.com

• hostmyapple.com

• macminivault.com

• xcloud.me

Or schedule CI to run at night,

on your own Mac,

while you sleep...

Securitysudo /usr/sbin/DevToolsSecurity --enable

Security

sudo xcodebuild -license

Credentials

• The CI server needs access to your repository

• SSH key

• The CI server needs Keychain credentials if you want to sign your builds

• Developer certificate and profiles

Keychain

• Export and Import using Keychain Access app

• Keys and Certificates

• Or use the command line:• sudo security import /path/to/distribution.cer -k /Library/Keychains/

System.keychain

• sudo security import /path/to/distribution.p12 -k /Library/Keychains/System.keychain

• Don’t forget about provisioning profiles

• Troubleshooting: http://tinyurl.com/iOSDevToolTroubles

Tip #2

Avoid this error from Xcode

RunPlatformUnitTests: warning: Skipping tests;

the iPhoneSimulator platform does not currently support application-hosted tests (TEST_HOST set).

The culprit

/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Tools/RunPlatformUnitTests

The workaround(s)

Xcode Plugin

• https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin

• Specify provisioning profile

• Keychain access

• Signed IPA

Xcode Plugin

• Recommended xcodebuild arguments:

• GCC_SYMBOLS_PRIVATE_EXTERN=NO

• COPY_PHASE_STRIP=NO

• Allows test bundle to link with Release build symbols

Signing with the Plugin

Tip #3

Prefer Scripts over CI Plugins

• More flexibility

• More resilient to Apple’s changes

• More power to adapt and expand (specificity)

• Less coupling to the specific CI server type

• You can version control the settings easier

xcodebuild

xcodebuild \ clean build

Tip #4

Be specific with xcodebuild options

xcodebuild

xcodebuild \ -scheme SOLID \ -target SOLID \ -configuration Debug \ -arch i386 \ -sdk iphonesimulator \ clean build

xcodebuild

Tip #5

Specify Alternate Output Locations

• Don’t rely on Xcode’s cryptic locations

• This will make your scripts easier to manage

# Build location for Instruments test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATIONmkdir -p $CI_BUILD_DIR

xcodebuild \ ... \ CONFIGURATION_BUILD_DIR=$CI_BUILD_DIR \ clean build

# Build location for Instruments test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATIONmkdir -p $CI_BUILD_DIR

# Build location for CI test runs. CI_BUILD_DIR=/tmp/SOLID_AUTOMATIONmkdir -p $CI_BUILD_DIR

Tip #6

Ensure Return Codes

• If the tool you are invoking in your script doesn’t return an exit code indicating pass/fail, it’s up to you.

• Unix-friendly 0 (success) or 1 (failure)

Tip #7

Test against multiple SDKs

-sdk iphonesimulator5.0-sdk iphonesimulator5.1-sdk iphonesimulator6.0

Other Tools to Know About

• xcode-select (manage xcode path)

• Overridden by DEVELOPER_DIR

• xcrun (find and run dev tools)

• ibtool (for Interface Builder files)

• /Applications/Xcode.app/Contents/Developer/usr/bin/

• Variables in Xcode

• “Build Settings Reference”

OCUnit

Other Related Tools

Other Related Tools• OCMock

• OCMockito

• OCHamcrest

• Expecta

• TinyMock

• LRMocky

• OCUnit2JUnit

• OCUnitReport

• Kicker

• xcodebuild-rb

• xcodearchive

• XcodeTest

• AutoBuild

• xcodeproj (CocoaPods)

• Circle?

• Simon?

• ios-maven-plugin

• Ceedling

• XcodeCoverage

• JMRTestTools

• Nocilla

• OHHTTPStubs

OCUnit Alternatives

• Kiwi

• Cedar

• OCDSpec

• C++ Automated Test Cases in Headers (CATCH)

• Objective-Shoulda

• Specta

Tip #8

OCUnit2JUnit for Test Results

• ocunit2junit.rb

• https://github.com/ciryon/OCUnit2JUnit

• Converts output to JUnit format for easier transformation to HTML

UIAutomation

Other UIAutomation Tools

• tunup_js

• jasmine-iphone

• uiautomation-jasmine-iphone

• Bwoken

• Zucchini Framework

• WaxSim

• ios-sim

• ios-sim-locale

• iphonesim

• ui-auto-monkey

• ui-screen-shooter

• uiautomation-rb?

UIAutomation Alternatives

• Frank

• (iCuke)

• KIF

• Calabash

• UISpec

• Bromine

• Objective C Slim (ocslim)

• AutomationKit

• Sikuli

• MonkeyTalk (formerly FoneMonkey)

• Plus other commercial tools, such as Telerik Test Studio for iOS

Tip #9

PListBuddy is Your Buddy

/usr/libexec/PlistBuddy

Tip #10

Always quit the simulator

• killall

• killall -m -KILL "iPhone Simulator"

• AppleScript

• osascript -e 'tell app "iPhone Simulator" to quit'

Tip #11

Use Jonathan Penn’s AutomationExample

https://github.com/jonathanpenn/AutomationExample

Code Coverage

“Risk coverage”

Coverage Tools

• gcov

• GCC’s coverage tool

• GUIs for gcov

• LCOV (HTML)

• CoverStory (Mac app)

• ocov?

Tip #12

gcovr for Coverage Automation

Produces Cobertura-formatted output from gcov data

Tip #12.1

Enable Test Coverage

Static Analysis

• Clang

• http://clang-analyzer.llvm.org/

• scan-build

Tip #13

Symbolic links can be your ally

Avoid scan-build Painsudo ln -s \

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/arc/

libarclite_iphonesimulator.a \/usr/lib/arc/libarclite_iphonesimulator.a

What about test build deployment?

Tip #14

Use TestFlight

• testflightapp.com

• OTA deployment

• SDK for more features

• Free!!

TestFlight

• Web UI

• Desktop App

• REST API

• Jenkins Plugin

TestFlight Alternatives

• HockeyApp

• http://hockeyapp.net

• Also has a Jenkins plugin (forked)

• BetaBuilder for iOS

• http://www.hanchorllc.com/betabuilder-for-ios/

• More players in this space, esp. for enterprises...

TestFlight Config

• API Token and Team Token

• Need a signed IPA

• They get an email with a link for OTA installation!

Tip #15

Protect Your Tokens

• API Token & Team Token

• Keep them out of scripts

• Instead, define them in Jenkins (and leverage Jenkins security)

• 'Configure System'

• 'Global properties' section

• Or use the plugin (same parameters)

TestFlight Plugin

TestFlight Script

Deployment

Q: Once you have a nightly job to build, unit test, sign an IPA and deploy it to QA,

what’s next?

Q: Once you have a nightly job to build, unit test, sign an IPA and deploy it to QA,

what’s next?

A: Sleeping easy.

Documentation

• Appledoc

• Doxygen

• HeaderDoc

DocSets

Appledoc

Doxygen

Tip #16

Don’t stop there!

Other Goodies

• Ensure your site or API is up

• Scan your code for TODOs and FIXMEs

• Get trend reports for lines of code (SLOCCount w/ sloc2html)

• Scan for duplicated blocks of code (CPD, Simian, etc.)

• Use agvtool (Apple-Generic Versioning Tool) to increment your build number

Still Missing

• Cyclomatic Complexity

• Coding convention/Style checker

• Code Formatter

• Uncrustify, UniversalIndentGUI

• UML Generation

• via CLI, as opposed to OmniGraffle (AppleScript?)

• ER Diagram from Core Data schema

Tip #n

Sources• Slides: http://www.slideshare.net/muncman

• http://blog.carbonfive.com/category/mobile/

• http://www.stewgleadow.com/

• http://lifeandcode.net/

• http://longweekendmobile.com/blog/

• http://longweekendmobile.com/2011/04/17/xcode4-running-application-tests-from-the-command-line-in-ios/

• http://qualitycoding.org

• et cetera

Thank you!

• CodeMash organizers

• CodeMash sponsors

• and YOU!

Thank you!

• CodeMash organizers

• CodeMash sponsors

• and YOU!

Questions?

Ci for i-os-codemash-01.2013

Technology