Recovering Configuration Information for Configurable Systems via Symbolic
Execution
Justin Gorham, Xusheng Xiao, Tao XieNorth Carolina State Universityjwgorham,xxiao,[email protected]
Modern Software Systems
• Designed to be highly customizable– Extensibility– Flexibility– Portability
Configurable Systems
• A software system with mechanisms in place for implementing preplanned variations in system behavior
• Characterized by configuration options• Each configuration option has a configuration
domain
Formal Definition
• Configuration option – A label that represents a dimension of a system that can be set
• Configuration domain – The set of possible values that can be selected for a configuration option
• ‘configuration’: - a mapping of configuration options to a subset of their respective domains to create an instantiation of a configurable system
Example: Microsoft Word
option
Example: Microsoft Word
option domain
Configuration Binding Time
• Early Feature Binding– Software Product Lines• Boeing: Bold Stroke Avionics Software Family• Nokia: Mobile Phones• Toshiba: Software Product Line for Electric Power
Generation Plant Monitoring and Control
• Dynamically Reconfigurable Systems– NASA’s Deep Space 1 Remote Agent software
• User Configurable Systems
Runtime User Configurable System
• Configuration options are assigned at runtime• Obtain values typically through configuration
files and command lines• Options typically bind at startup time or at an
early stage of runtime
Example: (win)grep
Command Line Options•/h|/H - printing a usage help explanatory text;•/c - print a count of matching lines for each input file;•/i - ignore case in pattern;•/l - print just files (scanning will stop on first match);•/n - prefix each line of output with line number;•/r - recursive search in subdirectories;•/E:reg_exp - the Regular Expression used as search pattern. •/F:files - the list of input files. The files can be separated by commas as in
Example: JPF
• Uses a dictionary object to parse a hierarchical set of property files– Site Properties– Project Properties– Application Properties
• Primarily uses a <key> = <value> style of configuration
JPF: Site Properties
• site.properties file is machine specific and not part of any JPF project
• Each project is listed as a <name>=<directory> pair
JPF: Site Properties
• site.properties file is machine specific and not part of any JPF project
• Each project is listed as a <name>=<directory> pair
jpf-core = ${user.home}/projects/jpf/jpf-corejpf-shell = ${user.home}/projects/jpf/jpf-shell jpf-awt = ${user.home}/projects/jpf/jpf-awt ... extensions=${jpf-core},${jpf-shell}
JPF: Project Properties
• Each JPF project contains a jpf.properties file in its root directory
• defines the paths that need to be set for the component to work properly
JPF: Project Properties
1. <project-name>.native_classpath2. <project-name>.classpath3. <project-name>.test_classpath 4. <project-name>.sourcepath
JPF: Project Properties
1. <project-name>.native_classpath2. <project-name>.classpath3. <project-name>.test_classpath 4. <project-name>.sourcepath
jpf-aprop = ${config_path}
#--- path specifications jpf-aprop.native_classpath = build/jpf-aprop.jar;lib/antlr-runtime-3.1.3.jarjpf-aprop.classpath = build/examples jpf-aprop.test_classpath = build/tests jpf-aprop.sourcepath = src/examples
#--- other project specific settingslistener.autoload=${listener.autoload},javax.annotation.Nonnull,...listener.javax.annotation.Nonnull=gov.nasa.jpf.aprop.listener.NonnullChecker ...
JPF: Project Properties
1. <project-name>.native_classpath2. <project-name>.classpath3. <project-name>.test_classpath 4. <project-name>.sourcepath
jpf-aprop = ${config_path}
#--- path specifications jpf-aprop.native_classpath = build/jpf-aprop.jar;lib/antlr-runtime-3.1.3.jarjpf-aprop.classpath = build/examples jpf-aprop.test_classpath = build/tests jpf-aprop.sourcepath = src/examples
#--- other project specific settingslistener.autoload=${listener.autoload},javax.annotation.Nonnull,...listener.javax.annotation.Nonnull=gov.nasa.jpf.aprop.listener.NonnullChecker ...
JPF: Application Properties
• Used to tell JPF what main class it should start to execute
• Stored in *.jpf properties files that are part of the test projects
• target setting defines the main class of SUT• Also define target_args along with other JPF
properties that define how the application is to be checked
JPF: Application Properties
• Used to tell JPF what main class it should start to execute
• Stored in *.jpf properties files that are part of the test projects
• target setting defines the main class of SUT• Also define target_args along with other JPF
properties that define how the application is to be checked
#--- dependencies on other JPF projects@using = jpf-awt@using = jpf-shell
#--- what JPF should runtarget = RobotManager
#--- other stuff that defines how to run JPFlistener+=,.listener.OverlappingMethodAnalyzer
shell=.shell.basicshell.BasicShellawt.script=${config_path}/RobotManager-thread.escg.enumerate_random=true...
Scenario
• Developer/Testers– QA to validate configurations– Software Configuration Space Explosion
• End Users
Software Configuration Space Explosion
• Example– 20 configuration options– The domain for each option is either ‘true’ or
‘false’– 220 = 1,048,576 possible configurations– Add another option, configuration space doubles
• Infeasible to test a large configuration space
Motivation
• Extract configuration information for a runtime user configurable system– How to extract available configuration options– How to extract constraints among options and
their values• domain of a configuration option• relationships among options and/or their values
– How to extract runtime behavior of configurations• Fault due to interaction
Extracting Configuration Information
• Manually search through source code– Expensive, error prone…
• Static analysis– Cannot capture runtime behavior of configurtions
• Combinatorial Testing– Black Box, Scalability Issues
• Symbolic Execution
Thanks to Dr. Tao Xie for reusing his slide
Symbolic Execution
Code to generate inputs for:
Constraints to solve
a!=null a!=null &&a.Length>0 a!=null &&a.Length>0 &&a[0]==1234567890
void CoverMe(int[] a){ if (a == null) return; if (a.Length > 0) if (a[0] == 1234567890) throw new Exception("bug");}
Observed constraints
a==nulla!=null &&!(a.Length>0)a!=null &&a.Length>0 &&a[0]!=1234567890
a!=null &&a.Length>0 &&a[0]==1234567890
Data
null
{}
{0}
{123…}a==null
a.Length>0
a[0]==123…T
TF
T
F
F
Execute&MonitorSolve
Choose next path
Done: There is no path left.
Negated condition
SE with Configurable Systems
• Using Symbolic Evaluation to Understand Behavior in Configurable Software Systems
• Reisner, E; Song, C; Ma, K; Foster, J; Porter A
Figures on this page from “Using Symbolic Evaluation to Understand Behavior in
Configurable Software Systems“
SE with Configurable Systems
Figures on this page from “Using Symbolic Evaluation to Understand Behavior in
Configurable Software Systems“
SE with Configurable Systems
Figures on this page from “Using Symbolic Evaluation to Understand Behavior in
Configurable Software Systems“
SE with Configurable Systems
• Required to know configuration options in advance
• Program inputs may interact with configuration options
Parameterized Unit Test
Parameterized Configuration Mechanism
Proposed Solution
Unit TestsCode Under
Test
Configuration Mechanism
Inputs
Option Value
Output: Test Suite with concrete
inputs
Proposed Solution
PUTsCode Under
Test
ParameterizedConfiguration Mechanism
Output: Test Suite with concrete inputs and
assumptions on configuration options
Our Approach
Implementation
Challenges
• Object Creation and Data Transformation• Static Configuration Options
Future Directions
END
• Questions