Configuration
Yanzhi Xin, Xiping Peng
Motivation
Why configuration tuning:
● Optimal configuration significantly improve system performance
● Improve the performance without application code modification
MotivationConfiguration tuning challenges:
● Difficult to achieve optimal config, Misconfig happens often:
1. Increased system complexity, potentially several hundred configurable parameters
2. Parameters increase with every update
Common goals
● Achieving best performance
● Reduce the tuning complexity
● Tune efficiently and effectively
ConfAdvisor: Performance-centric Configuration Tuning Framework for Containers
● Tuning monolithic systems already a headache
● To make matters worse: Microservices architecture
1. More complex, more cascadings
2. Different application with various version in each container
3. Deployment environment
Observation
● Existing solutions:
1. Config management tools:
for servers, not for containers, not performance-centric
2. Performance sensitive config tuning system:
e.g. Ottertune, works well independently, but not with container
environment
3. Misconfiguration validation language:
huge learning curve, not performance-centric
Observation
Configuration file Example
ConfAdvisor
● Insights
1. Validating default config in image
ConfAdvisor● Insights continued
2. Throughout Container life cycle config tuning:
ConfAdvisor● Insights continued
3. Accepts both customized declarative and imperative rules:
Declarative: more general, easy to understand rules
Imperative: more complex, detailed, direct specific rules
ConfAdvisor design
ConfAdvisor implementation
L1/L2 Analysis
L3 AnalysisAdvisor Framework: exposing metrics
Plugins:
Analyze the metrics and loads specific rules;
Allow User Defined Functions
What-if Engine:
Consume the rules and generate final advices
Analytic plugin exampleImportant predicates: “What-if” and “advice”
Constraint:
Currently only support 6 softwares(Liberty,
Nginx, Node.js, MongoDB, Redis, and
Cassandra)
What-if rule engine“what-if” “advice”
ConfAdvisor evluationConfig & metrics
crawling(Runtime overhead) :
Config Advice with L1/L2
analysis:
Example: YCSB workload
on Cassandra container on
Kubernetes
Default config could not
run a workload due to a
JVM container related
parameter
Config Advice with
L3 analysis:
Automatic Database Management System Tuning Through Large-scale Machine Learning
Background
● Why Knobs Are Annoying
● What Makes The Situation Worse
As databases and applications grow in both size and complexity, optimizing a DBMS (tuning) to meet the needs of an application has surpassed the abilities of humans.
Given this, many organizations resort to hiring expensive experts to configure the system’s knobs for the expected workload.
Motivation
● Dependencies● Continuous Settings● Non-reusable Configurations● Tuning Complexity
OtterTune: Automatic Database Tuning Tool
Part 1: Controller
● The first part is the client-side controller that interacts with the target DBMS to be
tuned. It collects runtime information from the DBMS using a standard API (e.g.,
JDBC), installs new configurations, and collects performance measurements.
Part 2: Manager
● The second part is OtterTune’s tuning manager. It receives the information collected
from the controller and stores it in its repository with data from previous tuning
sessions. This repository does not contain any confidential information about the
DBMSs or their databases; it only contains knob configurations and performance
data.
Pipeline
Pipeline
(1) select the most impactful knobs,
(2) map previously unseen database workloads to known workloads,
(3) recommend knob settings.
Workload Characterization
1. Statistics Collection2. Pruning Redundant Metrics
Workload Characterization
Identifying Important Knobs
Feature Selection with Lasso
OtterTune uses a popular feature selection technique for linear
regression, called Lasso, to expose the knobs that have the
strongest correlation to the system’s overall performance.
Why LassoLasso works by adding an L1 penalty that is equal to a constant λ times the sum of absolute weights to the loss function.
● Shrinkage
Shrink small weights towards zero (discard irrelevant features)
● Selection
Automatically select more relevant features (those with non-zero weights)
Dependencies in Feature Selection
Many configuration knobs are non-independent and changing one may affect another. We can capture such dependencies between knobs by including polynomial features in the regression.
Automatic Tuning● Step 1: Workload Mapping
Workload repository
Input workload
Mapped WorkloadRelevant metrics
Automatic Tuning● Step 2: Configuration Recommendation
OtterTune uses Gaussian Process (GP) regression and gradient descent to recommend configurations
Results
The results show that OtterTune produces configurations that achieve up to 94% lower latency compared to their default settings or configurations generated by other tuning advisors. OtterTune generates configurations in under 60 min that are comparable to ones created by human experts.
Limitations
● Administrative privileges
● Restarting the DBMS is often necessary
Background
Ancestry relies on Kubernetes to quickly integrate and deploy applications across Ancestry’s website, which receives 50-million visitors a month, and generates more than a billion dollars in revenue.
To get optimum performance out of Ancestry’s cloud applications, Ancestry employed artificial intelligence for continuous optimization of the application runtime environment.
Solution
Ancestry used AI to cut the resources of one application by more than 50 percent, with zero drop in performance. In this instance, Ancestry has been able to get two times the performance out of Kubernetes for every dollar spent.
Continuous Optimization
Comparisons● ConfAdvisor
General Microservices config tuning framework, sort of hard coded
● Ancestry
Also targets on microservices architecture, specific to their own product
● OtterTune
Also based on machine learning and AI concepts, DBMS specific framework
Comparisons
Future directions:
● More AI/Machine learning involved● More Microservice oriented software config tuning tools● More Microservice tuning integration framework
Q & A