Date post: | 11-Feb-2017 |
Category: |
Documents |
Upload: | nguyendien |
View: | 219 times |
Download: | 1 times |
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Managing Dynamic Memory Allocations in aCloud through Golondrina
4th International DMTF Academic Alliance Workshop onSystems and Virtualization Management:
Standards and the Cloud
Alexander Pokluda, Gastón Keller and Hanan Lutfiyya
Department of Computer ScienceUniversity of Western Ontario
October 29, 2010
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Introduction
AbstractWe present a policy-based framework that supports automateddynamic resource management in a virtualized environment. Thisallows for flexibility in how resources are allocated. We show how thisframework can be used to support memory management through theuse of migration and making local resource adjustments.
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Accommodating Peak Demand with Over-ProvisioningModern data centres are comprised of tens of thousands ofservers, and perform the processing for many Internet businessapplicationsUnit of allocation is typically one physical machineOne estimate is that servers are over-provisioned by more than500% in order to deal with peaks in demand
!"#$%&
'()*
%&
+,$-).$%'()*/)0)1"23'()*
!"#$%&'()"*'+","'-).'"/'0/,&./&,'1$$%23",2)/
!"#"$%&'()*+",*-./"0
!"#$%&#'()%*'+),-%$#*'.)'+%-/0.1'2)$'%3'43.#$3#.'!--50/%.0)3
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Over-Provisioning Leads To Underutilization
Over-provisioning means underutilizationOne approach to increasing utilization is server consolidation,which consists of hosting multiple servers on one physicalmachineServer consolidation is possible through virtualizationVirtualization provides an interface to the actual hardware thatcan support a number of virtual machines that have applicationsoftware installed on them
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Strategic Placement of Virtual MachinesCan Increase Utilization
If virtual machines are placed on physical machines based onpeak demand, physical machines may still be highly underutilizedIf virtual machines are placed on physical machines based onaverage demand, the virtual machines may compete for thesame resources when demand increasesMuch of the work in dynamic resource provisioning for Internetapplications involves constructing a performance modelPerformance models are used to periodically determine optimalplacements of virtual machines as part of periodic maintenance
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Contributions
Contributions1 Memory management using the OpenVZ virtualization platformwas studied
2 A simple heuristic for identifying memory stress situations wasdeveloped using a black-box approach
3 A simple heuristic for adjusting memory allocations wasdeveloped and experimented with
4 A flexible framework that supports the separation ofdecision-making from the specific virtualization technology wasdesigned
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Our Solution
An important aspect of amanagement framework is theability to determine theappropriate action in response toworkload fluctuations.
The appropriate action dependson strategies that can berepresented through policies.
We present a policy-basedmanagement framework that isdepicted in the figure to the right.
Figure: Golondrina’s architecture
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
System Behaviour is Configured Through PoliciesA policy associates an event with one or more rules of the formif conditions then actions
oblig NotifyMemoryStressViolation{target DepartmentContainerssubject Manageron memoryStress(containerID,
hostNameID,freePhysicalPages);
do IncreaseMemory(containerID,hostNameID)
when freePhysicalPages > N;do t = ChooseTarget()when freePhysicalpages < N ->MigrateContainer(containerID,
hostNameID,t);}
Example: A sample memory-stressviolation policy
oblig ConfigurePolicy{subject Manageron ConfigurePluginRequest(k,p);do ConfigureChoseTargetPlugin(p)when k = "ChooseTargetLocation";
}
Example: A sample configurationpolicy
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
System Behaviour is Configured Through PoliciesA policy associates an event with one or more rules of the formif conditions then actions
oblig NotifyMemoryStressViolation{target DepartmentContainerssubject Manageron memoryStress(containerID,
hostNameID,freePhysicalPages);
do IncreaseMemory(containerID,hostNameID)
when freePhysicalPages > N;do t = ChooseTarget()when freePhysicalpages < N ->MigrateContainer(containerID,
hostNameID,t);}
Example: A sample memory-stressviolation policy
oblig ConfigurePolicy{subject Manageron ConfigurePluginRequest(k,p);do ConfigureChoseTargetPlugin(p)when k = "ChooseTargetLocation";
}
Example: A sample configurationpolicy
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Collecting Resource Usage DataMonitoring requires someform of instrumentation, whichwe refer to as a Sensor,specific to each managedresourceA Sensor Agent provides astandard interface to sensorsThe set of sensors on ahardware node is managedby a Sensor ManagerAn entity that changes thesystem is referred to as anActuator; similar to sensors,there is an Actuator Managerand Actuator Agent(not shown)
Figure: Golondrina’s architecture
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Collecting Resource Usage DataMonitoring requires someform of instrumentation, whichwe refer to as a Sensor,specific to each managedresourceA Sensor Agent provides astandard interface to sensorsThe set of sensors on ahardware node is managedby a Sensor ManagerAn entity that changes thesystem is referred to as anActuator; similar to sensors,there is an Actuator Managerand Actuator Agent(not shown)
Figure: Golondrina’s architecture
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Collecting Resource Usage DataMonitoring requires someform of instrumentation, whichwe refer to as a Sensor,specific to each managedresourceA Sensor Agent provides astandard interface to sensorsThe set of sensors on ahardware node is managedby a Sensor ManagerAn entity that changes thesystem is referred to as anActuator; similar to sensors,there is an Actuator Managerand Actuator Agent(not shown)
Figure: Golondrina’s architecture
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Collecting Resource Usage DataMonitoring requires someform of instrumentation, whichwe refer to as a Sensor,specific to each managedresourceA Sensor Agent provides astandard interface to sensorsThe set of sensors on ahardware node is managedby a Sensor ManagerAn entity that changes thesystem is referred to as anActuator; similar to sensors,there is an Actuator Managerand Actuator Agent(not shown)
Figure: Golondrina’s architecture
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Analyzing Resource Usage DataThe System Manager includes an Event Manager that receivessensor data that it analyzes to determine whether or not asignificant event has occurredSignificant events are identified by the evaluation of a conditionassociated with the event, e.g. memoryStressScore > 0.80The event detection server maintains a set of tuples in the form(e, p), where p represents a policy identifier and e represents theevent expression, for each operational condition policyThe evaluation of operational conditions can also be done bysensor agents
Figure: A subset of Golondrina’s architecture
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Underlying Resource Management FrameworkThe Golondrina prototype currently uses OpenVZOpenVZ is a Linux kernel modified to run isolated containers(virtual user-space environments)
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
How Memory is Allocated
A process may increase its memory consumpution by eitherexplicit requests for more memory (e.g. calls to malloc) or stackexpansionLike most current operating systems, OpenVZ allocates memoryin units of pagesWe use the term alloc to refer to the number of pages currentlyowned by a containerWe use the term allocUsed to refer to the number of currentlyowned pages that have been written to
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Many Per-Container Resource Limits ExistallocationGuatantee number of pages guaranteed to be grantedallocationBarrier soft limit on number of pages that may be grantedallocationLimit hard limit on number of pages that may be grantedA memory allocation request for n pages is guaranteed to succeed if
n + alloc <= allocationGuarantee
All memory requests will succeed if n is less than the number of freephysical pages and
n + alloc <= allocationBarrier
If n + alloc > allocationBarrier then only a high priority request willsucceed provided that n is less than the number of free physicalpages and
n + alloc <= allocationLimitEvery request beyond allocationLimit fails
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
How a Memory Stress is Detected
A memory stress score is calculated at periodic intervals for eachcontainer according to the following algorithm:1: x = failcntKillsi - failcntKillsi!12: y = failcntAllocationsi - failcntAllocationsi!13: if x > 0 then4: s0 = 1;5: end if6: if y > 0 then7: s1 = 1;8: end if9: s2 = min (1, allocUsedi/minMemoryGuarantee);10: s3 = min (1, alloci/allocationBarrier );11: stressScorei = max (s0, s1, s2, s3);
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
How a Memory Stress is Resolved
Assume a container’s memory stress score exceeds a threshold,identifying it as memory-stressed. The system will attempt to resolvethe memory stress by applying the first action below that will notdestabilize the system.
1 Increase the container’s memory limits, allowing it to accessmore memory on the current hardware node
2 Migrate the stressed container to another hardware node whereit can be given access to more memory
3 Other container-specific actionsIf a hardware node is memory stressed but no containers are, then anattempt is made to migrate the container that is using the largestamount of memory on that hardware node.
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Outline
1 Introduction
2 Architecture
3 Memory Stress Detection and Resolution
4 Prototype Implementation and Experimental Results
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
The Prototype Implementation and Test Environment
We have developed a prototype implementation calledGolondrinaGolondrina is written in Python and interfaces with OpenVZGolondrina has three types of actuators:
Memory-Adjustment ActuatorsMigration AcuatorsReplication Actuators
We tested the system using several containers spread acrossthree identical harware nodesEach container hosted a PHP-based website that was sentHTTP requests
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Several Experiments Were Used to Validate thePrototype
Experiment 0 The containers were given enough memory so that nomemory-stress occurred
Experiment 1 The conatiners started with the minimum amount ofmemory required and policies were disabled
Experiment 2 Similar to Experiment 1 except local memoryadjustments were enabled
Experiment 3 Similar to Experiment 2 excpet that local memoryadjustments and migrations were enabled
Exp Avg Min Max Std Dev Err % Throughput Fail Count0 448 166 2275 243.72 0.00 8.0 01 1082 124 55576 5817.40 2.22 3.6 10022 866 115 49214 4748.84 1.63 4.9 8093 370 137 4354 419.42 9.24 5.5 1143
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina
IntroductionArchitecture
Memory Stress Detection and ResolutionPrototype Implementation and Experimental Results
Conclusion
Conclusion
Early work in dynamic resource management with an emphasison memory was presentedThe work presented assumes monitoring of utilization occursperiodicallyThis information is used to adjust resource allocations inresponse to workload fluctuationsThis work is complimentary to work that focuses on serverconsolidationThe approach to adjusting memory limits was straightforward;future work will focus on adaptive resource control andincorporate information gained from experiments with theprototype
A. Pokluda, G. Keller and H. Lutfiyya Managing Dynamic Memory Allocations in a Cloud through Golondrina