Garbage Collection

Introduction, Methods& Java implementations

POP QUIZ #1

What is the first language runtime with Garbage Collection?

POP QUIZ #1

What is the first language runtime with Garbage Collection?

LISPBy John McCarthy in 1959

This talk is about?1. Background on Memory Management to understand

what GC is expected to do

2. Methods to do GC & Implications

3. Different implementations of the methods

4. Finally, in Java

Thought experiment

Imagine there is no GC (think of C)

Thought experimentImagine there is no GC (think of C)

malloc(size) {

POP QUIZ #2

What does it do?

}

Thought experimentImagine there is no GC (think of C)

malloc(size) { /*

1. Check if any of the already allocated memory is free enough1. If yes, mark it as used and

return2. If no, see if OS can give

more memory1. If yes, return that2. If no, fail

*/}

Thought experimentImagine there is no GC (think of C)

malloc(size) { /*

1. Check if any of the already allocated memory is free enough1. If yes, mark it as used and

return2. If no, see if OS can give

more memory1. If yes, return that2. If no, fail

*/}

Free Memory List

maintained by C runtime

Thought experimentImagine there is no GC (think of C)

malloc(size) { /*

1. Check if any of the already allocated memory is free enough1. If yes, mark it as used and

return2. If no, see if OS can give

more memory1. If yes, return that2. If no, fail

*/}

Free Memory List

maintained by C runtime

free(_ptr) { /*

1. Add this memory to free list */}

POP QUIZ #3

What usually happens if malloc() and free() just do what’s described here?

Allocate & Free only

Free Memory List

M XS S XXL L X

S XL

maintained by C runtime

M XS S XXL

L XS XL

smallJob() {free( doSomething( malloc(XS) ));

}

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

HeapMemory

XS

Free memory with sizes >= ‘s’ for small Used

memoryThread

bulkPricing(

)

{ … … malloc

(XXXL)

…}

Allocate & Free only

smallJob() {free( doSomething( malloc(XS) ));

} CRASH – NO

MEMORY

Free Memory List

M XS S XXL L X

S XL

maintained by C runtime

M XS S XXL

L XS XL X

SXS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

HeapMemory

XS

XS

Free memory with sizes >= ‘s’ for small Used memoryFree memory with sizes >=

‘s’ for small Used memory

Thread

bulkPricing(

)

{ … … malloc

(XXXL)

…}

Allocate & Free + Compaction

smallJob() {free( doSomething( malloc(XS) ));

}

SUCCESSFULFree Memory List

MXS S XXL L X

S XL

maintained by C runtime

M

XS S XXL

L XS XL X

SXS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

XS

HeapMemory > XXXL

> XXXL

References to moved allocations must point to the new address – Not cheap

Free memory with sizes >= ‘s’ for small Used

memoryThread

Garbage Collection – Defined

An automatic memory management system that efficiently allocates memory at the point of need without requiring applications to free-up the

memory explicitly. To do so, it should:

1. Should allocate memory when application needs2. It should free up unused memory when necessary3. It should keep memory in a good shape to avoid

fragmentation

Garbage Collection - Methods

REFERENCE COUNTING Every object maintains the count of number of references to itself

This counter is incremented whenever a new reference is created for an object (e.g. Object o = x;)

This counter is decremented whenever a reference for an object goes out of ‘scope’

All objects with count = 0 can be freed Cyclic references must be detected and dealt with OR will not be collected (developers’ headache)

TRACING

When there is need for memory, Find all the objects that are not usable by the application

Free them The process of figuring out what is not useable (or useable) is called ‘tracing’

Garbage Collection - Methods

REFERENCE COUNTING Every object maintains the count of number of references to itself

This counter is incremented whenever a new reference is created for an object (e.g. Object o = x;)

This counter is decremented whenever a reference for an object goes out of ‘scope’

All objects with count = 0 can be freed Cyclic references must be detected and dealt with OR will not be collected (developers’ headache)

TRACING

When there is need for memory, Find all the objects that are not usable by the application

Free them The process of figuring out what is not useable (or useable) is called ‘tracing’

POP QUIZ #4What’s this?

Garbage Collection - Methods

REFERENCE COUNTING Every object maintains the count of number of references to itself

This counter is incremented whenever a new reference is created for an object (e.g. Object o = x;)

This counter is decremented whenever a reference for an object goes out of ‘scope’

All objects with count = 0 can be freed Cyclic references must be detected and dealt with OR will not be collected (developers’ headache)

TRACING

When there is need for memory, Find all the objects that are not usable by the application

Collect (=free) them The process of figuring out what is not useable (or useable) is called ‘tracing’

Find Garbage = {}public class ListTools {public static final int MAX_SIZE = Integer.MAX_VALUE/2;......public static T smallestElement(List<T> l, Comparator<? super T> c) {fail(l.size() > MAX_SIZE)

List<T> local = new ArrayList<>(l.size());Collections.copy(local, l);local.sort(c);return local.get(0);

}}

executing

Find Garbage = {}

executing

public class ListTools {public static final int MAX_SIZE = Integer.MAX_VALUE/2;......public static T smallestElement(List<T> l, Comparator<? super T> c) {fail(l.size() > MAX_SIZE)

List<T> local = new ArrayList<>(l.size());Collections.copy(local, l);local.sort(c);return local.get(0);

}}

Find Garbage =

executing

{ }?public class ListTools {public static final int MAX_SIZE = Integer.MAX_VALUE/2;......public static T smallestElement(List<T> l, Comparator<? super T> c) {fail(l.size() > MAX_SIZE)

List<T> local = new ArrayList<>(l.size());Collections.copy(local, l);local.sort(c);return local.get(0);

}}

Find Garbage =

executing

{ }__in red__public class ListTools {public static final int MAX_SIZE = Integer.MAX_VALUE/2;......public static T smallestElement(List<T> l, Comparator<? super T> c) {fail(l.size() > MAX_SIZE)

List<T> local = new ArrayList<>(l.size());Collections.copy(local, l);local.sort(c);return local.get(0);

}}

Garbage defined!{Root Set}

All objects reachable from current thread ‘stacks’All objects reachable from current threads (= ThreadLocals)Global references (= static, registers)JNI references (= let’s not bother about this!)

{Reachable Set} aka {Live Set}{Root Set} + All objects reachable from {Root Set}

{Garbage}{Allocated Objects} – {Reachable Set}

HEAP

Free Garbage: Method Mark & Sweep{Root Set}

A

B

C

HEAP

Free Garbage: Method Mark & Sweep{Root Set}

A

B

C

HEAP

Free Garbage: Method Mark & Sweep{Root Set}

A

B

C

HEAP

Free Garbage: Method Mark & Sweep{Root Set}

A

B

C

Free Garbage: Method Mark & Sweepgc() {

suspendAllThreads();markReachableSet();ALL_OBJECTS.stream().filter(o -> o.isMarked()).forEach(o -> FREE_LIST.add(o)); //sweepresumeAllThreads();

}

Free Garbage: Method Copy Collect

HALF THE OTHER HALF{Root Set}

A

B

C

Free Garbage: Method Copy Collect

HALF THE OTHER HALF{Root Set} A

B

C

Free Garbage: Method Copy Collect

HALF THE OTHER HALF{Root Set} A

C

B

Free Garbage: Method Copy Collect

THE OTHER HALF HALF

A

B CNothing to free, just treat it empty

Free Garbage: Method Copy Collect

gc() {suspendAllThreads();//Note: Only half of is active at any given

time//Note: Copy procedure includes updating refscopyReachableSetToTheOtherHalf();toggleHalf();resumeAllThreads();

}

POP QUIZ #5

Spot as many differences (data structures, performance characteristics etc.) between the

two methods?

Free Garbage: Method differences

Mark & Sweep

GC should maintain ALL_OBJECTS & FREE_LIST

Complexity linear to heap size (for sweeping)

Better suited for long staying objects (i.e. less garbage created)

Copy Collect

50% of available memory usable Always compact Complexity linear to {Reachable Set} Allocation is 0 cost (return curr += size)

Better suited for “mostly” garbage scenarios (and unsuited for long staying objects)

POP QUIZ #6

‘concurrent’ vs ‘parallel’ GC?

Free Garbage: Method Implementations

Stop The World

Time

Parallel Concurrent

Thread

GC

Incremental

Most of these techniques are orthogonal and so can be combined together (which is normally the case in standard implementations)

Typical Characteristics of Objects

“Most Objects die young. And the few that survive, live

long”

Age

Death Rate

Image derived from Oracle

GC: In java(7+)

Generational Garbage Collection

Performs GC based on Object’s age (= # of GCs survived)

Young objects (most of whom die young, i.e. high garbage rate) are ‘copy collected’ aka ‘MINOR GC’

Old (‘Tenured’) objects (most of whom live forever) are ‘marked and swept’ a.k.a ‘MAJOR GC’

HEAP: In java(7+)

EDEN SPACEAll new objects are

allocated here. Once full, live objects are copied to

active survivor.

HALF THE OTHER HALF

SURVIVOR SPACES

Young objects are ‘copied’ into active survivor spaces. As the survivor space is filled up, they are copy collected to the ‘other’.

These are long living objects. They are determined by how many minor GCs they survived. This region is

marked & swept. OLD GENERATION

HEAP: Object Ageing – Creation

EDEN SPACE

HALF THE OTHER HALF

SURVIVOR SPACES

OLD GENERATION

new Object()

HEAP: Object Ageing – Survival

HALF THE OTHER HALF

EDEN IS GCdEDEN SPACE

SURVIVOR SPACES

OLD GENERATION

HEAP: Object Ageing – Survival

HALF THE OTHER HALFCOPY COLLECTED SURVIVAL_COUNT++

EDEN SPACE

SURVIVOR SPACES

OLD GENERATION

HEAP: Object Ageing – Survival

HALF THE OTHER HALFCOPY COLLECTED SURVIVAL_COUNT++

EDEN SPACE

SURVIVOR SPACES

OLD GENERATION

HEAP: Object Ageing – Promotion

HALF THE OTHER HALF

PROMOTE

D

EDEN SPACE

SURVIVOR SPACES

OLD GENERATION

HEAP: Governed by

HALF THE OTHER HALF

EDEN SPACE

SURVIVOR SPACES

OLD GENERATION

-Xms <= size <= -Xmx

-XX:NewRatio

-XX:SurvivorRatio

-XX:MaxTenuringThreshold

GC: Implementations

-XX:+UseSerialGC

SERIAL GC-XX:+UseParallelGC

PARALLEL GC

-XX:+UseConcMarkSweepGC

CONCURRENT GC

-XX:+UseG1GC

G1 GC

Serially performed

For apps requiring small heaps (short living apps)

For embedded apps

Multiple threads are used to mark and sweep

Application is halted during this procedure

High throughput with larger GC pauses

Mark and Sweep is performed without stopping the application for most part

Trades off throughput (as object graph could change) for lower GC pauses

Garbage First Collector

Meant for Very Large Heaps (6G to 50+G)

Regional in nature and almost always compact

Reduced GC Pauses

Further to-dos/ references(=things omitted ) Key core concepts to be explored (based on interest):

Dealing with old generation to young generation references (“Cards”, “{Remembered Set}”) Concurrent Mark, Sweep & Compaction GC Tuning G1 GC

Tri-color Mark & Sweep (used by Go Lang):“On-the-fly garbage collection: an exercise in cooperation”, Dijkstra et. al., ACM, Volume 21 Issue 11, Nov. 1978, New York, NY, USA

Pauseless GC:“C4: The Continuously Concurrent Compacting Collector”, Tene et. al., ACM, June 4–5, 2011, San Jose, California, USA

Shenandoah GC:“JEP 189: Shenandoah: An Ultra-Low-Pause-Time Garbage Collector”, OpenJDK

A fantastic introductory reference:“Understanding Java Garbage Collection and what you can do about it”, a talk by Gil Tene