Download - Linked to ArrayList: the full story

#Devoxx

LinkedList to ArrayListThe Full Story!

José Paumard

#Devoxx #ListJ9 @JosePaumard

It all begin…

• November 22nd 2015 by a tweet


It all begin…

• Then the discussion started


It all begin…

• And about 30 tweets later:


And did not ended there!

• And around midnight, 7th nov. 2016:

• Since this is the subject of this Uni, we will take more time…


Questions ?#ListJ9

Questions?#ListJ9


• This discussion ArrayList vs LinkedList is no troll…

Even if it looks like one!

• The question is :

Is one of these implementations better than the other?

• And there is an answer to this question!

And the conclusion is…



• To fully understand this answer, we need to understand many other things



• To fully understand this answer, we need to understand many other things

• And once we have answered it (almost) fully , manyother questions will arise!


1st part

• Algorithms

• Complexity

• Implementation

• CPU architecture

• « cache friendly »

• Slides, bullet points…


2nd part

• Implementation of Set, List et Map (with a surprise!)

• Live coding!

• Some more slides but not that many

But don’t worry, still bullet points


Where to begin?

• In fact there are several ways to begin…


Where to begin?


• Algorithms!


Where to begin?


• Algorithms!

• Implementation vs CPU architecture


Where to begin?


• Algorithms!

• Implementation vs CPU architecture

• Use cases


What do we know?

1) ArrayList is built on an array

2) LinkedList is built on a linked list

And we have results on this


Approach

• Take a close look at the basic operations on lists

• Create / modify:

Add an element at the end of at the beginning of a list

Removing an element by its index

Removing an element

The contains() method

The removeIf() and sort() methods


Approach


• Add:

List<String> list = ...

list.add("one"); // addlist.add(12, "one"); // insert


Approach


• Remove:


list.remove("one"); // removelist.remove(12); // remove by index


Approach


• Sort / removeIf:


list.sort(comparingBy(String::length)); // sortlist.removeIf(s -> s.length() > 10); // remove if


Approach


• Read operations:

Iterate

Read an element with its index

Building a Stream


Approach

• Take a close look at the basis operations on lists

• Read operations:


list.forEach(System.out::println); // list traversallist.get(12); // access by indexlist.stream(); // stream creation


ArrayList

• Built on an array

• Access to the nth element: almost free

• Adding an element: almost free, but…

• Insertion: shift the elements to the right

• Deletion: shift the elements to the left


ArrayList


• Access to the nth element: almost free

n


ArrayList



private void grow(int minCapacity) {int oldCapacity = elementData.length;int newCapacity = oldCapacity + (oldCapacity >> 1);elementData = Arrays.copyOf(elementData, newCapacity);

}


ArrayList



• Sometimes the array has to be enlarge

• But we can fix the size of the array when we build it


ArrayList



• Sometimes the array has to be enlarge

• Bytheway growing a HashSet is much more expensive…


ArrayList


• Insertion

n, one


ArrayList


• Insertion

public void add(int index, E element) {System.arraycopy(elementData, index, elementData, index + 1,

size - index);elementData[index] = element;size++;

}


ArrayList


• Insertion

one


ArrayList


• Suppression

n

X


ArrayList


• Suppression

public E remove(int index) {E oldValue = elementData(index);System.arraycopy(elementData, index+1, elementData, index,

numMoved);return oldValue;

}


ArrayList

• (Very) Early conclusion

• Accessing an element is fast!

• Adding is fast, if there is no overflow

• Random insert / delete: overhead to handle the « holes »


LinkedList

• Built on a double linked list

• Access to the nth element

• Adding an element at the end of the list

• Insertion

• Deletion


LinkedList

• Base structure

next prev

item

Node

next prev

item

Node

next prev

item

Node

first last


LinkedList

• Built on a double linked list

• Access to the nth element: run through all the elements

• Adding: free, just a modification of pointers

• Insertion: free, just a modification of pointers

• Suppression: free, just a modification of pointers


LinkedList

• Access to the nth element

Node<E> node(int index) {if (index < (size >> 1)) {

Node<E> x = first;for (int i = 0; i < index; i++)

x = x.next;return x;

}// same for last

}


LinkedList

• Access to the nth element: one has to visit all the elements and count them, until the nth is found

• We say that the complexity of this operation is « N »

• Or is O(n)


Notation O(N)

• This notation means that:

if we multiply the amount of elements to be processed by 2

then the number of operations will also be multiplied by 2


Notation O(N)

• The complexity of an algorithm is given by O(f(n))

• Example: f(n) = n2


Notation O(N)

• The complexity of an algorithm is given by O(f(n))

• Example: f(n) = n2

• In fact, it means that the exact number of operations is

f(n) = an2 + bn + g

• And when n reaches a given value

f(n) ~ n2


Notation O(N)

• All right, but processing data is not anly about maths

• In our applications, n has a value, the same for a, band g…

• And the theory might not apply very well to our use case


Complexity comparison

log2(n)

1 0

10 3

100 7

1 000 10

1 000 000 20



log2(n) n

1 0 1

10 3 10

100 7 100

1 000 10 1 000

1 000 000 20 1 000 000



log2(n) n n x log2(n)

1 0 1 0

10 3 10 30

100 7 100 700

1 000 10 1 000 10 000

1 000 000 20 1 000 000 20 000 000



log2(n) n n x log2(n) n2

1 0 1 0 1

10 3 10 30 100

100 7 100 700 10 000

1 000 10 1 000 10 000 1 000 000

1 000 000 20 1 000 000 20 000 000 too much!



« too much » means that on any CPU, running a thousand billion of operations will take at least 20mn

• So we need to parallelize this operations of a large number of CPU (at least a few thousands)

We need to change our algorithm

We are not running this on a single JVM


Lists complexity

• On the basic operations

ArrayList LinkedList

add(e) 1

add(index, e) 1

set(index, e) 1

remove(index) 1

iterator() 1


Lists complexity

• On the basic operations

ArrayList LinkedList

add(e) 1 1

add(index, e) 1 N

set(index, e) 1 N

remove(index) 1 N

iterator() 1 1


Lists complexity

• We still must be cautious…

• There are System.arrayCopy() in ArrayList that are not here in LinkedList

• Is it a g? Or something that grows with n? And how?


Lists complexity

• We still must be cautious…

• Example: if we consider add(index, e), isSystem.arrayCopy() more costly than going throughthe pointers of the LinkedList?

• Are there any hidden costs that we did not see?


Benchmarks!

• There is a standard tool (Java 9) to bench Java code : JMH

• A benchmark is an annotated class

• Maven can generate a « transformed » JAR

• Then we run this JAR to get the result of the bench


Benchmarks!

• Here is a « bench » class

@Warmup(iterations=5, time=10, timeUnit=TimeUnit.MILLISECONDS)@Measurement(iterations=10, time=10, timeUnit=TimeUnit.MILLISECONDS)@Fork(value=1,

jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms3g", "-Xmx3g"})@BenchmarkMode(Mode.AverageTime)@OutputTimeUnit(TimeUnit.NANOSECONDS)@State(Scope.Benchmark)public class Bench {

// bench}


Benchmarks!

• POM dependencies, version is 1.15

<dependency><groupId>org.openjdk.jmh</groupId><artifactId>jmh-core</artifactId><version>${jmh.version}</version> 

</dependency><dependency>

<groupId>org.openjdk.jmh</groupId><artifactId>jmh-generator-annprocess</artifactId><version>${jmh.version}</version><scope>provided</scope>

</dependency>


Benchmarks!

• In the class

@Param({"10", "100", "1000"})private int size;

@Param({LINKED_LIST, ARRAY_LIST})private String type;


Benchmarks!

• In the class

@Setuppublic void setup() {

switch(type) {case ARRAY_LIST :

list = IntStream.range(0, size).mapToObj(Integer::toString).collect(Collectors.toCollection(

() -> new ArrayList<String>(size))); break;// other cases

}


Benchmarks!

• Simple add operation:

Can trigger a System.arrayCopy() on ArrayList

Fast on LinkedList since there is a pointer to the end of the list

@Benchmarkpublic boolean simpleAdd() {

return list.add("one more");}


Benchmarks!

• Simple add with index:

Triggers at least one System.arrayCopy() one ArrayList

Slower on LinkedList since half of the list must be visited

@Benchmarkpublic boolean simpleAdd() {

list.add(size / 2, "one more");return true;

}


Benchmarks!

• Results for Simple Add

LinkedList ArrayList Big ArrayList

10 5,3 ns 3,6 ns 3,6 ns

100 5,3 ns 3,6 ns 3,7 ns

1 000 5,3 ns 3,7 ns 3,7 ns

1 000 000 5,5 ns 3,7 ns 3,7 ns


Benchmarks!

• Index Read :

Read operation in the middle of the list

We expect the worst performance for LinkedList

@Benchmarkpublic boolean indexRead() {

return list.get(size / 2);}


Benchmarks!

• Index Set:

Write operation in the middle of the list

We also expect the worst performance for LinkedList

@Benchmarkpublic boolean indexSet() {

return list.set(size / 2, "one more");}


Benchmarks!

• Results for Index Read

LinkedList ArrayList

10 5,5 ns 2,8 ns

100 37,9 ns 2,8 ns

1 000 596 ns 2,8 ns

1 000 000 3,4 ms 2,8 ns


Benchmarks!

• Results for Index Set


10 5,5 ns 3,4 ns

100 36,0 ns 3,5 ns

1 000 602 ns 3,5 ns

1 000 000 3,4 ms 3,5 ns


Benchmarks!

• Iteration :

Let us iterate over the whole list

1st pattern: iterator

@Benchmarkpublic long iterate() {

long sum = 0;for (String s : list) {

sum += s.length();}return sum;

}


Benchmarks!

• Iteration :

Let us iterate over the whole list

2nd pattern: stream

@Benchmarkpublic long streamSum() {

return list.stream().mapToLong(String::length).sum();

}


Benchmarks!

• Results for Iterate


10 14,7 ns 11,6 ns

100 127 ns 84,3 ns

1 000 2,07 ms 895 ns

1 000 000 6,2 ms 3,3 ms


Benchmarks!

• Results for Stream


10 33 ns 64 ns

100 174 ns 440 ns

1 000 1,57 ms 3,6 ms

1 000 000 6,1 ms 4,8 ms


removeIf() and sort()

• Let us check the code


First conclusion

• The overhead of the System.arrayCopy() is not thatbad

It can be avoided in the case of growing lists

• That been said, there is still an unexpectedperformance difference, that we need to explain


CPU architecture

• The multicores CPU work in a special way…

• Between the main memory and the ALU (arithmetic & logic computation) there are at least 3 levels of cache, and registers


CPU architecture

Core 1Access to the registers< 1ns


CPU architecture

Core 1

L1

Access to the registers< 1ns

Access time ~1ns32kB data / code


CPU architecture

Core 1

L1

Access to the registers< 1ns

Access time ~1ns32kB data / code

L2Access time ~3ns256 kB


CPU architecture

Core 1

L1

L2

Core N

L1

L2


CPU architecture

Core 1

L1

L2

L3 (~12ns)

Core N

L1

L2


CPU architecture

C 1

L1

L2

L3

C N

L1

L2

C 1

L1

L2

L3

C N

L1

L2


CPU architecture

C 1

L1

L2

L3

QPI

C N

L1

L2

C 1

L1

L2

L3

QPI

C N

L1

L2

~40 ns


CPU architecture

C 1

L1

L2

L3

QPI

DRAM (~65 ns)

C N

L1

L2

C 1

L1

L2

L3

QPI

C N

L1

L2

~40 ns


So…

• Writing good algorithms is not enough…

• They also need to be adapted to this structure!


So…

• Writing good algorithms is not enough…

• They also need to be adapted to this structure!

• What makes a processing fast is its capacity to bringthe processed data in the L1 cache as fast as possible


The ennemy is the cache miss!

• A cache miss occurs when the CPU needs a data that isnot in the L1 cache

• This data has to be fetched somewhere else

• In the worst case: in main memory

• A cache miss is a loss of about 500 instructions


Structure of the L1 cache

• The L1 cache is organized in lines

• Each line is 8 long = 64 bytes

• The data is transfered line by line between the caches and the main memory

• And here lies a big difference between ArrayList and LinkedList…


Transferring a list in L1

• The elements of an ArrayList are stored in contiguous zones of the memory, since it is an array

• Where the nodes of a LinkedList are randomlyspread…

• So transferring an ArrayList in the L1 cache can be 8 times faster than a LinkedList (at least)


Pointer chasing

• This phenomenon is called « pointer chasing », and itcan kill performances!


Pointer chasing

• Example

int[] tab = {1, 2, 3, 4, 5};Integer[] tab = {1, 2, 3, 4, 5};


Pointer chasing

• Example


1 2 3 4 5


Pointer chasing

• Example


1 2 3 4 5

12

4

5

3


Pointer chasing

• Example

public class Point {Integer x, y;

}

List<Point> points = new ArrayList<>();


Pointer chasing

• Example


}


x

y1

2


• Example

• With such a structure, an algorithm will spend all itstime chasing pointers…

Pointer chasing


}


x

y1

2


Pointer chasing

• Clearly, the LinkedList structure is not adapted to the structure of CPU

• It is not a « cache friendly » structure


Pointer chasing

• An other example: HashMap

What is a HashMap? An array of Map.Entry objects


Pointer chasing



key

value


Pointer chasing



key

value

12

« twelve »




Pointer chasing

key

value

12

« twelve »




• The same goes for HashSet…

Pointer chasing

key

value

12

« twelve »


Some more benchmarks


HashSet ArrayList

10 26,2 ns 11,6 ns

100 223 ns 84,3 ns

1 000 3,4 ms 895 ns

1 000 000 11 ms 3,3 ms


• In our bench, the LinkedList is built like that:

• It means that out Node elements are still in contiguousplaces in memory…

A closer look at our bench

list = IntStream.range(0, size).mapToObj(Integer::toString).collect(toCollection(LinkedList::new));


• What about building our LinkedList like that:

• No more contiguous nodes…


list = IntStream.range(0, size).mapToObj(i -> IntStream.range(i, i + 100)

.mapToObj(Integer::toString)

.collect(toList()) ).map(l -> l.get(0)) .collect(toCollection(LinkedList::new));



• Results for Indexed Iterate

LinkedList Sparse LinkedList

10 14,7 ns 16,7 ns

100 127 ns 393 ns

1 000 2,07 ms 11 ms

1 000 000 6,2 ms 18 ms



• Results for Indexed Read


10 5,5 ns 5,4 ns

100 37,9 ns 69 ns

1 000 596 ns 2,0 ms

1 000 000 3,4 ms 15,3 ms



• Results for Indexed Set


10 5,5 ns 6,2 ns

100 36,0 ns 83,7 ns

1 000 602 ns 2,0 ms

1 000 000 3,4 ms 14,0 ms


Any other way?

• In a not-so-distant future: List<int> !

Project Valhalla: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/

• Motto:

Codes like a class, works like an int!

http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/


Any other way?

• In the much closer present:

Eclipse Collections (prev. GS Collections)

HPPC

Koloboke

Trove

http://java-performance.info/hashmap-overview-jdk-fastutil-goldman-sachs-hppc-koloboke-trove-january-2015/


A list without object?

• Is it possible to create implementations of Set / List / Map without any pointer chasing?


A list without object?

• Construire des implémentations de Set / List / Mapsans pointer chasing ?

• Pointer chasing comes from objects… Can we get rid of objects in those implementations?

#DevoxxJosé Paumard

Factory methods for Collections in Java 9Live coding: unexpected implementations of

lists, sets, maps

#Devoxx

José Paumard


So where do we want to go?

• Starting point: Java 9!

• Question: how to create a list with objects in it

• Answer: not easily done in Java 8…


A glimpse at Java 9

• Until Java 8: (mustache syntax)

• And if you want the immutable version, you need to create in two steps…

Map<Integer, String> map = new HashMap<Integer, String>() {{put(1, "one") ;put(2, "two") ;put(3, "three") ;

}} ;


A glimpse at Java 9

• New patterns in Java 9 for lists and sets:

The implementations are immutables

Null elements are not allowed (NPE)

The Set throws an IllegalArgumentException in case of a double

Serializable…

List<Integer> list = List.of(1, 2, 3) ;Set<Integer> set = Set.of(1, 2, 3) ;


A glimpse at Java 9

• New patterns in Java 9 for lists and sets:

• But they are also:

Randomly iterable

And with optimized implementations

List<Integer> list = List.of(1, 2, 3) ;Set<Integer> set = Set.of(1, 2, 3) ;


A glimpse at Java 9

• Optimized implementations:public static List<E> of(E e1);public static List<E> of(E e1, E e2);public static List<E> of(E e1, E e2, E e3);public static List<E> of(E e1, E e2, E e3, E e4);...public static List<E> of(E... e);


A glimpse at Java 9

• In the case of Mappublic static Map<K, V> of(K key1, V value1);public static Map<K, V> of(K key1, V value1, K key2, V value2);...public static Map<K, V> of(K... K, V... v);


A glimpse at Java 9

• In the case of Map

ofEntries() throws an exception in the case of a double key

public static Map<K, V> of(K key1, V value1);public static Map<K, V> of(K key1, V value1, K key2, V value2);...public static Map<K, V> of(K... K, V... v);

public static Entry<K, V> of(K key, V value);

public static Map<K, V> ofEntries(Entry... e);


A glimpse at Java 9

Map<String, TokenType> tokens =Map.ofEntries(

entry("for", KEYWORD),entry("while", KEYWORD),entry("do", KEYWORD),entry("break", KEYWORD),entry(":", COLON),entry("+", PLUS),entry("--‐", MINUS),entry(">", GREATER),entry("<", LESS),entry(":", PAAMAYIM_NEKUDOTAYIM),entry("(", LPAREN),entry(")", RPAREN)

); © Stuart Mark


A glimpse at Java 9

Map<String, TokenType> tokens =Map.ofEntries(

entry("for", KEYWORD),entry("while", KEYWORD),entry("do", KEYWORD),entry("break", KEYWORD),entry(":", COLON),entry("+", PLUS),entry("--‐", MINUS),entry(">", GREATER),entry("<", LESS),entry(":", PAAMAYIM_NEKUDOTAYIM),entry("(", LPAREN),entry(")", RPAREN)

); © Stuart Mark


What do we want to do?

• Question: how can we build an optimizedimplementation of a list with 1 or 2 elements?

• Set, List and Map (and Map.Entry)

• Of course we want to minimize pointer chasing

#Devoxx

Don’t worry, they’ll be back

And bullet points too


Some more bench

• Results for indexed read

ArrayList SingletonList TwoElementList

1 3,5 ns 2,7 ns

2 2,9 ns 2,6 ns


Some more bench


ArrayList SingletonList TwoElementList

1 4,7 ns 2,3 ns

2 6,5 ns 3,4 ns


Some more bench


HashSet SingletonSet TwoElementSet

1 7,3 ns 2,3 ns

2 9,5 ns 3,5 ns


Conclusion

• By Stuart Marks:


Conclusion

• Many patterns from the GoF can be implemented withlambdas

• We saw it for the template method

• And used it for lists, sets and maps…

But should be avoided!

• Avoiding pointer chasing = best performances!

#Devoxx

José Paumard

#Devoxx

Thank you!

José Paumard