Linked to ArrayList: the full story

#Devoxx
LinkedList to ArrayList
The Full Story!
José Paumard

#Devoxx #ListJ9 @JosePaumard
It all begin…
• November 22nd 2015 by a tweet

It all begin…
• Then the discussion started

It all begin…
• And about 30 tweets later:

And did not ended there!
• And around midnight, 7th nov. 2016:
• Since this is the subject of this Uni, we will take more
time…

Linked to ArrayList: the full story

Questions ?
#ListJ9
Questions?
#ListJ9

• This discussion ArrayList vs LinkedList is no troll…
 Even if it looks like one!
• The question is :
Is one of these implementations better than the other?
• And there is an answer to this question!
And the conclusion is…

• To fully understand this answer, we need to
understand many other things

• To fully understand this answer, we need to
understand many other things
• And once we have answered it (almost) fully , many
other questions will arise!

1st part
• Algorithms
• Complexity
• Implementation
• CPU architecture
• « cache friendly »
• Slides, bullet points…

2nd part
• Implementation of Set, List et Map (with a surprise!)
• Live coding!
• Some more slides but not that many
 But don’t worry, still bullet points

Where to begin?
• In fact there are several ways to begin…

Where to begin?
• Algorithms!

Where to begin?
• Algorithms!
• Implementation vs CPU architecture

Where to begin?
• Algorithms!
• Implementation vs CPU architecture
• Use cases

What do we know?
1) ArrayList is built on an array
2) LinkedList is built on a linked list
And we have results on this

Approach
• Take a close look at the basic operations on lists
• Create / modify:
 Add an element at the end of at the beginning of a list
 Removing an element by its index
 Removing an element
 The contains() method
 The removeIf() and sort() methods

Approach
• Add:
List<String> list = ...
list.add("one"); // add
list.add(12, "one"); // insert

Approach
• Remove:
list.remove("one"); // remove
list.remove(12); // remove by index

Approach
• Sort / removeIf:
list.sort(comparingBy(String::length)); // sort
list.removeIf(s -> s.length() > 10); // remove if

Approach
• Read operations:
 Iterate
 Read an element with its index
 Building a Stream

Approach
• Take a close look at the basis operations on lists
• Read operations:
list.forEach(System.out::println); // list traversal
list.get(12); // access by index
list.stream(); // stream creation

ArrayList
• Built on an array
• Access to the nth element: almost free
• Adding an element: almost free, but…
• Insertion: shift the elements to the right
• Deletion: shift the elements to the left

ArrayList
• Access to the nth element: almost free
n

ArrayList
private void grow(int minCapacity) {
int oldCapacity = elementData.length;
int newCapacity = oldCapacity + (oldCapacity >> 1);
elementData = Arrays.copyOf(elementData, newCapacity);
}

ArrayList
• Sometimes the array has to be enlarge
• But we can fix the size of the array when we build it

ArrayList
• Sometimes the array has to be enlarge
• Bytheway growing a HashSet is much more
expensive…

ArrayList
• Insertion
n, one

ArrayList
• Insertion
public void add(int index, E element) {
System.arraycopy(elementData, index, elementData, index + 1,
size - index);
elementData[index] = element;
size++;
}

ArrayList
• Insertion
one

ArrayList
• Suppression
n
X

ArrayList
• Suppression
public E remove(int index) {
E oldValue = elementData(index);
System.arraycopy(elementData, index+1, elementData, index,
numMoved);
return oldValue;
}

ArrayList
• (Very) Early conclusion
• Accessing an element is fast!
• Adding is fast, if there is no overflow
• Random insert / delete: overhead to handle the
« holes »

LinkedList
• Built on a double linked list
• Access to the nth element
• Adding an element at the end of the list
• Insertion
• Deletion

LinkedList
• Base structure
next prev
item
Node
next prev
item
Node
next prev
item
Node
first last

LinkedList
• Built on a double linked list
• Access to the nth element: run through all the elements
• Adding: free, just a modification of pointers
• Insertion: free, just a modification of pointers
• Suppression: free, just a modification of pointers

LinkedList
• Access to the nth element
Node<E> node(int index) {
if (index < (size >> 1)) {
Node<E> x = first;
for (int i = 0; i < index; i++)
x = x.next;
return x;
}
// same for last
}

LinkedList
• Access to the nth element: one has to visit all the
elements and count them, until the nth is found
• We say that the complexity of this operation is « N »
• Or is O(n)

Notation O(N)
• This notation means that:
 if we multiply the amount of elements to be processed by 2
 then the number of operations will also be multiplied by 2

Notation O(N)
• The complexity of an algorithm is given by O(f(n))
• Example: f(n) = n2

Notation O(N)
• The complexity of an algorithm is given by O(f(n))
• Example: f(n) = n2
• In fact, it means that the exact number of operations is
f(n) = an2 + bn + g
• And when n reaches a given value
f(n) ~ n2

Notation O(N)
• All right, but processing data is not anly about maths
• In our applications, n has a value, the same for a, b
and g…
• And the theory might not apply very well to our use
case

Complexity comparison
log2(n)
1 0
10 3
100 7
1 000 10
1 000 000 20

log2(n) n
1 0 1
10 3 10
100 7 100
1 000 10 1 000
1 000 000 20 1 000 000

log2(n) n n x log2(n)
1 0 1 0
10 3 10 30
100 7 100 700
1 000 10 1 000 10 000
1 000 000 20 1 000 000 20 000 000

log2(n) n n x log2(n) n2
1 0 1 0 1
10 3 10 30 100
100 7 100 700 10 000
1 000 10 1 000 10 000 1 000 000
1 000 000 20 1 000 000 20 000 000 too much!

« too much » means that on any CPU, running a
thousand billion of operations will take at least 20mn
• So we need to parallelize this operations of a large
number of CPU (at least a few thousands)
 We need to change our algorithm
 We are not running this on a single JVM

Lists complexity
• On the basic operations
ArrayList LinkedList
add(e) 1
add(index, e) 1
set(index, e) 1
remove(index) 1
iterator() 1

Lists complexity
• On the basic operations
ArrayList LinkedList
add(e) 1 1
add(index, e) 1 N
set(index, e) 1 N
remove(index) 1 N
iterator() 1 1

Lists complexity
• We still must be cautious…
• There are System.arrayCopy() in ArrayList that are
not here in LinkedList
• Is it a g? Or something that grows with n? And how?

Lists complexity
• We still must be cautious…
• Example: if we consider add(index, e), is
System.arrayCopy() more costly than going through
the pointers of the LinkedList?
• Are there any hidden costs that we did not see?

Benchmarks!
• There is a standard tool (Java 9) to bench Java code :
JMH
• A benchmark is an annotated class
• Maven can generate a « transformed » JAR
• Then we run this JAR to get the result of the bench

Benchmarks!
• Here is a « bench » class
@Warmup(iterations=5, time=10, timeUnit=TimeUnit.MILLISECONDS)
@Measurement(iterations=10, time=10, timeUnit=TimeUnit.MILLISECONDS)
@Fork(value=1,
jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms3g", "-Xmx3g"})
@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.NANOSECONDS)
@State(Scope.Benchmark)
public class Bench {
// bench
}

Benchmarks!
• POM dependencies, version is 1.15
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-core</artifactId>
<version>${jmh.version}</version> 
</dependency>
<dependency>
<groupId>org.openjdk.jmh</groupId>
<artifactId>jmh-generator-annprocess</artifactId>
<version>${jmh.version}</version>
<scope>provided</scope>
</dependency>

Benchmarks!
• In the class
@Param({"10", "100", "1000"})
private int size;
@Param({LINKED_LIST, ARRAY_LIST})
private String type;

Benchmarks!
• In the class
@Setup
public void setup() {
switch(type) {
case ARRAY_LIST :
list = IntStream.range(0, size)
.mapToObj(Integer::toString)
.collect(Collectors.toCollection(
() -> new ArrayList<String>(size)));
break;
// other cases
}

Benchmarks!
• Simple add operation:
 Can trigger a System.arrayCopy() on ArrayList
 Fast on LinkedList since there is a pointer to the end of the
list
@Benchmark
public boolean simpleAdd() {
return list.add("one more");
}

Benchmarks!
• Simple add with index:
 Triggers at least one System.arrayCopy() one ArrayList
 Slower on LinkedList since half of the list must be visited
@Benchmark
public boolean simpleAdd() {
list.add(size / 2, "one more");
return true;
}

Benchmarks!
• Results for Simple Add
LinkedList ArrayList Big ArrayList
10 5,3 ns 3,6 ns 3,6 ns
100 5,3 ns 3,6 ns 3,7 ns
1 000 5,3 ns 3,7 ns 3,7 ns
1 000 000 5,5 ns 3,7 ns 3,7 ns

Benchmarks!
• Index Read :
 Read operation in the middle of the list
 We expect the worst performance for LinkedList
@Benchmark
public boolean indexRead() {
return list.get(size / 2);
}

Benchmarks!
• Index Set:
 Write operation in the middle of the list
 We also expect the worst performance for LinkedList
@Benchmark
public boolean indexSet() {
return list.set(size / 2, "one more");
}

Benchmarks!
• Results for Index Read
LinkedList ArrayList
10 5,5 ns 2,8 ns
100 37,9 ns 2,8 ns
1 000 596 ns 2,8 ns
1 000 000 3,4 ms 2,8 ns

Benchmarks!
• Results for Index Set
10 5,5 ns 3,4 ns
100 36,0 ns 3,5 ns
1 000 602 ns 3,5 ns
1 000 000 3,4 ms 3,5 ns

Benchmarks!
• Iteration :
 Let us iterate over the whole list
 1st pattern: iterator
@Benchmark
public long iterate() {
long sum = 0;
for (String s : list) {
sum += s.length();
}
return sum;
}

Benchmarks!
• Iteration :
 Let us iterate over the whole list
 2nd pattern: stream
@Benchmark
public long streamSum() {
return list.stream()
.mapToLong(String::length)
.sum();
}

Benchmarks!
• Results for Iterate
10 14,7 ns 11,6 ns
100 127 ns 84,3 ns
1 000 2,07 ms 895 ns
1 000 000 6,2 ms 3,3 ms

Benchmarks!
• Results for Stream
10 33 ns 64 ns
100 174 ns 440 ns
1 000 1,57 ms 3,6 ms
1 000 000 6,1 ms 4,8 ms

removeIf() and sort()
• Let us check the code

First conclusion
• The overhead of the System.arrayCopy() is not that
bad
 It can be avoided in the case of growing lists
• That been said, there is still an unexpected
performance difference, that we need to explain

CPU architecture
• The multicores CPU work in a special way…
• Between the main memory and the ALU (arithmetic &
logic computation) there are at least 3 levels of cache,
and registers

CPU architecture
Core 1
Access to the registers
< 1ns

CPU architecture
Core 1
L1
< 1ns
Access time ~1ns
32kB data / code

CPU architecture
Core 1
L1
< 1ns
Access time ~1ns
32kB data / code
L2
Access time ~3ns
256 kB

CPU architecture
Core 1
L1
L2
Core N
L1
L2

CPU architecture
Core 1
L1
L2
L3 (~12ns)
Core N
L1
L2

CPU architecture
C 1
L1
L2
L3
C N
L1
L2
C 1
L1
L2
L3
C N
L1
L2

CPU architecture
C 1
L1
L2
L3
QPI
C N
L1
L2
C 1
L1
L2
L3
QPI
C N
L1
L2
~40 ns

CPU architecture
C 1
L1
L2
L3
QPI
DRAM (~65 ns)
C N
L1
L2
C 1
L1
L2
L3
QPI
C N
L1
L2
~40 ns

So…
• Writing good algorithms is not enough…
• They also need to be adapted to this structure!

So…
• Writing good algorithms is not enough…
• They also need to be adapted to this structure!
• What makes a processing fast is its capacity to bring
the processed data in the L1 cache as fast as possible

The ennemy is the cache miss!
• A cache miss occurs when the CPU needs a data that is
not in the L1 cache
• This data has to be fetched somewhere else
• In the worst case: in main memory
• A cache miss is a loss of about 500 instructions

Structure of the L1 cache
• The L1 cache is organized in lines
• Each line is 8 long = 64 bytes
• The data is transfered line by line between the caches
and the main memory
• And here lies a big difference between ArrayList and
LinkedList…

Transferring a list in L1
• The elements of an ArrayList are stored in
contiguous zones of the memory, since it is an array
• Where the nodes of a LinkedList are randomly
spread…
• So transferring an ArrayList in the L1 cache can be 8
times faster than a LinkedList (at least)

Pointer chasing
• This phenomenon is called « pointer chasing », and it
can kill performances!

Pointer chasing
• Example
int[] tab = {1, 2, 3, 4, 5};
Integer[] tab = {1, 2, 3, 4, 5};

Pointer chasing
• Example
int[] tab = {1, 2, 3, 4, 5};
Integer[] tab = {1, 2, 3, 4, 5};
1 2 3 4 5

Pointer chasing
• Example
int[] tab = {1, 2, 3, 4, 5};
Integer[] tab = {1, 2, 3, 4, 5};
1 2 3 4 5
    
1
2
4
5
3

Pointer chasing
• Example
public class Point {
Integer x, y;
}
List<Point> points = new ArrayList<>();

Pointer chasing
• Example
Integer x, y;
}





x
y
1
2

• Example
• With such a structure, an algorithm will spend all its
time chasing pointers…
Pointer chasing
Integer x, y;
}





x
y
1
2

Pointer chasing
• Clearly, the LinkedList structure is not adapted to the
structure of CPU
• It is not a « cache friendly » structure

Pointer chasing
• An other example: HashMap
 What is a HashMap? An array of Map.Entry objects

Pointer chasing
key
value

Pointer chasing
key
value
12
« twelve »

Pointer chasing
key
value
12
« twelve »




• The same goes for HashSet…
Pointer chasing
key
value
12
« twelve »




Some more benchmarks
HashSet ArrayList
10 26,2 ns 11,6 ns
100 223 ns 84,3 ns
1 000 3,4 ms 895 ns
1 000 000 11 ms 3,3 ms

• In our bench, the LinkedList is built like that:
• It means that out Node elements are still in contiguous
places in memory…
A closer look at our bench
.collect(toCollection(LinkedList::new));

• What about building our LinkedList like that:
• No more contiguous nodes…
.mapToObj(i -> IntStream.range(i, i + 100)
.collect(toList())
)
.map(l -> l.get(0))
.collect(toCollection(LinkedList::new));

• Results for Indexed Iterate
LinkedList Sparse LinkedList
10 14,7 ns 16,7 ns
100 127 ns 393 ns
1 000 2,07 ms 11 ms
1 000 000 6,2 ms 18 ms

• Results for Indexed Read
10 5,5 ns 5,4 ns
100 37,9 ns 69 ns
1 000 596 ns 2,0 ms
1 000 000 3,4 ms 15,3 ms

• Results for Indexed Set
10 5,5 ns 6,2 ns
100 36,0 ns 83,7 ns
1 000 602 ns 2,0 ms
1 000 000 3,4 ms 14,0 ms

Any other way?
• In a not-so-distant future: List<int> !
 Project Valhalla:
http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/
• Motto:
Codes like a class, works like an int!

Any other way?
• In the much closer present:
 Eclipse Collections (prev. GS Collections)
 HPPC
 Koloboke
 Trove
http://java-performance.info/hashmap-overview-jdk-fastutil-
goldman-sachs-hppc-koloboke-trove-january-2015/

A list without object?
• Is it possible to create implementations of Set / List /
Map without any pointer chasing?

A list without object?
• Construire des implémentations de Set / List / Map
sans pointer chasing ?
• Pointer chasing comes from objects… Can we get rid of
objects in those implementations?

#Devoxx
José Paumard
Factory methods for Collections in Java 9
Live coding: unexpected implementations of
lists, sets, maps

So where do we want to go?
• Starting point: Java 9!
• Question: how to create a list with objects in it
• Answer: not easily done in Java 8…

A glimpse at Java 9
• Until Java 8: (mustache syntax)
• And if you want the immutable version, you need to
create in two steps…
Map<Integer, String> map = new HashMap<Integer, String>() {{
put(1, "one") ;
put(2, "two") ;
put(3, "three") ;
}} ;

A glimpse at Java 9
• New patterns in Java 9 for lists and sets:
 The implementations are immutables
 Null elements are not allowed (NPE)
 The Set throws an IllegalArgumentException in case of a
double
 Serializable…
List<Integer> list = List.of(1, 2, 3) ;
Set<Integer> set = Set.of(1, 2, 3) ;

A glimpse at Java 9
• New patterns in Java 9 for lists and sets:
• But they are also:
 Randomly iterable
 And with optimized implementations
List<Integer> list = List.of(1, 2, 3) ;
Set<Integer> set = Set.of(1, 2, 3) ;

A glimpse at Java 9
• Optimized implementations:
public static List<E> of(E e1);
public static List<E> of(E e1, E e2);
public static List<E> of(E e1, E e2, E e3);
public static List<E> of(E e1, E e2, E e3, E e4);
...
public static List<E> of(E... e);

A glimpse at Java 9
• In the case of Map
public static Map<K, V> of(K key1, V value1);
public static Map<K, V> of(K key1, V value1, K key2, V value2);
...
public static Map<K, V> of(K... K, V... v);

A glimpse at Java 9
• In the case of Map
 ofEntries() throws an exception in the case of a double key
public static Map<K, V> of(K key1, V value1);
public static Map<K, V> of(K key1, V value1, K key2, V value2);
...
public static Map<K, V> of(K... K, V... v);
public static Entry<K, V> of(K key, V value);
public static Map<K, V> ofEntries(Entry... e);

A glimpse at Java 9
Map<String, TokenType> tokens =
Map.ofEntries(
entry("for", KEYWORD),
entry("while", KEYWORD),
entry("do", KEYWORD),
entry("break", KEYWORD),
entry(":", COLON),
entry("+", PLUS),
entry("--‐", MINUS),
entry(">", GREATER),
entry("<", LESS),
entry(":", PAAMAYIM_NEKUDOTAYIM),
entry("(", LPAREN),
entry(")", RPAREN)
); © Stuart Mark

What do we want to do?
• Question: how can we build an optimized
implementation of a list with 1 or 2 elements?
• Set, List and Map (and Map.Entry)
• Of course we want to minimize pointer chasing

#Devoxx
Don’t worry, they’ll be back
And bullet points too

Some more bench
• Results for indexed read
ArrayList SingletonList TwoElementList
1 3,5 ns 2,7 ns
2 2,9 ns 2,6 ns

Some more bench
ArrayList SingletonList TwoElementList
1 4,7 ns 2,3 ns
2 6,5 ns 3,4 ns

Some more bench
HashSet SingletonSet TwoElementSet
1 7,3 ns 2,3 ns
2 9,5 ns 3,5 ns

Conclusion
• By Stuart Marks:

Conclusion
• Many patterns from the GoF can be implemented with
lambdas
• We saw it for the template method
• And used it for lists, sets and maps…
 But should be avoided!
• Avoiding pointer chasing = best performances!

#Devoxx
Thank you!
José Paumard

Linked to ArrayList: the full story

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Linked to ArrayList: the full story (20)

More from José Paumard (20)

Recently uploaded (20)

Linked to ArrayList: the full story