Functional programming with_jdk8-s_ritter

Functional Programming
With JDK8
Simon Ritter
Head of Java Technology Evangelism
Oracle Corp.
Twitter: @speakjava
Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
2

Lambda Expressions

1996 … 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014
1.0 5.0 6 7 8
java.lang.Thread
java.util.concurrent
(jsr166)
Fork/Join Framework
(jsr166y)
Project LambdaConcurrency in Java
Phasers, etc
(jsr166)

The Problem: External Iteration
List<Student> students = ...
double highestScore = 0.0;
for (Student s : students) {
if (s.getGradYear() == 2011) {
if (s.getScore() > highestScore)
highestScore = s.score;
}
}
• Our code controls iteration
• Inherently serial: iterate from
beginning to end
• Not thread-safe
• Business logic is stateful
• Mutable accumulator variable

Internal Iteration With Inner Classes
• Iteration handled by the library
• Not inherently serial – traversal may
be done in parallel
• Traversal may be done lazily – so one
pass, rather than three
• Thread safe – client logic is stateless
• High barrier to use
– Syntactically ugly
More Functional
double highestScore = students
.filter(new Predicate<Student>() {
public boolean op(Student s) {
return s.getGradYear() == 2011;
}
})
.map(new Mapper<Student,Double>() {
public Double extract(Student s) {
return s.getScore();
}
})
.max();

Internal Iteration With Lambdas
List<Student> students = ...
.filter(Student s -> s.getGradYear() == 2011)
.map(Student s -> s.getScore())
.max();
• More readable
• More abstract
• Less error-prone
NOTE: This is not JDK8 code

Lambda Expressions
• Lambda expressions represent anonymous functions
– Same structure as a method
• typed argument list, return type, set of thrown exceptions, and a body
– Not associated with a class
• We now have parameterised behaviour, not just values
Some Details
.filter(Student s -> s.getGradYear() == 2011)
.map(Student s -> s.getScore())
.max();
What
How

Lambda Expression Types
• Single-method interfaces are used extensively in Java
– Definition: a functional interface is an interface with one abstract method
– Functional interfaces are identified structurally
– The type of a lambda expression will be a functional interface
• Lambda expressions provide implementations of the abstract method
interface Comparator<T> { boolean compare(T x, T y); }
interface FileFilter { boolean accept(File x); }
interface Runnable { void run(); }
interface ActionListener { void actionPerformed(…); }
interface Callable<T> { T call(); }

Local Variable Capture
• Lambda expressions can refer to effectively final local variables from the
surrounding scope
– Effectively final: A variable that meets the requirements for final variables (i.e.,
assigned once), even if not explicitly declared final
– Closures on values, not variables
void expire(File root, long before) {
root.listFiles(File p -> p.lastModified() <= before);
}

What Does ‘this’ Mean In A Lambda
• ‘this’ refers to the enclosing object, not the lambda itself
• Think of ‘this’ as a final predefined local
• Remember the Lambda is an anonymous function
– It is not associated with a class
– Therefore there can be no ‘this’ for the Lambda

Referencing Instance Variables
Which are not final, or effectively final
class DataProcessor {
private int currentValue;
public void process() {
DataSet myData = myFactory.getDataSet();
dataSet.forEach(d -> d.use(currentValue++));
}
}

Referencing Instance Variables
The compiler helps us out
class DataProcessor {
private int currentValue;
public void process() {
DataSet myData = myFactory.getDataSet();
dataSet.forEach(d -> d.use(this.currentValue++);
}
}
‘this’ (which is effectively final)
inserted by the compiler

Type Inference
• The compiler can often infer parameter types in a lambda expression
 Inferrence based on the target functional interface’s method signature
• Fully statically typed (no dynamic typing sneaking in)
– More typing with less typing
List<String> list = getList();
Collections.sort(list, (String x, String y) -> x.length() - y.length());
Collections.sort(list, (x, y) -> x.length() - y.length());
static T void sort(List<T> l, Comparator<? super T> c);

Method References
• Method references let us reuse a method as a lambda expression
FileFilter x = File f -> f.canRead();
FileFilter x = File::canRead;

Method References
• Format: target_reference::method_name
• Three kinds of method reference
– Static method (e.g. Integer::parseInt)
– Instance method of an arbitary type (e.g. String::length)
– Instance method of an existing object
16
More Detail

Method References
17
Rules For Construction
Lambda
Method Ref
Lambda
Method Ref
Lambda
Method Ref
(args) -> ClassName.staticMethod(args)
(arg0, rest) -> arg0.instanceMethod(rest)
(args) -> expr.instanceMethod(args)
ClassName::staticMethod
ClassName::instanceMethod
expr::instanceMethod
instanceOf

Constructor References
• Same concept as a method reference
– For the constructor
Factory<List<String>> f = ArrayList<String>::new;
Factory<List<String>> f = () -> return new ArrayList<String>();
Replace with

Useful Methods That Can Use Lambdas
• Iterable.forEach(Consumer c)
List<String> myList = ...
myList.forEach(s -> System.out.println(s));
myList.forEach(System.out::println);

• Collection.removeIf(Predicate p)
myList.removeIf(s -> s.length() == 0)

• List.replaceAll(UnaryOperator o)
myList.replaceAll(s -> s.toUpperCase());
myList.replaceAll(String::toUpper);

• List.sort(Comparator c)
• Replaces Collections.sort(List l, Comparator c)
myList.sort((x, y) -> x.length() – y.length());

Library Evolution

Library Evolution Goal
• Requirement: aggregate operations on collections
–New methods required on Collections to facilitate this
• This is problematic
– Can’t add new methods to interfaces without modifying all implementations
– Can’t necessarily find or control all implementations
int heaviestBlueBlock = blocks
.filter(b -> b.getColor() == BLUE)
.map(Block::getWeight)
.reduce(0, Integer::max);

Solution: Default Methods
• Specified in the interface
• From the caller’s perspective, just an ordinary interface method
• Provides a default implementation
• Default only used when implementation classes do not provide a body for the
extension method
• Implementation classes can provide a better version, or not
interface Collection<E> {
default Stream<E> stream() {
return StreamSupport.stream(spliterator());
}
}

Virtual Extension Methods
• Err, isn’t this implementing multiple inheritance for Java?
• Yes, but Java already has multiple inheritance of types
• This adds multiple inheritance of behavior too
• But not state, which is where most of the trouble is
• Can still be a source of complexity
• Class implements two interfaces, both of which have default methods
• Same signature
• How does the compiler differentiate?
• Static methods also allowed in interfaces in Java SE 8
Stop right there!

Functional Interface Definition
• An interface
• Must have only one abstract method
– In JDK 7 this would mean only one method (like ActionListener)
• JDK 8 introduced default methods
– Adding multiple inheritance of types to Java
– These are, by definition, not abstract (they have an implementation)
• JDK 8 also now allows interfaces to have static methods
– Again, not abstract
• @FunctionalInterface can be used to have the compiler check
27

Is This A Functional Interface?
28
@FunctionalInterface
public interface Runnable {
public abstract void run();
}
Yes. There is only
one abstract method

29
public interface Predicate<T> {
default Predicate<T> and(Predicate<? super T> p) {…};
default Predicate<T> negate() {…};
default Predicate<T> or(Predicate<? super T> p) {…};
static <T> Predicate<T> isEqual(Object target) {…};
boolean test(T t);
}
Yes. There is still only
one abstract method

30
public interface Comparator {
// default and static methods elided
int compare(T o1, T o2);
boolean equals(Object obj);
}
The equals(Object)
method is implicit
from the Object class
Therefore only one
abstract method

Lambdas In Full Flow:
Streams

Stream Overview
• Abstraction for specifying aggregate computations
– Not a data structure
– Can be infinite
• Simplifies the description of aggregate computations
– Exposes opportunities for optimisation
– Fusing, laziness and parallelism
At The High Level

Stream Overview
• A stream pipeline consists of three types of things
– A source
– Zero or more intermediate operations
– A terminal operation
• Producing a result or a side-effect
Pipeline
int total = transactions.stream()
.filter(t -> t.getBuyer().getCity().equals(“London”))
.mapToInt(Transaction::getPrice)
.sum();
Source
Intermediate operation
Terminal operation

Stream Sources
• From collections and arrays
– Collection.stream()
– Collection.parallelStream()
– Arrays.stream(T array) or Stream.of()
• Static factories
– IntStream.range()
– Files.walk()
• Roll your own
– java.util.Spliterator
Many Ways To Create

Stream Terminal Operations
• The pipeline is only evaluated when the terminal operation is called
– All operations can execute sequentially or in parallel
– Intermediate operations can be merged
• Avoiding multiple redundant passes on data
• Short-circuit operations (e.g. findFirst)
• Lazy evaluation
– Stream characteristics help identify optimisations
• DISTINT stream passed to distinct() is a no-op

Maps and FlatMaps
Map Values in a Stream
Map
FlatMap
Input Stream
Input Stream
1-to-1 mapping
1-to-many mapping
Output Stream
Output Stream

The Optional Class

Optional<T>
Reducing NullPointerException Occurrences
String direction = gpsData.getPosition().getLatitude().getDirection();
String direction = “UNKNOWN”;
if (gpsData != null) {
Position p = gpsData.getPosition();
if (p != null) {
Latitude latitude = p.getLatitude();
if (latitude != null)
direction = latitude.getDirection();
}
}
String direction = gpsData.getPosition().getLatitude().getDirection();

Optional Class
• Terminal operations like min(), max(), etc do not return a direct result
• Suppose the input Stream is empty?
• Optional<T>
– Container for an object reference (null, or real object)
– Think of it like a Stream of 0 or 1 elements
– use get(), ifPresent() and orElse() to access the stored reference
– Can use in more complex ways: filter(), map(), etc
– gpsMaybe.filter(r -> r.lastReading() < 2).ifPresent(GPSData::display);
Helping To Eliminate the NullPointerException

Optional ifPresent()
Do something when set
if (x != null) {
print(x);
}
opt.ifPresent(x -> print(x));
opt.ifPresent(this::print);

Optional filter()
Reject certain values of the Optional
if (x != null && x.contains("a")) {
print(x);
}
opt.filter(x -> x.contains("a"))
.ifPresent(this::print);

Optional map()
Transform value if present
if (x != null) {
String t = x.trim();
if (t.length() > 1)
print(t);
}
opt.map(String::trim)
.filter(t -> t.length() > 1)
.ifPresent(this::print);

Optional flatMap()
Going deeper
public String findSimilar(String s)
Optional<String> tryFindSimilar(String s)
Optional<Optional<String>> bad = opt.map(this::tryFindSimilar);
Optional<String> similar = opt.flatMap(this::tryFindSimilar);

Update Our GPS Code
class GPSData {
public Optional<Position> getPosition() { ... }
}
class Position {
public Optional<Latitude> getLatitude() { ... }
}
class Latitude {
public String getString() { ... }
}

Update Our GPS Code
String direction = Optional.ofNullable(gpsData)
.flatMap(GPSData::getPosition)
.flatMap(Position::getLatitude)
.map(Latitude::getDirection)
.orElse(“None”);

Stream Examples

Example 1
Convert words in list to upper case
List<String> output = wordList
.stream()
.map(String::toUpperCase)
.collect(Collectors.toList());

Example 1
Convert words in list to upper case (in parallel)
List<String> output = wordList
.parallelStream()
.map(String::toUpperCase)

Example 2
• BufferedReader has new method
– Stream<String> lines()
Count lines in a file
long count = bufferedReader
.lines()
.count();

Example 3
Join lines 3-4 into a single string
String output = bufferedReader
.lines()
.skip(2)
.limit(2)
.collect(Collectors.joining());

Example 4
Collect all words in a file into a list
List<String> output = reader
.lines()
.flatMap(line -> Stream.of(line.split(REGEXP)))
.filter(word -> word.length() > 0)

Example 5
List of unique words in lowercase, sorted by length
.lines()
.map(String::toLowerCase)
.distinct()
.sorted((x, y) -> x.length() - y.length())

Example 6
Create a map: Keys are word length, values are lists of words of that length
Map<Integer, List<String>> result = reader.lines()
.collect(groupingBy(String::length));

Example 7
Create a map: Keys are word length, values are how many words of that length
Map<Integer, Long> result = reader.lines()
.collect(groupingBy(String::length, counting()));
Downstream collector
to return count of elements

Example 8: Real World
Infinite stream from thermal sensor
private int double currentTemperature;
...
thermalReader
.lines()
.mapToDouble(s ->
Double.parseDouble(s.substring(0, s.length() - 1)))
.map(t -> ((t – 32) * 5 / 9)
.filter(t -> t != currentTemperature)
.peek(t -> listener.ifPresent(l -> l.temperatureChanged(t)))
.forEach(t -> currentTemperature = t);

Example 8: Real World
Infinite stream from thermal sensor
private int double currentTemperature;
...
thermalReader
.lines()
.mapToDouble(s ->
Double.parseDouble(s.substring(0, s.length() - )))
.map(t -> ((t – 32) * 5 / 9)
.filter(t -> t != this.currentTemperature)
.peek(t -> listener.ifPresent(l -> l.temperatureChanged(t)))
.forEach(t -> this.currentTemperature = t);

Advanced Lambdas And Streams

Exception Handling In
Lambdas

Exception Handling In Lambda Expressions
• Compiler error: uncaught exception in filter()
Can Be Problematic
public boolean isValid(Object o) throws IOException { ... }
public List<String> processList(List<String> list) throws IOException
{
return list.stream()
.filter(w -> isValid(w))
}

60
Catch the Exception In The Lambda
public double processList(List list) {
return list.stream()
.filter(w -> {
try {
return isValid(w)
} catch (IOException ioe) {
return false;
}
})
}

• Which is great, except you can't use it with Streams
61
Define A New Functional Interface
public interface PredicateWithException {
public boolean test(String s) throws IOException;
}

Debugging

Logging Mid-Stream
The peek() method
.lines()
.peek(s -> System.out.println(“Word = “ + s))

Using peek() To Set A Breakpoint
.lines()
.peek(s -> s)
Set breakpoint on peek
Some debuggers don’t like empty bodies

Streams and Concurrency

Serial And Parallel Streams
• Collection Stream sources
– stream()
– parallelStream()
• Stream can be made parallel or sequential at any point
– parallel()
– sequential()
• The last call wins
– Whole stream is either sequential or parallel
66

Parallel Streams
• Implemented underneath using the fork-join framework
• Will default to as many threads for the pool as the OS reports processors
– Which may not be what you want
System.setProperty(
"java.util.concurrent.ForkJoinPool.common.parallelism",
"32767");
• Remember, parallel streams always need more work to process
– But they might finish it more quickly
67

When To Use Parallel Streams
• Data set size is important, as is the type of data structure
– ArrayList: GOOD
– HashSet, TreeSet: OK
– LinkedList: BAD
• Operations are also important
– Certain operations decompose to parallel tasks better than others
– filter() and map() are excellent
– sorted() and distinct() do not decompose well
68
Sadly, no simple answer

When To Use Parallel Streams
• N = size of the data set
• Q = Cost per element through the Stream pipeline
• N x Q = Total cost of pipeline operations
• The bigger N x Q is the better a parallel stream will perform
• It is easier to know N than Q, but Q can be estimated
• If in doubt, profile
69
Quantative Considerations

Streams: Pitfalls For The Unwary

Functional v. Imperative
• For functional programming you should not modify state
• Java supports closures over values, not closures over variables
• But state is really useful…
71

Counting Methods That Return Streams
72
Still Thinking Imperatively
Set<String> sourceKeySet = streamReturningMethodMap.keySet();
LongAdder sourceCount = new LongAdder();
sourceKeySet.stream()
.forEach(c ->
sourceCount.add(streamReturningMethodMap.get(c).size()));

Counting Methods That Return Streams
73
Functional Way
sourceKeySet.stream()
.mapToInt(c -> streamReturningMethodMap.get(c).size())
.sum();

Printing And Counting Functional Interfaces
74
Still Thinking Imperatively
LongAdder newMethodCount = new LongAdder();
functionalParameterMethodMap.get(c).stream()
.forEach(m -> {
output.println(m);
if (isNewMethod(c, m))
newMethodCount.increment();
});
return newMethodCount.intValue();

75
More Functional, But Not Pure Functional
int count = functionalParameterMethodMap.get(c).stream()
.mapToInt(m -> {
int newMethod = 0;
output.println(m);
if (isNewMethod(c, m))
newMethod = 1;
return newMethod
})
.sum();
There is still state being
modified in the Lambda

76
Even More Functional, But Still Not Pure Functional
int count = functionalParameterMethodMap.get(nameOfClass)
.stream()
.peek(method -> output.println(method))
.mapToInt(m -> isNewMethod(nameOfClass, m) ? 1 : 0)
.sum();
Strictly speaking printing
is a side effect, which is
not purely functional

The Art Of Reduction
(Or The Need to Think Differently)

A Simple Problem
• Find the length of the longest line in a file
• Hint: BufferedReader has a new method, lines(), that returns a Stream
78
BufferedReader reader = ...
reader.lines()
.mapToInt(String::length)
.max()
.getAsInt();

Another Simple Problem
• Find the length of the longest line in a file
79

Naïve Stream Solution
• That works, so job done, right?
• Not really. Big files will take a long time and a lot of resources
• Must be a better approach
80
String longest = reader.lines().
sort((x, y) -> y.length() - x.length()).
findFirst().
get();

External Iteration Solution
• Simple, but inherently serial
• Not thread safe due to mutable state
81
String longest = "";
while ((String s = reader.readLine()) != null)
if (s.length() > longest.length())
longest = s;

Recursive Approach: The Method
82
String findLongestString(String s, int index, List<String> l) {
if (index >= l.size())
return s;
if (index == l.size() - 1) {
if (s.length() > l.get(index).length())
return s;
return l.get(index);
}
String s2 = findLongestString(l.get(start), index + 1, l);
if (s.length() > s2.length())
return s;
return s2;
}

Recursive Approach: Solving The Problem
• No explicit loop, no mutable state, we’re all good now, right?
• Unfortunately not - larger data sets will generate an OOM exception
83
List<String> lines = new ArrayList<>();
while ((String s = reader.readLine()) != null)
lines.add(s);
String longest = findLongestString("", 0, lines);

A Better Stream Solution
• The Stream API uses the well known filter-map-reduce pattern
• For this problem we do not need to filter or map, just reduce
Optional<T> reduce(BinaryOperator<T> accumulator)
• The key is to find the right accumulator
– The accumulator takes a partial result and the next element, and returns a new
partial result
– In essence it does the same as our recursive solution
– Without all the stack frames
• BinaryOperator is a subclass of BiFunction, but all types are the same
• R apply(T t, U u) or T apply(T x, T y)
84

• Use the recursive approach as an accululator for a reduction
85
String longestLine = reader.lines()
.reduce((x, y) -> {
if (x.length() > y.length())
return x;
return y;
})
.get();

• Use the recursive approach as an accululator for a reduction
86
String longestLine = reader.lines()
.reduce((x, y) -> {
if (x.length() > y.length())
return x;
return y;
})
.get();
x in effect maintains state for
us, by always holding the
longest string found so far

The Simplest Stream Solution
• Use a specialised form of max()
• One that takes a Comparator as a parameter
• comparingInt() is a static method on Comparator
– Comparator<T> comparingInt(ToIntFunction<? extends T> keyExtractor)
87
reader.lines()
.max(comparingInt(String::length))
.get();

Conclusions
• Java needs lambda statements
– Significant improvements in existing libraries are required
• Require a mechanism for interface evolution
– Solution: virtual extension methods
• Bulk operations on Collections
– Much simpler with Lambdas
• Java SE 8 evolves the language, libraries, and VM together

Simon Ritter
Oracle Corporartion
Twitter: @speakjava

Functional programming with_jdk8-s_ritter

Recommended

More Related Content

What's hot (20)

Similar to Functional programming with_jdk8-s_ritter (20)

More from Simon Ritter (20)

Recently uploaded (20)

Functional programming with_jdk8-s_ritter