Scalable Applications with Scala
Download as pptx, pdf3 likes1,717 views
This document discusses how to build scalable applications using Scala. It defines scalability as a system's ability to handle growing workloads through methods like supporting more users, adding new features easily, or maintaining performance across different locations. The document then covers how Scala supports scalability through features like concurrency/parallelism, immutability, and functional programming patterns. It provides examples of how Scala's collections library allows parallel operations and how futures can be composed to perform asynchronous and parallel work efficiently.
![Scala Futures
val p = Promise[T]
val f: Future[T] = p.future](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-41-320.jpg)
![Java Futures
val p = Promise[T]
val f: Future[T] = p.future
// Java-like variant:
doSomeWork()
f.get()](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-42-320.jpg)
![Callbacks
Thread 1
val p = Promise[T]
val f: Future[T] = p.future
Thread 2
f onComplete {
case Success(calcResult) = println(calcResult)
case Failure(t) = println(“Error: “ + t.getMessage)
}](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-46-320.jpg)
![Callbacks
Thread 1
val p = Promise[T]
val f: Future[T] = p.future
Thread 2
f onComplete{
case Success(calcResult) = println(calcResult)
case Failure(t) = println(“Error: “ + t.getMessage)
}](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-47-320.jpg)
![Try
Try[T]
Success[T](value: T)
Failure[T](t: throwable)](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-48-320.jpg)
![Try
Try[T]
Success[T](value: T)
Failure[T](t: throwable)
Not Asynchronous](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-49-320.jpg)
![Try
val result: Try[int] = calcSomeValue
result match{
case Success(n) = println(“The result is “ + n)
case Failure(t) = println(“Error: “ + t.getMessage)
}](https://image.slidesharecdn.com/scalableapplicationswithscala-130911045718-phpapp02/85/Scalable-Applications-with-Scala-50-320.jpg)
Scalable Applications with Scala
- 1. Scalable Applications with Scala Nimrod Argov Tikal Knowledge
- 2. What is a Scalable Application?
- 3. What is a Scalable Application? Scalability is the ability of a system, network, or process to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth. -- wikipedia.org
- 4. What is a Scalable Application? Administrative - More People using the system, or more programmers writing it
- 5. What is a Scalable Application? Functional - Adding new functionality with minimal effort
- 6. What is a Scalable Application? Geographical - Maintaining performance, usefulness or usability regardless of locale specific user concentration
- 7. What is a Scalable Application? Load - The ability of a system to expand and contract to accommodate heavier or lighter loads
- 8. Concurrency / Parallelism Optimizing CPU Usage Efficiency
- 9. Scala Scala brings together the Functional and Object Oriented worlds, where until now both sides considered the other to be totally wrong, or the work of the devil. -- Martin Oderski
- 10. Fun!
- 12. Java When is it enough?
- 13. Java When is it enough? Too Little - Race Conditions Too Much - Loss of Parallelism / Deadlocks
- 14. Immutability Concurrency requires Immutability Classes should be immutable unless there's a very good reason to make them mutable....If a class cannot be made immutable, limit its mutability as much as possible. -- Effective Java (Joshua Bloch)
- 15. Immutability //java public final class Student private final int id; private final String name; public Student(int id, String name){ this.id = id; this.name = name; } .. getters… }
- 16. Immutability //java public final class Student private final int id; private final String name; public Student(int id, String name){ this.id = id; this.name = name; } .. getters… }
- 17. Immutability //java List<Student> students = new ArrayList<Student>(); List<Student> immutableStudents = Collections.unmodifiableList(students); List<Students> ohBother = new CopyOnWriteArrayList<Student>();
- 18. Immutability //java List<Student> students = new ArrayList<Student>(); List<Student> immutableStudents = Collections.unmodifiableList(students); students.add(new Student(15, “Harry Potter”)); immutableStudents now shows a different list!
- 19. Scala Collections ● Mutable/Immutable variants ● Immutable variant is imported by default ● Operations are Fast ● Cost of object creation is highly exaggerated by programmers
- 20. Scala Collections ● Set ● Map ● List ● Stream ● Vector ● Stack ● Queue ● Tree
- 21. Scala Collections Many Operations: ● ++ (add all) ● collect ● combinations ● contains ● count ● diff ● endsWith ● ● filter ● find ● flatMap ● flatten ● forEach ● intersect ● lift ● ● min ● max ● mkString ● partition ● permutations ● reduce ● reverse
- 22. Scala Parallel Collections Motivation “The hope was, and still is, that implicit parallelism behind a collections abstraction will bring reliable parallel execution one step closer to the workflow of mainstream developers.” --Scala Documentation
- 23. Scala Parallel Collections Example: Calculate function over list of numbers
- 24. Scala Parallel Collections Example: Calculate function over list of numbers Java: // Somehow cut the list to N parts
- 25. Scala Parallel Collections Example: Calculate function over list of numbers Java: // Somehow cut the list to N parts // Create ForkJoinTask (or Just a Runnable if pre JDK 6)
- 26. Scala Parallel Collections Example: Calculate function over list of numbers Java: // Somehow cut the list to N parts // Create ForkJoinTask (or Just a Runnable if pre JDK 6) // Use ForkJoinPool to run the tasks // Wait for all to finish (join)
- 27. Scala Parallel Collections public class Operation extends RecursiveAction{ private List<Integer> numbers; private List<Integer> results; private int start, end; public Operation(List<Integer> nums){ numbers = nums; start = 0; end = numbers.size(); } protected void doComputation(){ for(int i = start; i < end; i++) results.set(i, calculate(number)); } }
- 28. Scala Parallel Collections public class Operation extends RecursiveAction{ protected static int threshold = 1000; ... public Operation(List<Integer> nums){ … } protected void doComputation(){ … } protected void compute(){ if (numbers.length < threshold) doComputation(); else invokeAll(new SumOperation(numbers, 0, numbers.size() / 2), new SumOperation(numbers, numbers.size() / 2, numbers.size()); }
- 29. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_)
- 30. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95
- 31. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95
- 32. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 11 22 66 11 55 22 66 55 99 33 88 88 55 22 77 11
- 33. Scala Parallel Collections val myParallelList = someList.par myParallelList map calculate(_) 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 5 8 12 7 13 86 14 2 37 23 67 41 1 77 24 95 11 22 66 11 55 22 66 55 99 33 88 88 55 22 77 11 11 22 66 11 55 22 66 55 99 33 88 88 55 22 77 11
- 34. Scala Parallel Collections • ParArray • ParVector • mutable.ParHashMap • mutable.ParHashSet • immutable.ParHashMap • immutable.ParHashSet • ParRange
- 35. Scala Parallel Collections Each collection defines: ● Splitter ● Combiner Default # of threads: how many cores have you got? Underlying implementation depends on configuration
- 36. Scala Parallel Collections ForkJoinTaskSupport (JDK 1.6 or higher) ThreadPoolTaskSupport ExecutionContextTaskSupport
- 37. Scala Parallel Collections <<< WARNING… WARNING… >>> Side Effects or Non-Associative operations will create non-deterministic results!
- 38. Scala Parallel Collections var sum = 0; List(1 to 10000).par.forEach(sum += _) List(1 to 10000).par.reduce(_-_)
- 39. Scala Futures Futures provide a nice way to reason about performing many operations in parallel, in an efficient and non- blocking way. -- Scala Documentation
- 41. Scala Futures val p = Promise[T] val f: Future[T] = p.future
- 42. Java Futures val p = Promise[T] val f: Future[T] = p.future // Java-like variant: doSomeWork() f.get()
- 43. Efficient?
- 44. Efficient? Also, Java Futures are NOT Composable
- 45. What We Want
- 46. Callbacks Thread 1 val p = Promise[T] val f: Future[T] = p.future Thread 2 f onComplete { case Success(calcResult) = println(calcResult) case Failure(t) = println(“Error: “ + t.getMessage) }
- 47. Callbacks Thread 1 val p = Promise[T] val f: Future[T] = p.future Thread 2 f onComplete{ case Success(calcResult) = println(calcResult) case Failure(t) = println(“Error: “ + t.getMessage) }
- 49. Try Try[T] Success[T](value: T) Failure[T](t: throwable) Not Asynchronous
- 50. Try val result: Try[int] = calcSomeValue result match{ case Success(n) = println(“The result is “ + n) case Failure(t) = println(“Error: “ + t.getMessage) }
- 51. Monadic Combinators val result: Int = calcSomeValue getOrElse -1 calcSomeValue.map(doSomethingWithVal).recover (handleException)
- 52. val rateQuote = future { service.getCurrentValue(USD) } val purchase = rateQuote map { quote => if (isProfitable(quote)) service.buy(amount, quote) else throw new Exception("not profitable") } purchase onSuccess { case _ => println("Purchased " + amount + " USD") } Future Composition
- 53. val usdQuote = future { service.getCurrentValue(USD) } val chfQuote = future { service.getCurrentValue(CHF) } val purchase = for { usd <- usdQuote chf <- chfQuote if isProfitable(usd, chf) } yield service.buy(amount, chf) purchase onSuccess { case _ => println("Purchased " + amount + " CHF") } Future Composition
- 54. def generate = future { (random, random) } def calculate(x:(Double, Double)) = future{ pow(x._1, x._2) } def filter(y:Any) = y.toString.contains("22222") while (true) { for { i <- generate j <- calculate(i) if filter(j) } println(j) } Pipe Example