Towards Aggregate Programming in Scala

Towards Aggregate Programming in Scala
Roberto Casadei Mirko Viroli
Department of Computer Science and Engineering
University of Bologna
1st International Workshop on Programming Models and Languages for
Distributed Computing (PMLDC), 2016, Rome
R. Casadei (Università di Bologna) Towards Aggregate Programming in Scala PMLDC 2016 1 / 46

Outline
1 Context and Issues
2 Aggregate Computing
3 Aggregate Programming in Scala
4 Summary

Context and Issues
Outline
4 Summary

Context and Issues
Context
Environment + (Mobile, Large-scale) Networks of { people + devices }

Context and Issues
The systems of (((((((
tomorrow today...
Complex/Collective Adaptive Systems (CASs)
• Socio-Technical Systems (STS)
• Internet of Things (IoT)
• Pervasive systems
• Ubiquitous systems
• Cyber-Physical Systems (CPS)
• Smart-cities/buildings/homes, ...

Context and Issues
Ecosystems of CASs services
The crowd engineering example

Context and Issues
Gathering local context

Context and Issues
Sensing global patterns of data

Context and Issues
Crowd detection

Context and Issues
Crowd anticipation

Context and Issues
Crowd-aware steering

Context and Issues
Crowd dispersal

Context and Issues
Crowd evacuation upon alerts

Context and Issues
What? How?
The problem
• Large-scale
• Complex dynamics
• Substantial unpredictability
Approach
• Decentralisation
• Adaptation
• To occasional disruptions
• To on-going perturbation
• To device distribution
• To available infrastructure
Design issues
• Abstraction gap — i.e., the distance from the problem to the solution
• Capturing complex behaviours in a simple way
• Supporting modularity and reusability
• Guiding engineering for resilience

Aggregate Computing
Outline
4 Summary

Aggregate Computing
The origins: self-organisation patterns [FMSM+
13]

Aggregate Computing
The origins: space-time programming [BV15]
Space-oriented computation – Thematic areas
(a) Intensive computing
(b) Computation embedded in space
(c) Space computation
Decentralised spatial computing
Computing “somewhere” [Duc13]
• Location-related information
• Spatial constraints to communication
Space-time programming [BV15]
• Computation expressed in terms of properties of the physical time and
space in which it occurs
• Spatial abstractions

Aggregate Computing
Discrete system vs. continuous space-time

Aggregate Computing
Computational ﬁelds
• (Abstract interpretation) Mapping space-time to computational objects
• (Concrete interpretation) Mapping devices to values: φ : δ →
• “Distributed” data structure working as the global abstraction

Aggregate Computing
The Computational Field Calculus I
Field calculus
A calculus of computational ﬁelds [DVB16]
• Functional
• Basis set of operators for ﬁeld manipulation
• Minimal expressive model
≈ Space-time universal
Higher-Order Field Calculus (HOFC) [DVPB15]
• First-class distributed functions
• Code mobility

Aggregate Computing
The Computational Field Calculus II
Syntax
e = x | v | eλ(e) | rep(e0){eλ} | nbr{e } Expression
v = <standard-values> | λ Value
λ = f | o | (x) ⇒ e Functional value
F = def f(x){e } Function definition
• v includes numbers, booleans, strings, collections, custom ADTs, etc.
• f is a user-defined function
• o is a built-in functional operator (e.g., pure math or a sensor)
A sort of λ-calculus where values are computational fields, plus
(i) a construct for field evolution (rep)
(ii) a construct for interacting with the neighbourhood (nbr)

Aggregate Computing
The Computational Field Calculus III
Global-level semantics, intuitively
0
(x)=>x+1
true t<0,1>
()0
1
+
-
1
-1
ef(0,1)
ef
rep
0
(x)=>x+1
t
v0
t
v1
..
rep(0){(x)=>x+1}
nbr de
nbr{e}
φd=[d1→v1,..,dn→vn]

Aggregate Computing
Example: the channel I

Aggregate Computing
Example: the channel II

Aggregate Computing
Aggregate functions [DVPB15]
Left: branch(c){ f(x) }{ g(x) } Right: (branch(c){ f }{ g })(x)

Aggregate Computing
Aggregate programming [BPV15]
Programming (collective adaptive) systems: viewpoints
(a) Local viewpoint (device-centric view)
(b) Global viewpoint (aggregate view)
Aggregate programming: what
Goal: programming the collective behaviour of aggregates
Global-to-local mapping
Aggregate programming: how
Prominent approach founded on ﬁeld calculus and self-org patterns
Composable self-organisation
• Self-stabilisation
• Building blocks for resilient coordination

Aggregate Computing
Device-centric programming
Issues
• Local-to-global mapping problem (generally intractable)
• Explicit design of adaptation and communication
• Mixing of concerns — state management, interaction, adaptation, resiliency, etc.

Aggregate Computing
Aggregate-level programming
Collective services on the computational fabric available in space
• Automatic global-to-local mapping
• Implicit adaptation and communication
• Composition of loosely coupled services

Aggregate Computing
Aggregate (computing) systems
Structure
A set of networked devices
• Each one is given the same ﬁeld calculus program
Dynamics
Each device computes the given program at asyn / partially-sync rounds of
execution
(1) Retrieve context
⇐ Messages from neighbours
⇐ Sensor values
(2) Aggregate program execution
⇒ export
(3) Broadcast export to neighbourhood
(4) Execute actuators

Aggregate Computing
Aggregate computing and abstraction
(Partial) insensitiveness to system structure and situation details
Aggregate behaviours are highly insensitive to:
• network size
• network density
• network topology
This makes algorithms intrinsically robust to failures and changes to working
conditions.
(Partial) insensitiveness to execution strategy
Note: network nodes ultimately correspond to components operating in some
context via sensors/actuators.
Aggregate computations can be carried out in different ways:
• Computing nodes and “native” local communication;
• Computations performed by a central server;
• Computations performed in the cloud; ...
Idea: opportunistically exploit the underlying infrastructure

Aggregate Computing
Aggregate Programming Stack

Aggregate Computing
Approaching the engineering of CASs
Approaching the aforementioned issues
• Abstraction gap — i.e., the distance from the problem to the solution
=⇒ Abstractions close to problem domain
=⇒ Declarative approach – from how to what
=⇒ Layered approach – raising abstraction incrementally
• Capturing complex behaviours in a simple way
=⇒ Instrumental assumptions + effective building blocks
=⇒ Balance of top-down and bottom-up problem solving
• Supporting modularity and reusability
=⇒ Compositional approach
• Guiding engineering for resilience
=⇒ Adaptation by construction

Aggregate Computing
Aggregate Programming Toolchain
Up to a few months ago..
Protelis [PVB15] + Alchemist simulator [PMV13]
• External DSL
• Focus on simulation
Technological gap: there is no aggregate programming framework, aimed at the
construction of real-world applications, integrated in a ≈mainstream language
⇒ SCAFI (scala with computational ﬁelds)
• Scala-internal DSL for expressing aggregate computations
• Distributed platform for execution of aggregate systems
SCAFI: where?
http://scafi.apice.unibo.it
• A more serious release soon :)

Aggregate Programming in Scala
Outline
4 Summary

SCAFI: Scala with Computational Fields
Goal
The goal is to provide an integrated framework for building systems with aggregate
programming
• Scala-internal DSL for expressing aggregate computations
• Linguistic support + execution support (interpreter/VM)
• Correct, complete, efficient implementation of the HOFC semantics
• Distributed platform for execution of aggregate systems
• Support for multiple architectural styles and system configurations
• Support for code mobility
Issues
• Implementation of a language for the field calculus within Scala
• Constructs as method calls
• Integration with the Scala type system
• Development of a distributed middleware
• Typical issues from distribution (concurrency, communication, fault-tolerance)
⇒ Akka actor framework

SCAFI: language
trait Constructs {
def rep[A](init: A)
(fun: (A) => A): A
def nbr[A](expr: => A): A
def foldhood[A](init: => A)
(acc: (A,A)=>A)
(expr: => A): A
def branch[A](cond: => Boolean)
(th: => A)
(el: => A): A
def aggregate[A](f: => A): A
def sense[A](name: LSNS): A
def nbrvar[A](name: NSNS): A
}
Where are computational ﬁelds?

Example: the channel I
def channel(src: Boolean, dest: Boolean, width: Double) =
distanceTo(src) + distanceTo(dest) <=
distBetween(src, dest) + width

Example: the channel (full code)
def channel(src: Boolean, dest: Boolean, width: Double) =
distanceTo(src) + distanceTo(dest) <=
distBetween(src, dest) + width
def nbrRange(): Double = nbrvar[Double](NBR_RANGE_NAME)
def G[V:OB](src: Boolean, field: V, acc: V=>V, metric: =>Double): V =
rep( (Double.MaxValue, field) ){ dv =>
mux(src) { (0.0, field) } {
minHoodMinus {
val (d, v) = nbr { (dv._1, dv._2) }
(d + metric, acc(v))
}
}
}._2
def broadcast[V:OB](source: Boolean, field: V): V =
G[V](source, field, x=>x, nbrRange())
def distanceTo(source: Boolean): Double =
G[Double](source, 0, _ + nbrRange(), nbrRange())
def distBetween(source: Boolean, target: Boolean): Double =
broadcast(source, distanceTo(target))
def isSource = sense[Boolean]("source")
def isObstacle = sense[Boolean]("obstacle")

P2p, ad-hoc net
// STEP 1: CHOOSE INCARNATION
import it.unibo.scafi.incarnations.{ BasicActorP2P => Platform }
// STEP 2: DEFINE AGGREGATE PROGRAM SCHEMA
class Demo0C_AggregateProgram extends Platform.AggregateProgram {
override def main(): Any = foldhood(0){_ + _}(1)
}
// STEP 3: DEFINE MAIN PROGRAM
object Demo0C_MainProgram extends Platform.CmdLineMain
1) Demo_MainProgram -h 127.0.0.1 -p 9000
-e 1:2,4,5;2;3 --subsystems
127.0.0.1:9500:4:5
--program "demos.Demo_AggregateProgram"
2) Demo_MainProgram -h 127.0.0.1 -p 9500
-e 4;5:4
--program "demos.Demo_AggregateProgram"

Server-based, mobile spatial net I
import it.unibo.scafi.distrib.actor.server.{SpatialPlatform =>
SpatialServerBasedActorPlatform}
import it.unibo.scafi.incarnations.BasicAbstractActorIncarnation
import it.unibo.scafi.space.{Point2D, BasicSpatialAbstraction}
object Demo3_Platform extends BasicAbstractActorIncarnation
with SpatialServerBasedActorPlatform
with BasicSpatialAbstraction with Serializable {
override val LocationSensorName: String = "LOCATION_SENSOR"
override type P = Point2D
override def buildNewSpace[E](elems: Iterable[(E,P)]): SPACE[E] =
new Basic3DSpace(elems.toMap) { override val proximityThreshold = 1.1 }
}
// STEP 2: DEFINE AGGREGATE PROGRAM SCHEMA
class Demo3_AggregateProgram extends Demo3_Platform.AggregateProgram {
def hopGradient(source: Boolean): Double = {
rep(Double.PositiveInfinity){
hops => { mux(source) { 0.0 } { 1+minHood(nbr{ hops }) } }
}
}
def main() = hopGradient(sense("source"))
}

Server-based, mobile spatial net II
// STEP 3: DEFINE MAIN PROGRAMS
object Demo3_ServerMain extends Demo3_Platform.ServerCmdLineMain {
override def refineSettings(s: Demo3_Platform.Settings) = {
s.copy(profile = s.profile.copy(
serverGuiActorProps = tm => Some(ServerGUIActor.props(Demo3_Platform, tm
))
))
}
}
object Demo4_MainProgram extends Demo3_Platform.CmdLineMain {
override def refineSettings(s: Demo3_Platform.Settings) = {
s.copy(profile = s.profile.copy(
devGuiActorProps = ref => Some(DevGUIActor.props(Demo3_Platform, ref))
))
}
override def onDeviceStarted(dm: Demo3_Platform.DeviceManager,
sys: Demo3_Platform.SystemFacade) = {
dm.addSensorValue(Demo3_Platform.LocationSensorName, Point2D(dm.selfId%5,(
dm.selfId/5.0).floor))
dm.addSensorValue("source", dm.selfId==4)
dm.start
}
}

Server-based, mobile spatial net III

Summary
Outline
4 Summary

Summary
Summary: key ideas and future directions
Aggregate programming
• An (((((((
programmingengineering approach to CASs
• Formally grounded in the Field Calculus
• Allows to compose “emergent” phenomena
• Provides layers of building blocks (proven to self-stabilise!)
A Scala framework for aggregate programming is out!
• Supports the construction of aggregate systems
• Integrated in Scala
No need to learn ad-hoc external DSLs
Easier to get started and play
• Better release (e.g., to Maven Central) soon, I promise :)
Many open problems [VB] (want to join?)
• Space-time universality
• Fields vs. objects/agents/processes/worﬂows
• Better resilience properties than self-stabilisation
• Adapting the execution strategy to available infrastructure (e.g., cloud, fog, ...)

Appendix References
References I
[BPV15] Jacob Beal, Danilo Pianini, and Mirko Viroli.
Aggregate Programming for the Internet of Things.
IEEE Computer, 2015.
[BV15] Jacob Beal and Mirko Viroli.
Space–time programming.
Phil. Trans. R. Soc. A, 373(2046):20140220, 2015.
[Duc13] Matt Duckham.
Decentralized Spatial Computing - Foundations of Geosensor Networks.
Springer, 2013.
[DVB16] Ferruccio Damiani, Mirko Viroli, and Jacob Beal.
A type-sound calculus of computational ﬁelds.
Science of Computer Programming, 117:17 – 44, 2016.
[DVPB15] Ferruccio Damiani, Mirko Viroli, Danilo Pianini, and Jacob Beal.
Code mobility meets self-organisation: A higher-order calculus of
computational ﬁelds.
volume 9039 of Lecture Notes in Computer Science, pages 113–128.
Springer International Publishing, 2015.

Appendix References
References II
[FMSM+
13] Jose Luis Fernandez-Marquez, Giovanna Di Marzo Serugendo, Sara
Montagna, Mirko Viroli, and Josep Lluís Arcos.
Description and composition of bio-inspired design patterns: a complete
overview.
Natural Computing, 12(1):43–67, 2013.
[PMV13] Danilo Pianini, Sara Montagna, and Mirko Viroli.
Chemical-oriented simulation of computational systems with alchemist.
Journal of Simulation, 7(3):202–215, 2013.
[PVB15] Danilo Pianini, Mirko Viroli, and Jacob Beal.
Protelis: practical aggregate programming.
In Proceedings of the 30th Annual ACM Symposium on Applied
Computing, pages 1846–1853. ACM, 2015.
[VB] Mirko Viroli and Jacob Beal.
Resiliency with Aggregate Computing : State of the Art and Roadmap.

Towards Aggregate Programming in Scala

Recommended

More Related Content

Viewers also liked (6)

Similar to Towards Aggregate Programming in Scala (20)

More from Roberto Casadei (19)

Recently uploaded (20)

Towards Aggregate Programming in Scala