Implementation of 'go-like' language constructions in scala [english version]

Go / Scala
Implementation of go language constructions
in scala
Ruslan Shevchenko
<ruslan@shevchenko.kiev.ua>
https://github.com/rssh/scala-gopher

Whats in go ?

scope

( defer/ panic / destroy )

parallelism

channels

select statement

Differences from 'native' scala approach

How to implement go variant
Future directions...

Scope
def copy(inf: File, outf: File): Long = goScope {
val in = new FileInputStream(inf)
defer{ in.close() }
val out = new FileOutputStream(outf);
defer{ out.close() }
out.getChannel() transferFrom(in.getChannel(), 0, Long.MaxValue)
}
def copy(inf: File, outf: File): Long =
{
try {
val out = new FileOutputStream(outf)
try {
} finally {
out.close();
}
} finally {
in.close();
}
}
Old way:

Traditional way:
def copy(inf: File, outf: File): Long =
{
try {
val out = new FileOutputStream(out)
try {
out.getChannel() transferFrom(in.getChannel(), 0,Long.MaxValue)
} finally {
out.close();
}
} finally {
in.close();
}
}

def copy(inf: File, outf: File): Long = goScope {
defer{ in.close() }
defer{ out.close() }
}
Go way

try
try
try
finally
finally
finally
op/defer

How this done is scala:

Macroses

Call-by-name-params

Traditional try/cath
goScope = .... (macro wich rewrite arg)
defer => @compileTimeOnly

def copy(inf: File, outf: File): Long = {
val sc0 = new ScopeContext();
sc0.eval {
sc0.pushDefer{ in.close() }
sc0.pushDefer{ out.close() }
}
sc0.unwindDefers[Long]()
}
Unsugared:

Scope: defer / panic / recover

panic => throw new PanicException()

Recover => return
try {
....
} catch {
case ex: Exception =>
return x
}
defer{
.........
recover(x)
}

Implementation of 'go-like' language constructions in scala [english version]

Channels
Read:
go: channel -> x
Scala:
channel ~> x
x = channel ?
?!
??
Write
go: channel <- x
Scala:
channel <~ x
channel ! x
!! (immediatly)
!? (with timeout)

Select
Channels connect execution flows

go func() {
for {
select {
case msg1 := <- c1:
fmt.Println(msg1)
case msg2 := <- c2:
fmt.Println(msg2)
}
}
}()
Go: typical pattern:
select inside for inside goroutine

go
for(s <- select) s match {
case c1 ~> (msg1: Msg1Type) => println(msg1)
case c2 ~>(msg2:Msg2Type) => println(msg2)
}
Scala Analog:
- implemented as foreach macros
- ~> pattern: require typing
- active queue
Open question: native non-macro syntax

Unsugared (simplicified) [1]:
go {
val s = new SelectContext();
s.addReader(c1) {
(msg1:Msg1Type) => println(msg1)
}
s.addReader(c2) {
}
s.run()
}

Unsugared (simplicified) [2]:
val scope = new ScopeContext();
val s = new SelectContext();
Future {
s.addReader(c1) {
}
s.addReader(c2) {
}
} map { s.runUnblocked() }
map { scope.finish() }
(Future work)
S

Just select:
for(s <- select.once) s match {
case c1 ~> (msg1: Msg1Type) => println(msg1)
case c2 ~>(msg2:Msg2Type) => println(msg2)
}
select {
case msg1 := <- c1: fmt.Println(msg1)
case msg2 := <- c2: fmt.Println(msg2)
}
GO
SCALA

Internal implementation:

Active channel

Channel = [Blocked Query + Execution Context ]

Coordination primitives + wait for select

Count down latch

Memory barrier

Extensions over Go model

async operations

context shutdown.

val consumer = go {
for(s <- select) {
s match {
case `channel` ~> (i:Int) =>
sum = sum + i
if (i==1000) s.shutdown()
}
}
sum
}
Future[Int]

Akka
(Erlang-like actors)
Channels (go-like)

Reactive

No blocking operations

Request/Reply paradigm.

Mailbox overflow
on overload

Handle something.

Flow oriented

Wait is normal
(switch during wait)

One-direction pipe paradigm

wait on send
On overload

Compute something

x1, x2, x3, ............
x1*y1, x2*y2, x3*y3, ............
y1, y2, y3, ............
go {
while(;) {
val x = channelX ?
val y = channelY ?
channelZ ! x*y
}
}
Example: Easy with channels, hard with Akka

bindWrite
bindRead
? [blocking operation]

Future directions

Redefining of base primitivies
[select in loop in go] is too fundamental, to be
combination of 3 constructions.

Native non-macro syntax

Make scala type inference works

Pluggable implementations

Optimize corner cases

Pluggable implementations.

Exists many strategies.
(current implemented is not best)

Schedule/callback instead waits

Play with own dispatcher

.....

Performance tests,

Implement/Prototype/Test loop,

Problems:

Lack of import-based configuration in scala

@exported import

Community help needed.

Rewriting blocking API into reactive
- like CPS (continuations passing style)

go {
...
val x = channelX ?
val y = channelY ?
channelZ ! x*y
...
}
go { .....
channelX.read map { x =>
channelY.read map { y =>
channelZ.write map { ch =>
ch <~! x*y
} } }
}
}

someCollection foldLeft {
(s,x) => s + (x* channel ?)
}
Problem – same as with CPS:
- foldLeft – implemented not here.
- foreach
for(x <- something) channel <~~ x
Continuations not supported directly by JVM

Continuations in JVM
- Patched JVM
- Byte-code rewriting.
- Do nothing – thread pools now big
(utilize wait by running
Flow of non-blocking tasks here)

The Da Vinci machine
http://openjdk.java.net/projects/mlvm/index.html
AVIAN VM
http://oss.readytalk.com/avian/

Someday everything will be fine.
For now - here is a cat

Continuations: Bytecode rewriting.
Frames:
Let's save frame during function end, instead deleting one.
foldLeft
? channel
+
*

Frames:
foldLeft
+
*

Frames:
Suspended
CPS transformation on bytecode level
foldLeft
+
*
Channel ?

Community help needed.
https://github.com/rssh/scala-gopher

Build examples of typical usage.

Instrument ones for performance tests

Provide better implementations.
Let's think together. Thanks for attention.
Ruslan Shevchenko
<Ruslan@Shevchenko.Kiev.UA>

Implementation of 'go-like' language constructions in scala [english version]

More Related Content

What's hot (20)

Viewers also liked (12)

Similar to Implementation of 'go-like' language constructions in scala [english version] (20)

More from Ruslan Shevchenko (16)

Recently uploaded (20)

Implementation of 'go-like' language constructions in scala [english version]