concurrency with gpars

Chicago, October 19 - 22, 2010

Dr Paul King, @paulk_asert

paulk at asert.com.au

Concurrency with GPars

"Andy giveth and Bill taketh away"

GPars - 2

So

urc

e: H

erb

Su

tte

r: h

ttp

://w

ww

.gotw

.ca

/pub

lica

tion

s/c

on

cu

rre

ncy-d

dj.h

tm

Why is it hard?

• Many issues to deal with: – Doing things in parallel, concurrently,

asynchronously

• Processes, Threads, Co-routines, Events, Scheduling

– Sharing/Synchronization Mechanisms

• shared memory, locks, transactions, wait/notify, STM,

message passing, actors, serializability, persistence,

immutability

– Abstractions

• Shared memory on top of messaging passing

• Message passing on top of shared memory

• Dataflow, Selective Communication, Continuations

– Data Structures and Algorithms

• Queues, Heaps, Trees

• Sorting, Graph Algorithms

GPars - 3

GPars - 4

Java Concurrency Features

• The early years – Threads, synchronised and non-synchronised

collections, synchronisation at the language level,

Monitors (wait/notify), Locks, ThreadLocal, final, ...

• More recent enhancements – java.util.concurrent: Executors, Thread Pools,

Optimistic updates, Blocking queues, Synchronizers,

Callables, Futures, Atomic operations, Deques, ...

• Emerging – Fork/Join & others, Kilim, Phasers, PicoThreads ...

• Leverage related APIs/technologies – Networking, real-time, GUIs, simulation, database,

multimedia, operating systems, parallel processing,

distribution, mobile agents, nio, ...

Java Concurrency Best Practice?

• Java Concurrency in Practice:

– “If mutable threads access the

same mutable state variable

without appropriate

synchronization,

your program is broken”

– “When designing thread-safe classes,

good object-oriented techniques –

encapsulation, immutability, and clear

specification of invariants – are your

best friends”

GPars - 5

Topics

Groovy Intro

• Useful Groovy features for Concurrency

• Related Concurrency Libraries & Tools

• GPars

• More Info

GPars - 6

© A

SE

RT

2006-2

010

What is Groovy?

GPars - 7

© A

SE

RT

2006-2

010

• “Groovy is like a super version

of Java. It can leverage Java's

enterprise capabilities but also

has cool productivity features like closures,

DSL support, builders and dynamic typing.”

Groovy = Java – boiler plate code + optional dynamic typing + closures + domain specific languages + builders + metaprogramming

Groovy Starter

GPars - 8

© A

SE

RT

2006-2

010

System.out.println("Hello, World!"); // supports Java syntax println 'Hello, World!' // but can remove some syntax String name = 'Guillaume' // Explicit typing/awareness println "$name, I'll get the car." // Gstring (interpolation) def longer = """${name}, the car is in the next row.""" // multi-line, implicit type assert 0.5 == 1/2 // BigDecimal equals() assert 0.1 + 0.2 == 0.3 // and arithmetic def printSize(obj) { // implicit/duck typing print obj?.size() // safe dereferencing } def pets = ['ant', 'bee', 'cat'] // native list syntax pets.each { pet -> // closure support assert pet < 'dog' // overloading '<' on String } // or: for (pet in pets)...

A Better Java...

GPars - 9

© A

SE

RT

2006-2

010

import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { String s = (String) strings.get(i); if (s.length() <= length) { result.add(s); } } return result; } public static void main(String[] args) { List names = new ArrayList(); names.add("Ted"); names.add("Fred"); names.add("Jed"); names.add("Ned"); System.out.println(names); Erase e = new Erase(); List shortNames = e.removeLongerThan(names, 3); System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { String s = (String) shortNames.get(i); System.out.println(s); } } }

This code

is valid

Java and

valid Groovy

Based on an

example by

Jim Weirich

& Ted Leung

...A Better Java...

GPars - 10

© A

SE

RT

2006-2

010

import java.util.List; import java.util.ArrayList; class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList(); for (int i = 0; i < strings.size(); i++) { String s = (String) strings.get(i); if (s.length() <= length) { result.add(s); } } return result; } public static void main(String[] args) { List names = new ArrayList(); names.add("Ted"); names.add("Fred"); names.add("Jed"); names.add("Ned"); System.out.println(names); Erase e = new Erase(); List shortNames = e.removeLongerThan(names, 3); System.out.println(shortNames.size()); for (int i = 0; i < shortNames.size(); i++) { String s = (String) shortNames.get(i); System.out.println(s); } } }

Do the

semicolons

add anything?

And shouldn‟t

we us more

modern list

notation?

Why not

import common

libraries?

...A Better Java...

GPars - 11

© A

SE

RT

2006-2

010

class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) } } return result } public static void main(String[] args) { List names = new ArrayList() names.add("Ted"); names.add("Fred") names.add("Jed"); names.add("Ned") System.out.println(names) Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) System.out.println(shortNames.size()) for (String s in shortNames) { System.out.println(s) } } }

...A Better Java...

GPars - 12

© A

SE

RT

2006-2

010

class Erase { private List removeLongerThan(List strings, int length) { List result = new ArrayList() for (String s in strings) { if (s.length() <= length) { result.add(s) } } return result } public static void main(String[] args) { List names = new ArrayList() names.add("Ted"); names.add("Fred") names.add("Jed"); names.add("Ned") System.out.println(names) Erase e = new Erase() List shortNames = e.removeLongerThan(names, 3) System.out.println(shortNames.size()) for (String s in shortNames) { System.out.println(s) } } }

Do we need

the static types?

Must we always

have a main

method and

class definition?

How about

improved

consistency?

...A Better Java...

GPars - 13

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } return result } names = new ArrayList() names.add("Ted") names.add("Fred") names.add("Jed") names.add("Ned") System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) for (s in shortNames) { System.out.println(s) }

...A Better Java...

GPars - 14

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) { def result = new ArrayList() for (s in strings) { if (s.size() <= length) { result.add(s) } } return result } names = new ArrayList() names.add("Ted") names.add("Fred") names.add("Jed") names.add("Ned") System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) for (s in shortNames) { System.out.println(s) }

Shouldn‟t we

have special

notation for lists?

And special

facilities for

list processing?

Is „return‟

needed at end?

...A Better Java...

GPars - 15

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) shortNames.each{ System.out.println(s) }

...A Better Java...

GPars - 16

© A

SE

RT

2006-2

010

def removeLongerThan(strings, length) { strings.findAll{ it.size() <= length } } names = ["Ted", "Fred", "Jed", "Ned"] System.out.println(names) shortNames = removeLongerThan(names, 3) System.out.println(shortNames.size()) shortNames.each{ System.out.println(s) }

Is the method

now needed?

Easier ways to

use common

methods?

Are brackets

required here?

...A Better Java

GPars - 17

© A

SE

RT

2006-2

010

names = ["Ted", "Fred", "Jed", "Ned"] println names shortNames = names.findAll{ it.size() <= 3 } println shortNames.size() shortNames.each{ println it }

["Ted", "Fred", "Jed", "Ned"] 3 Ted Jed Ned

Output:

Grapes / Grab: Google collections

GPars - 18

© A

SE

RT

2006-2

010

@Grab('com.google.collections:google-collections:1.0') import com.google.common.collect.HashBiMap HashBiMap fruit = [grape:'purple', lemon:'yellow', lime:'green'] assert fruit.lemon == 'yellow' assert fruit.inverse().yellow == 'lemon'

GPars - 19

© A

SE

RT

2006-2

010

Better Design Patterns: Delegate…

import java.util.Date; public class Event { private String title; private String url; private Date when; public String getUrl() { return url; } public void setUrl(String url) { this.url = url; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } // ...

public Date getWhen() { return when; }

public void setWhen(Date when) { this.when = when; }

public boolean before(Date other) { return when.before(other); }

public void setTime(long time) { when.setTime(time); }

public long getTime() { return when.getTime(); }

public boolean after(Date other) { return when.after(other); } // ...

GPars - 20

© A

SE

RT

2006-2

010

…Better Design Patterns: Delegate…

import java.util.Date; public class Event { private String title; private String url; private Date when; public String getUrl() { return url; } public void setUrl(String url) { this.url = url; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } // ...

public Date getWhen() { return when; }

public void setWhen(Date when) { this.when = when; }

public boolean before(Date other) { return when.before(other); }

public void setTime(long time) { when.setTime(time); }

public long getTime() { return when.getTime(); }

public boolean after(Date other) { return when.after(other); } // ...

boilerplate

GPars - 21

© A

SE

RT

2006-2

010

…Better Design Patterns: Delegate

class Event { String title, url @Delegate Date when }

def gr8conf = new Event(title: "GR8 Conference", url: "http://www.gr8conf.org", when: Date.parse("yyyy/MM/dd", "2009/05/18")) def javaOne = new Event(title: "JavaOne", url: "http://java.sun.com/javaone/", when: Date.parse("yyyy/MM/dd", "2009/06/02")) assert gr8conf.before(javaOne.when)

Why Groovy? Technical Answer

• Minimal learning curve

• Compiles to bytecode

• Java object model & integration

• Annotations

• "Optional" static typing

• Both run-time and compile-time

metaprogramming

GPars - 22

Why Groovy? Adoption Assessment

• Innovators/Thought leaders – Ideas, power, flexibility, novelty, thinking community

• Early adopters – Productivity benefits and collegiate community

– Leverage JVM and potential for mainstream

• Mainstream – Leverage existing Java skills, low learning curve

– Leverage JVM and production infrastructure

– Professional community

– Tools, tools, tools

Topics

• Groovy Intro

Useful Groovy features for Concurrency


• GPars

• More Info

GPars - 24

© A

SE

RT

2006-2

010

Concurrent Programming in Groovy • Java concurrent programming enhancements

– Normal OO methods

– Ability to have immutable types

– Some concurrency building blocks

– Annotations with baked in goodness

• Process/Thread ease of use – AntBuilder and GDK methods

• Closures for greater flexibility – Enabler for concurrency

– Closure is Runnable and Callable

• Third-party libraries – GPars, Functional Java (Actors), Multiverse, JCSP

– Cascading.groovy subproject for Hadoop clusters

– Jetlang, JPPF, GridGain, Google Collections, Gruple

– Groovy actors: http://www.groovyactors.org GPars - 25

Thread & Process Enhancements

• DGM methods – Thread.start{ … }

• Runtime metaprogramming – Add custom control structures

e.g. ReentrantLock.withLock{ … }

• Process enhancements – "ls –l".execute()

– proc1 | proc2 | proc3

– proc1.consumeProcessErrorStream()

– proc2.waitForOrKill(1000)

• AntBuilder – ant.parallel { … }

– ant.exec() GPars - 26

Immutability options

• Built-in

• Google Collections – Numerous improved immutable collection types

• Groovy run-time metaprogramming

• Groovy compile-time metaprogramming

– @Immutable can help us create such classes

– Also gives us @Synchronized and @Lazy GPars - 27

import com.google.common.collect.* List<String> animals = ImmutableList.of("cat", "dog", "horse") animals << 'fish' // => java.lang.UnsupportedOperationException

def animals = ['cat', 'dog', 'horse'].asImmutable() animals << 'fish' // => java.lang.UnsupportedOperationException

def animals = ['cat', 'dog', 'horse'] ArrayList.metaClass.leftShift = { throw new UnsupportedOperationException() } animals << 'fish' // => java.lang.UnsupportedOperationException

@Immutable...

• Java Immutable Class – As per Joshua Bloch

Effective Java

GPars - 28

© A

SE

RT

2006-2

010

public final class Punter { private final String first; private final String last; public String getFirst() { return first; } public String getLast() { return last; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((first == null) ? 0 : first.hashCode()); result = prime * result + ((last == null) ? 0 : last.hashCode()); return result; } public Punter(String first, String last) { this.first = first; this.last = last; } // ...

// ... @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Punter other = (Punter) obj; if (first == null) { if (other.first != null) return false; } else if (!first.equals(other.first)) return false; if (last == null) { if (other.last != null) return false; } else if (!last.equals(other.last)) return false; return true; } @Override public String toString() { return "Punter(first:" + first + ", last:" + last + ")"; } }

...@Immutable...

• Java Immutable Class – As per Joshua Bloch

Effective Java

GPars - 29

© A

SE

RT

2006-2

010

public final class Punter { private final String first; private final String last; public String getFirst() { return first; } public String getLast() { return last; } @Override public int hashCode() { final int prime = 31; int result = 1; result = prime * result + ((first == null) ? 0 : first.hashCode()); result = prime * result + ((last == null) ? 0 : last.hashCode()); return result; } public Punter(String first, String last) { this.first = first; this.last = last; } // ...

// ... @Override public boolean equals(Object obj) { if (this == obj) return true; if (obj == null) return false; if (getClass() != obj.getClass()) return false; Punter other = (Punter) obj; if (first == null) { if (other.first != null) return false; } else if (!first.equals(other.first)) return false; if (last == null) { if (other.last != null) return false; } else if (!last.equals(other.last)) return false; return true; } @Override public String toString() { return "Punter(first:" + first + ", last:" + last + ")"; } }

boilerplate

...@Immutable

GPars - 30

© A

SE

RT

2006-2

010

@Immutable class Punter { String first, last }

Topics

• Groovy Intro


Related Concurrency Libraries & Tools

• GPars

• More Info

GPars - 31

© A

SE

RT

2006-2

010

Lightweight threads: Jetlang

• Jetlang – A high performance threading library

– http://code.google.com/p/jetlang/

GPars - 32

import org.jetlang.fibers.ThreadFiber import org.jetlang.core.Callback import org.jetlang.channels.MemoryRequestChannel import org.jetlang.channels.AsyncRequest def req = new ThreadFiber() // or pool def reply = new ThreadFiber() def channel = new MemoryRequestChannel() req.start() reply.start() channel.subscribe(reply, { it.reply(it.request.sum()) } as Callback) AsyncRequest.withOneReply(req, channel, [3, 4, 5], { println it } as Callback) sleep 1000 req.dispose() reply.dispose()

Other High-Level Libraries: JPPF

– Open source Grid Computing platform

– http://www.jppf.org/

GPars - 33

import org.jppf.client.* import java.util.concurrent.Callable

class Task implements Callable, Serializable { private static final long serialVersionUID = 1162L public Object call() { println 'Executing Groovy' "Hello JPPF from Groovy" } } def client = new JPPFClient() def job = new JPPFJob() def task = new Task() job.addTask task def results = client.submit(job) for (t in results) { if (t.exception) throw t.exception println "Result: " + t.result }

Other High-Level Libraries: Gruple...

– http://code.google.com/p/gruple

– Simple abstraction to coordinate and synchronize

threads with ease – based on Tuplespaces

• Tuplespaces provide the illusion of a shared memory on top

of a message passing system, along with a small set of

operations to greatly simplify parallel programming

– Example Tuple: [fname:"Vanessa", lname:"Williams", project:"Gruple"]

– Basic operations within a Tuplespace are:

• put - insert a tuple into the space

• get - read a tuple from the space (non-destructively)

• take - take a tuple from the space (a destructive read)

– Further reading: Eric Freeman, Susanne Hupfer, and

Ken Arnold. JavaSpaces Principles, Patterns, and

Practice, Addison Wesley, 1999

GPars - 34

…Other High-Level Libraries: Gruple...

GPars - 35

import org.gruple.SpaceService

def defaultSpace = SpaceService.getSpace()

defaultSpace << [fname:"Vanessa",

lname:"Williams",

project:"Gruple"]

println defaultSpace.get(fname:"Vanessa",

lname:"Williams",

project:"Gruple")

[project:Gruple, lname:Williams, fname:Vanessa]

Other High-Level Libraries: ...Gruple...

– Mandelbrot example (included in Gruple download)

GPars - 36

... Space space = SpaceService.getSpace("mandelbrot") Map template = createTaskTemplate() Map task String threadName = Thread.currentThread().name while(true) { ArrayList points task = space.take(template) println "Worker $threadName got task ${task['start']} for job ${task['jobId']}" points = calculateMandelbrot(task) Map result = createResult(task['jobId'], task['start'], points) println "Worker $threadName writing result for task ${result['start']} for job ${result['jobId']}" space.put(result) } ...

Other High-Level Libraries: ...Gruple

GPars - 37

Other High-Level Libraries: Cascading.groovy

– API/DSL for executing tasks on a Hadoop cluster

– http://www.cascading.org/

GPars - 38

def assembly = builder.assembly(name: "wordcount") { eachTuple(args: ["line"], results: ["word"]) { regexSplitGenerator(declared: ["word"], pattern: /[.,]*\s+/) } group(["word"]) everyGroup(args: ["word"], results: ["word", "count"]) { count() } group(["count"], reverse: true) } def map = builder.map() { source(name: "wordcount") { hfs(input) { text(["line"]) } } sink(name: "wordcount") { hfs(output) { text() } } } def flow = builder.flow(name: "wordcount", map: map, assembly: assembly)

Other High-Level Libraries: GridGain…

– Simple & productive to use grid computing platform

– http://www.gridgain.com/

GPars - 39

class GridHelloWorldGroovyTask extends GridTaskSplitAdapter<String, Integer> { Collection split(int gridSize, Object phrase) throws GridException { // ... } Object reduce(List results) throws GridException { // ... } }

import static GridFactory.* start() def grid = getGrid() def future = grid.execute(GridHelloWorldGroovyTask, "Hello World") def phraseLen = future.get() stop(true)

…Other High-Level Libraries: GridGain

• http://gridgain.blogspot.com/2010/10/worlds-shortest-mapreduce-

app.html

GPars - 40

words = "Counting Letters In This Phrase".split(' ') map = new C1() { def apply(word) { word.size() } } reduce = sumIntReducer() println grid.forkjoin(SPREAD, yield(words, map), reduce) // => 27

grid.forkjoin(SPREAD,yield("Counting Letters In This Phrase".split(' '), new C1(){def apply(w){w.size()}}),sumReducer())

Multiverse STM…

– http://www.multiverse.org/

GPars - 41

import org.multiverse.api.GlobalStmInstance import org.multiverse.api.Transaction import org.multiverse.templates.TransactionTemplate import org.multiverse.transactional.refs.LongRef def from = new Account(10) def to = new Account(10) atomic { from.balance -= 5 to.balance += 5 } println "from $from.balance" println "to $to.balance" ...

…Multiverse STM…

GPars - 42

... void atomic(Closure block) { atomic([:], block) } void atomic(Map args, Closure block) { boolean readonly = args['readonly'] ?: false boolean trackreads = args['trackreads'] ?: true def txFactory = GlobalStmInstance.globalStmInstance. transactionFactoryBuilder. setReadonly(readonly). setReadTrackingEnabled(trackreads).build() new TransactionTemplate(txFactory) { Object execute(Transaction transaction) { block.call() return null } }.execute() } ...

…Multiverse STM

GPars - 43

class Account { private final balance = new LongRef() Account(long initial) { balance.set initial } void setBalance(long newBalance) { if (newBalance < 0) throw new RuntimeException("not enough money") balance.set newBalance } long getBalance() { balance.get() } }

Testing multi-threaded applications: ConTest...

• Advanced Testing for Multi-Threaded Applications

– Tool for testing, debugging, and coverage-measuring

of concurrent programs (collects runtime statistics)

– Systematically and transparently (using a java agent)

schedules the execution of program threads in ways

likely to reveal race conditions, deadlocks, and other

intermittent bugs (collectively called synchronization

problems) with higher than normal frequency

– The ConTest run-time engine adds heuristically

controlled conditional instructions (adjustable by a

preferences file) that force thread switches, thus

helping to reveal concurrent bugs. You can use

existing tests and run ConTest multiple times – by

default different heuristics used each time it is run

• http://www.alphaworks.ibm.com/tech/contest

GPars - 44

...Testing multi-threaded applications: ConTest

GPars - 45

NUM = 5

count = 0

def incThread = { n -> Thread.start{

sleep n*10

//synchronized(ParalInc) {

count++

//}

} }

def threads = (1..NUM).collect(incThread)

threads.each{ it.join() }

assert count == NUM

targetClasses = ParalInc

timeoutTampering = true

noiseFrequency = 500

strength = 10000

Exception in thread "main" Assertion failed:

assert count == NUM

| | |

4 | 5

false

> groovyc ParalInc.groovy

> java -javaagent:../../Lib/ConTest.jar -cp %GROOVY_JAR%;. ParalInc

ParalInc.groovy

GPars - 46

GContracts

@Grab('org.gcontracts:gcontracts:1.0.2') import org.gcontracts.annotations.* @Invariant({ first != null && last != null }) class Person { String first, last @Requires({ delimiter in ['.', ',', ' '] }) @Ensures({ result == first + delimiter + last }) String getName(String delimiter) { first + delimiter + last } } new Person(first: 'John', last: 'Smith').getName('.')

1.8+

Testing: Spock

GPars - 47

class HelloSpock extends spock.lang.Specification { def "length of Spock's and his friends' names"() { expect: name.size() == length where: name | length "Spock" | 5 "Kirk" | 4 "Scotty" | 6 } }

Topics

• Groovy Intro



GPars

• More Info

GPars - 48

© A

SE

RT

2006-2

010

Ralph Johnson: Parallel Programming Patterns…

GPars - 49

© A

SE

RT

2006-2

010

http://strangeloop2010.com/talk/presentation_file/14485/Johnson-DataParallelism.pdf

…Ralph Johnson: Parallel Programming Patterns

GPars - 50

© A

SE

RT

2006-2

010


GPars • http://gpars.codehaus.org/

• Library classes and DSL sugar providing

intuitive ways for Groovy developers to

handle tasks concurrently. Logical parts:

– Actors provide a Groovy implementation of Scala-like

actors including "remote" actors on other machines

– Dataflow Concurrency supports natural shared-memory

concurrency model, using single-assignment variables

– Asynchronizer extends the Java 1.5 built-in support for

executor services to enable multi-threaded collection and

closure processing

– Parallelizer uses JSR-166y Parallel Arrays to enable

multi-threaded collection processing

– Safe a non-blocking mt-safe reference to mutable state

that is inspired by "agents" in Clojure

GPars - 51

© A

SE

RT

2006-2

010

GPars: Parallel Collection Functions

GPars - 52

© A

SE

RT

2006-2

010

def nums = 1..100000 def squares = nums .collect{ it ** 2 } .grep{ it % 7 == it % 5 } .grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296]

@Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.withPool

def nums = 1..100000 withPool(5) { def squares = nums. collectParallel{ it ** 2 }. grepParallel{ it % 7 == it % 5 }. grepParallel{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] }

GPars: Transparent Parallel Collections

• Applies some Groovy metaprogramming

GPars - 53

© A

SE

RT

2006-2

010

import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 nums.makeTransparent() def squares = nums. collect{ it ** 2 }. grep{ it % 7 == it % 5 }. grep{ it % 3 == 0 } println squares[0..3] + "..." + squares[-3..-1] assert squares[0..3] == [36, 144, 1089, 1296] }

GPars concurrency-aware methods Transparent Transitive? Parallel Lazy?

any { ... } anyParallel { ... } yes

collect { ... } yes collectParallel { ... }

count(filter) countParallel(filter)

each { ... } eachParallel { ... }

eachWithIndex { ... } eachWithIndexParallel { ... }

every { ... } everyParallel { ... } yes

find { ... } findParallel { ... }

findAll { ... } yes findAllParallel { ... }

findAny { ... } findAnyParallel { ... }

fold { ... } foldParallel { ... }

fold(seed) { ... } foldParallel(seed) { ... }

grep(filter) yes grepParallel(filter)

groupBy { ... } groupByParallel { ... }

max { ... } maxParallel { ... }

max() maxParallel()

min { ... } minParallel { ... }

min() minParallel()

split { ... } yes splitParallel { ... }

sum sumParallel // foldParallel +

GPars - 54 Transitive means result is automatically transparent; Lazy means fails fast

GPars: Map-Reduce...

GPars - 55

© A

SE

RT

2006-2

010

import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. collection }

...GPars: Map-Reduce

GPars - 56

© A

SE

RT

2006-2

010

import static groovyx.gpars.GParsPool.withPool withPool(5) { def nums = 1..100000 println nums.parallel. map{ it ** 2 }. filter{ it % 7 == it % 5 }. filter{ it % 3 == 0 }. reduce{ a, b -> a + b } }

GPars parallel array methods

Method Return Type

combine(initValue) { ... } Map

filter { ... } Parallel array

collection Collection

groupBy { ... } Map

map { ... } Parallel array

max() T

max { ... } T

min() T

min { ... } T

reduce { ... } T

reduce(seed) { ... } T

size() int

sort { ... } Parallel array

sum() T

parallel // on a Collection Parallel array

GPars - 57

Parallel Collections vs Map-Reduce

GPars - 58

Fork Fork

Join Join

Map

Map

Reduce

Map

Map

Reduce

Reduce

Map

Filter

Filter Map

GPars: Dataflows...

GPars - 59

© A

SE

RT

2006-2

010

import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.result = flow.x + flow.y } task { flow.x = 10 } task { flow.y = 5 } assert 15 == flow.result

new DataFlows().with { task { result = x * y } task { x = 10 } task { y = 5 } assert 50 == result }

...GPars: Dataflows...

• Evaluating:

GPars - 60

© A

SE

RT

2006-2

010

import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.a = 10 } task { flow.b = 5 } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } assert flow.result == 75

b

10 5

a

+ -

*

result = (a – b) * (a + b)

x y

Question: what happens if I change the order of the task statements here?


• Naive attempt for loops

GPars - 61

© A

SE

RT

2006-2

010

import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() [10, 20].each { thisA -> [4, 5].each { thisB -> task { flow.a = thisA } task { flow.b = thisB } task { flow.x = flow.a - flow.b } task { flow.y = flow.a + flow.b } task { flow.result = flow.x * flow.y } println flow.result } } // => java.lang.IllegalStateException: A DataFlowVariable can only be assigned once.

... task { flow.a = 10 } ... task { flow.a = 20 }


GPars - 62

© A

SE

RT

2006-2

010

import groovyx.gpars.dataflow.DataFlowStream import static groovyx.gpars.dataflow.DataFlow.* final streamA = new DataFlowStream() final streamB = new DataFlowStream() final streamX = new DataFlowStream() final streamY = new DataFlowStream() final results = new DataFlowStream() operator(inputs: [streamA, streamB], outputs: [streamX, streamY]) { a, b -> streamX << a - b; streamY << a + b } operator(inputs: [streamX, streamY], outputs: [results]) { x, y -> results << x * y } [[10, 20], [4, 5]].combinations().each{ thisA, thisB -> task { streamA << thisA } task { streamB << thisB } } 4.times { println results.val }

b

10

10

20

20

4

5

4

5

a

+ -

*

84

75

384

375

...GPars: Dataflows

• Amenable to static analysis

• Race conditions avoided

• Deadlocks “typically” become repeatable

GPars - 63

© A

SE

RT

2006-2

010

import groovyx.gpars.dataflow.DataFlows import static groovyx.gpars.dataflow.DataFlow.task final flow = new DataFlows() task { flow.x = flow.y } task { flow.y = flow.x }

GPars: Dataflow Sieve

GPars - 64

© A

SE

RT

2006-2

010

final int requestedPrimeNumberCount = 1000 final DataFlowStream initialChannel = new DataFlowStream() task { (2..10000).each { initialChannel << it } } def filter(inChannel, int prime) { def outChannel = new DataFlowStream() operator([inputs: [inChannel], outputs: [outChannel]]) { if (it % prime != 0) { bindOutput it } } return outChannel } def currentOutput = initialChannel requestedPrimeNumberCount.times { int prime = currentOutput.val println "Found: $prime" currentOutput = filter(currentOutput, prime) }

Source: http://groovyconsole.appspot.com/script/235002

GPars: Actors...

• Actors provide explicit coordination: they

don‟t share state, instead coordinating via

asynchronous messages – Contrasting with predefined coordination for fork/join &

map/filter/reduce & implicit coordination for dataflow

– Messages are processed one at a time normally in the

order they were sent (which is non-deterministic due to

asynchronous nature)

– Some actor systems allowing message delivery to be

prioritised; others allow for sharing some (readonly) state;

some allow remote actors for load balancing/robustness

• Not new in concept – But has received recent publicity due to special

support in Erlang, Scala and other languages GPars - 65

© A

SE

RT

2006-2

010

…GPars: Actors...

• Class with the following lifecycle &

methods – But also DSL sugar & enhancements

GPars - 66

© A

SE

RT

2006-2

010

start() stop() act() send(msg) sendAndWait(msg) loop { } react { msg -> } msg.reply(replyMsg) receive() join()

…GPars: Actors...

GPars - 67

© A

SE

RT

2006-2

010

import static groovyx.gpars.actor.Actors.* def decrypt = reactor { code -> code.reverse() } def audit = reactor { println it } def main = actor { decrypt 'terces pot' react { plainText -> audit plainText } } main.join() audit.stop() audit.join()

Source: ReGina

…GPars: Actors...

GPars - 68

© A

SE

RT

2006-2

010

final class FilterActor extends DynamicDispatchActor { private final int myPrime private def follower def FilterActor(final myPrime) { this.myPrime = myPrime; } def onMessage(int value) { if (value % myPrime != 0) { if (follower) follower value else { println "Found $value" follower = new FilterActor(value).start() } } } def onMessage(def poisson) { if (follower) { def sender = poisson.sender follower.sendAndContinue(poisson, {this.stop(); sender?.send('Done')}) //Pass the poisson along and stop after a reply } else { //I am the last in the chain stop() reply 'Done' } } }


…GPars: Actors

GPars - 69

© A

SE

RT

2006-2

010

(2..requestedPrimeNumberBoundary).each { firstFilter it } firstFilter.sendAndWait 'Poisson'


Dining Philosophers…

GPars - 70

Philosopher Thinking | Eating





Fork Available | InUse





…Dining Philosophers

GPars - 71











Accepted | Rejected

Take | Release Take | Release

Dining Philosophers: Actors...

GPars - 72

© A

SE

RT

2006-2

010

// adapted from GPars example, repo: http://git.codehaus.org/gitweb.cgi?p=gpars.git // file: src/test/groovy/groovyx/gpars/samples/actors/DemoDiningPhilosophers.groovy @Grab('org.codehaus.gpars:gpars:0.10') import groovyx.gpars.actor.* import groovy.beans.Bindable def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] Actors.defaultActorPGroup.resize names.size() class Philosopher extends AbstractPooledActor { private random = new Random() String name int timesEaten = 0 def forks @Bindable String status void act() { assert 2 == forks.size() loop { think() forks*.send new Take() react {a -> react {b -> if ([a, b].any {Rejected.isCase it}) { [a, b].find {Accepted.isCase it}?.reply new Release() } else { eat() [a, b]*.reply new Release() } } } } }

…Dining Philosophers: Actors...

GPars - 73

© A

SE

RT

2006-2

010

… void think() { setStatus('thinking') sleep random.nextInt(5000) setStatus('') } void eat() { setStatus("eating ${++timesEaten}") sleep random.nextInt(3000) setStatus('') } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } } final class Take {} final class Accepted {} final class Rejected {} final class Release {}

…Dining Philosophers: Actors...

GPars - 74

© A

SE

RT

2006-2

010

… class Fork extends AbstractPooledActor { String name boolean available = true void act() { loop { react {message -> switch (message) { case Take: if (available) { available = false reply new Accepted() } else reply new Rejected() break case Release: assert !available available = true break default: throw new IllegalStateException("Cannot process the message: $message") } } } } } def forks = (1..names.size()).collect { new Fork(name: "Fork $it") } def philosophers = (1..names.size()).collect { new Philosopher(name: names[it - 1], forks: [forks[it - 1], forks[it % names.size()]]) }

…Dining Philosophers: Actors

GPars - 75

© A

SE

RT

2006-2

010

… import groovy.swing.* import java.awt.Font import static javax.swing.JFrame.* def frame = new SwingBuilder().frame(title: 'Philosophers', defaultCloseOperation: EXIT_ON_CLOSE) { vbox { hbox { (0..<names.size()).each { i -> def widget = textField(id: names[i], text: names[i].center(14)) widget.font = new Font(widget.font.name, widget.font.style, 36) philosophers[i].propertyChange = { widget.text = philosophers[i].status.center(14) } } } } } frame.pack() frame.visible = true forks*.start() sleep 1000 philosophers*.start() sleep 10000 forks*.stop() forks*.join() philosophers*.stop() philosophers*.join() frame.dispose() philosophers.each { println it }

socrates has eaten 3 times

plato has eaten 3 times

aristotle has eaten 6 times

descartes has eaten 2 times

nietzsche has eaten 5 times

Dining Philosophers: CSP...

GPars - 76

© A

SE

RT

2006-2

010

// inspired by similar examples at the web sites below: // http://www.cs.kent.ac.uk/projects/ofa/jcsp/ // http://www.soc.napier.ac.uk/~jmk/#_Toc271192596 @Grab('org.codehaus.gpars:gpars:0.10') import org.jcsp.lang.* import groovyx.gpars.csp.PAR import groovyx.gpars.csp.ALT import static java.lang.System.currentTimeMillis

def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] enum ForkAction { Take, Release, Stop } import static ForkAction.*

class Philosopher implements CSProcess { ChannelOutput leftFork, rightFork String name def forks = [] private random = new Random() private timesEaten = 0 private start = currentTimeMillis()

void run() { while (currentTimeMillis() - start < 10000) { think() eat() } [leftFork, rightFork].each { it.write(Stop) } println toString() } …

…Dining Philosophers: CSP...

GPars - 77

© A

SE

RT

2006-2

010

… void think() { println "$name is thinking" sleep random.nextInt(50) } void eat() { [leftFork, rightFork].each { it.write(Take) } println "$name is EATING" timesEaten++ sleep random.nextInt(200) [leftFork, rightFork].each { it.write(Release) } } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } }

…Dining Philosophers: CSP...

GPars - 78

© A

SE

RT

2006-2

010

… class Fork implements CSProcess { ChannelInput left, right private active = [0, 1] as Set void run() { def fromPhilosopher = [left, right] def forkAlt = new ALT(fromPhilosopher) while (active) { def i = forkAlt.select() read fromPhilosopher, i, Take read fromPhilosopher, i, Release } } void read(phil, index, expected) { if (!active.contains(index)) return def m = phil[index].read() if (m == Stop) active -= index else assert m == expected } } …

…Dining Philosophers: CSP

GPars - 79

© A

SE

RT

2006-2

010

… def lefts = Channel.createOne2One(names.size())

def rights = Channel.createOne2One(names.size())

def philosophers = (0..<names.size()).collect { i ->

return new Philosopher(leftFork: lefts[i].out(),

rightFork: rights[i].out(),

name: names[i])

}

def forks = (0..<names.size()).collect { i ->

return new Fork(left: lefts[i].in(),

right: rights[(i + 1) % names.size()].in())

}

def processList = philosophers + forks

new PAR(processList).run()

Why CSP?

• Amenable to proof

and analysis

GPars - 80 Picture source: http://wotug.org/parallel/theory/formal/csp/Deadlock/

GPars: Agents...

• Agents safeguard non-thread safe objects

• Only the agent can update the underlying

object

• “Code” to update the protected object is

sent to the agent

• Can be used with other approaches

GPars - 81

© A

SE

RT

2006-2

010

…GPars: Agents

GPars - 82

© A

SE

RT

2006-2

010

@Grab('org.codehaus.gpars:gpars:0.10')

import groovyx.gpars.agent.Agent

def speakers = new Agent<List>(['Alex'], {it?.clone()}) // add Alex

speakers.send {updateValue it << 'Hilary'} // add Hilary

final Thread t1 = Thread.start {

speakers.send {updateValue it << 'Ken'} // add Ken

}

final Thread t2 = Thread.start {

speakers << {updateValue it << 'Guy'} // add Guy

speakers << {updateValue it << 'Ralph'} // add Ralph

}

[t1, t2]*.join()

assert new HashSet(speakers.val) ==

new HashSet(['Alex', 'Hilary', 'Ken', 'Guy', 'Ralph'])

Source: Gpars examples

GPars for testing

GPars - 83

© A

SE

RT

2006-2

010

@Grab('net.sourceforge.htmlunit:htmlunit:2.6') import com.gargoylesoftware.htmlunit.WebClient @Grab('org.codehaus.gpars:gpars:0.10') import static groovyx.gpars.GParsPool.*

def testCases = [ ['Home', 'Bart', 'Content 1'], ['Work', 'Homer', 'Content 2'], ['Travel', 'Marge', 'Content 3'], ['Food', 'Lisa', 'Content 4'] ]

withPool(3) { testCases.eachParallel{ category, author, content -> postAndCheck category, author, content } }

private postAndCheck(category, author, content) { ...

Guy Steele example in Groovy…

GPars - 84

© A

SE

RT

2006-2

010

def words = { s -> def result = [] def word = '' s.each{ ch -> if (ch == ' ') { if (word) result += word word = '' } else word += ch } if (word) result += word result } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == []

Sequential version

Guy Steele‟s example from keynote (from slide 52 onwards for several slides):

http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf


GPars - 86

© A

SE

RT

2006-2

010

Guy Steele‟s example from keynote (from slide 52 onwards for several slides):

http://strangeloop2010.com/talk/presentation_file/14299/GuySteele-parallel.pdf


GPars - 87

© A

SE

RT

2006-2

010

@Immutable class Chunk { String s def plus(Chunk other) { new Chunk(s + other.s) } def plus(Segment other) { new Segment(s + other.l, other.m, other.r) } }

@Immutable class Segment { String l; List m; String r def plus(Chunk other) { new Segment(l, m, r + other.s) } def plus(Segment other) { new Segment(l, m + maybeWord(r + other.l) + other.m, other.r) } }

class Util { static processChar(ch) { ch == ' ' ? new Segment('', [], '') : new Chunk(ch) } static maybeWord(s) { s ? [s] : [] } } import static Util.* ...

Refactored sequential version


GPars - 88

© A

SE

RT

2006-2

010

def words = { s -> def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } case null: return [] } } assert words("This is a sample") == ['This', 'is', 'a', 'sample'] assert words(" Here is another sample ") == ['Here', 'is', 'another', 'sample'] assert words("JustOneWord") == ['JustOneWord'] assert words("Here is a sesquipedalian string of words") == ['Here', 'is', 'a', 'sesquipedalian', 'string', 'of', 'words'] assert words(" ") == [] && words("") == []

Refactored sequential version


GPars - 89

© A

SE

RT

2006-2

010

def swords = { s -> def result s.each{ ch -> if (!result) result = processChar(ch) else result += processChar(ch) } result ?: new Chunk('') } THREADS = 4 def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def map = new java.util.concurrent.ConcurrentHashMap() (0..<THREADS).collect { i -> Thread.start { def (min, max) = [[s.size(),i*n].min(), [s.size(),(i+1)*n].min()] map[i] = swords(s[min..<max]) }}*.join() def result = map.entrySet().sort{ it.key }.sum{ it.value } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } }

Roll your own threading with ConcurrentHashMap version


GPars - 90

© A

SE

RT

2006-2

010

def words = { s -> int n = (s.size() + THREADS - 1) / THREADS def min = (0..<THREADS).collectEntries{ [it, [s.size(),it*n].min()] } def max = (0..<THREADS).collectEntries{ [it, [s.size(),(it+1)*n].min()] } def result = new DataFlows().with { task { a = swords(s[min[0]..<max[0]]) } task { b = swords(s[min[1]..<max[1]]) } task { c = swords(s[min[2]..<max[2]]) } task { d = swords(s[min[3]..<max[3]]) } task { sum1 = a + b } task { sum2 = c + d } task { sum = sum1 + sum2 } println 'Tasks ahoy!' sum } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } }

DataFlow version: partially hard-coded to 4 partitions for easier reading


GPars - 91

© A

SE

RT

2006-2

010

GRANULARITY_THRESHHOLD = 10 THREADS = 4 println GParsPool.withPool(THREADS) { def result = runForkJoin(0, input.size(), input){ first, last, s -> def size = last - first if (size <= GRANULARITY_THRESHHOLD) { swords(s[first..<last]) } else { // divide and conquer def mid = first + ((last - first) >> 1) forkOffChild(first, mid, s) forkOffChild(mid, last, s) childrenResults.sum() } } switch(result) { case Chunk: return maybeWord(result.s) case Segment: return result.with{ maybeWord(l) + m + maybeWord(r) } } }

Fork/Join version


GPars - 92

© A

SE

RT

2006-2

010

THRESHHOLD = 10 def split(raw) { raw.size() <= THRESHHOLD ? raw : [raw[0..<THRESHHOLD]] + split(raw.substring(THRESHHOLD)) } println GParsPool.withPool(THREADS) { def ans = split(input).parallel.map(swords).reduce{ a,b -> a + b } switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } }

Map/Reduce version

…Guy Steele example in Groovy

GPars - 93

© A

SE

RT

2006-2

010

println GParsPool.withPool(THREADS) { def ans = input.collectParallel{ processChar(it) }.sum() switch(ans) { case Chunk: return maybeWord(ans.s) case Segment: return ans.with{ maybeWord(l) + m + maybeWord(r) } } }

Just leveraging the algorithm‟s parallel nature

More Information about Concurrency

• Web sites – http://gpars.codehaus.org/

– http://g.oswego.edu/

Doug Lea's home page

– http://gee.cs.oswego.edu/dl/concurrency-interest/

– http://jcip.net/

Companion site for Java Concurrency in Practice

– http://www.eecs.usma.edu/webs/people/okasaki/pubs.html#cup98

Purely Functional Data Structures

– http://delicious.com/kragen/concurrency

Concurrency bookmark list

– http://www.gotw.ca/publications/concurrency-ddj.htm

The Free Lunch is Over, Herb Sutter

– http://manticore.cs.uchicago.edu/papers/damp07.pdf

– http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10142

Concepts, Techniques, and Models of Computer Programming

GPars - 95

http://gpars.codehaus.org/

http://g.oswego.edu/

http://gee.cs.oswego.edu/dl/concurrency-interest/



http://jcip.net/

http://www.eecs.usma.edu/webs/people/okasaki/pubs.html

http://delicious.com/kragen/

http://www.gotw.ca/publications/concurrency-ddj.htm



http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=10142

More Information about Groovy

• Web sites – http://groovy.codehaus.org

– http://grails.codehaus.org

– http://pleac.sourceforge.net/pleac_groovy (many examples)

– http://www.asert.com.au/training/java/GV110.htm (workshop)

• Mailing list for users – [email protected]

• Information portals – http://www.aboutgroovy.org

– http://www.groovyblogs.org

• Documentation (1000+ pages) – Getting Started Guide, User Guide, Developer Guide, Testing

Guide, Cookbook Examples, Advanced Usage Guide

• Books – Several to choose from ...

GPars - 96

http://groovy.codehaus.org/

http://grails.codehaus.org/

http://pleac.sourceforge.net/pleac_groovy

http://www.asert.com.au/training/java/GV110.htm

http://www.aboutgroovy.org/

http://www.groovyblogs.org/

More Information: Groovy in Action

GPars - 97

Contains a

chapter on

GPars!

Topics

• Bonus Material – Multiverse Philosophers

– Jetlang Philosophers

– Gruple Philosophers

GPars - 98

© A

SE

RT

2006-2

010

Multiverse Philosophers…

GPars - 99

//@Grab('org.multiverse:multiverse-core:0.7-SNAPSHOT') //@Grab('org.multiverse:multiverse-alpha:0.7-SNAPSHOT') //@Grab('org.multiverse:multiverse-groovy:0.7-SNAPSHOT') //@GrabConfig(systemClassLoader=true, initContextClassLoader = true) // adapted multiverse Groovy example: http://git.codehaus.org/gitweb.cgi?p=multiverse.git // file: multiverse-groovy/src/test/groovy/org/multiverse/integration/ org/multiverse/integration/examples/DiningPhilosphersTest.groovy import org.multiverse.transactional.refs.BooleanRef import org.multiverse.transactional.refs.IntRef import static MultiverseGroovyLibrary.* def food = new IntRef(5) def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] def forks = (1..5).collect { new Fork(id: it, free: new BooleanRef(true)) } def philosophers = (0..4).collect { new Philosopher(name: names[it], food: food, left: forks[(it + 1) % 5], right: forks[it]) } def threads = philosophers.collect { new Thread(it) } threads*.start() threads*.join() philosophers.each { println it } class Fork { int id BooleanRef free void take() { free.set(false) } void release() { free.set(true) } }

…Multiverse Philosophers

GPars - 100

class Philosopher implements Runnable { String name Fork left, right IntRef timesEaten = new IntRef() IntRef food

void eat() { atomic(trackreads: true, explicitRetryAllowed: true) { left.free.await(true) right.free.await(true) if (food.get() > 0) { left.take(); right.take() timesEaten.inc(); sleep 10; food.dec() } } }

void think() { atomic(trackreads: true, explicitRetryAllowed: true) { left.release(); right.release() } sleep 10 }

void run() { 10.times { eat(); think() } }

String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } }

Jetlang Philosophers…

GPars - 101

import org.jetlang.core.Callback import org.jetlang.fibers.ThreadFiber import org.jetlang.channels.* def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] class Philosopher implements Callback { private random = new Random() String name int timesEaten = 0 String status def forks private channels = [new MemoryRequestChannel(), new MemoryRequestChannel()] private req = new ThreadFiber() // or from pool private reply = new ThreadFiber() private responses = [] private gotFork = { it instanceof Accepted } void start() { assert forks.size() == 2 req.start() reply.start() (0..1).each{ channels[it].subscribe(reply, forks[it]) } think() } String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } …

…Jetlang Philosophers…

GPars - 102

… void think() { println(name + ' is thinking') sleep random.nextInt(3000) (0..1).each{ AsyncRequest.withOneReply(req, channels[it], new Take(it), this); } }

void eat() { timesEaten++ println toString() sleep random.nextInt(2000) }

void onMessage(Object message) { responses << message if (responses.size() == 2) { if (responses.every(gotFork)) { eat() } responses.findAll(gotFork).each { int index = it.index channels[index].publish(req, new Release(index), forks[index]) } responses = [] think() } } }

@Immutable class Take { int index } @Immutable class Accepted { int index } @Immutable class Rejected { int index } @Immutable class Release { int index } …

…Jetlang Philosophers

GPars - 103

… class Fork implements Callback { String name def holder = [] void onMessage(message) { def msg = message instanceof Request ? message.request : message def index = msg.index switch (msg) { case Take: if (!holder) { holder << index message.reply(new Accepted(index)) } else message.reply(new Rejected(index)) break case Release: assert holder == [index] holder = [] break default: throw new IllegalStateException("Cannot process the message: $message") } } } def forks = (1..names.size()).collect { new Fork(name: "Fork $it") } def philosophers = (1..names.size()).collect { new Philosopher(name: names[it - 1], forks: [forks[it - 1], forks[it % names.size()]]) } philosophers*.start() sleep 10000 philosophers.each { println it }

Gruple Philosophers…

GPars - 104

import org.gruple.SpaceService import org.gruple.Space

class Philosopher { private random = new Random() String name Space space private timesEaten = 0 int id, num boolean done = false

void run() { while (true) { think() if (done) return space.take(fork: id) space.take(fork: (id + 1) % num) eat() space.put(fork: id) space.put(fork: (id + 1) % num) } }

void think() { println "$name is thinking" sleep random.nextInt(500) }

void eat() { println "$name is EATING" timesEaten++ sleep random.nextInt(1000) } …

… socrates is thinking nietzsche is thinking descartes is EATING aristotle is EATING descartes is thinking plato is EATING aristotle is thinking socrates is EATING plato is thinking nietzsche is EATING socrates is thinking nietzsche is thinking descartes is EATING descartes is thinking socrates has eaten 5 times plato has eaten 4 times aristotle has eaten 4 times descartes has eaten 4 times nietzsche has eaten 5 times

…Gruple Philosophers

GPars - 105

… String toString() { switch (timesEaten) { case 0: return "$name has starved" case 1: return "$name has eaten once" default: return "$name has eaten $timesEaten times" } } } def names = ['socrates', 'plato', 'aristotle', 'descartes', 'nietzsche'] def diningSpace = SpaceService.getSpace('Dining') def philosophers = (0..<names.size()).collect{ new Philosopher(name: names[it], id: it, space: diningSpace, num: names.size()) } (0..<names.size()).each{ diningSpace << [fork: it] } sleep 500 def threads = (0..<names.size()).collect{ n -> Thread.start{ philosophers[n].run() } } sleep 10000 philosophers*.done = true sleep 2000 threads.join() println() philosophers.each{ println it }

concurrency with gpars

Technology