CSCI 2041

ADVANCED PROGRAMMING PRINCIPLES

Parallel & Concurrent Evaluation

Parallelism

Parallelism:
the practice of evaluating more than one instruction at the same time, in parallel.

This makes it a property of a particular architecture:

SIMD
multi-core
GPU…

For example: in the expression (map f l1::l2::l3::[]) we could evaluate (f l1), (f l2), and (f l3) at the same time.

MapReduce: a framework for repeated (parallel) map followed by special reduce operations.

Example: HW3 similarity task

What are some limitations of this approach?

Data structure support: lists, arrays, trees?
Identifying operations that can be done in parallel
Writing programs to exploit degrees of parallelism
Managing dependencies and side effects…

Concurrency

Concurrency:
the practice of dividing programs into separate threads of execution that might be interleaved in arbitrary order.

Why would you do this?

Take advantage of parallelism
Interleave i/o and computation
Prevent unnecessary waiting (network,GUI…)

The major difficulty with concurrent programs is side effects due to arbitrary (nondeterministic) interleaving of executions.

Threads

Ocaml supports threading via the Thread module:

Ask OS/runtime for a “thread” t
```
let t = Thread.create
```

Give t a function to compute

(fun () -> print_endline "boring thread");;

Tell os to run t
```
let result = t ()
```

Example (Kozen notes)

let prog1 n =
  let result = ref 0 in
  let rec f i j = if j = n then () else
      let v = !result in
      Thread.delay (Random.float 1.);
      result := v + i;
      Printf.printf "Value %d\n" !result;
      flush stdout; f i (j+1) in
  ignore (Thread.create (f 1) 0);
  ignore (Thread.create (f 2) 0)

Can result ever decrease in value? [Let’s find out…]{.class=“fragment”}

Mutex

In order to prevent this interference between threads, most OS/runtimes provide “locks” that can be used for mutual exclusion:

Mutex.create : unit -> Mutex.t creates a mutex that can be “locked” by one thread only.
Mutex.lock : Mutex.t -> unit tries to lock a mutex. If it’s already locked, will not return until the lock is acquired.
Mutex.try_lock : Mutex.t -> bool returns true if the thread acquires the lock, and false if it is already locked by another thread.
Mutex.unlock : Mutex.t -> unit releases the lock on a mutex.

Note: mutexes are a “cooperative” mechanism; if one thread does not use the mutex they cannot guarantee synchronization.

Applying to previous example:

let prog2 n =
  let result = ref 0 in
  let m = Mutex.create () in
  let rec f i j = if j >= n then () else begin
      Thread.delay (Random.float 1.); Mutex.lock m;
      let v = !result in
      Thread.delay (Random.float 1.);
      result := v + i;
      Printf.printf "Value %d\n" !result; flush stdout;
      Mutex.unlock m; f i (j+1) end in
  ignore (Thread.create (f 1) 0);
  ignore (Thread.create (f 2) 0)

Locks can slow down a program, or lead to deadlock when there is a cycle in the “waiting for” graph:

let locker d mlist l =
    let rec loop ls = match ls with
    | [] -> Printf.printf "Thread %s\n" l; flush stdout
    | m::t -> Mutex.lock m; Thread.delay (Random.float d);
        loop t;
        Mutex.unlock m
    in Thread.delay (Random.float d); loop mlist

let deadlocker d =
  let m = Mutex.create () in
  let n = Mutex.create () in
  ignore(Thread.create (locker d [m;n]) "1");
  ignore(Thread.create (locker d [n;m]) "2")