CSCI 2041


Parallel & Concurrent Evaluation


the practice of evaluating more than one instruction at the same time, in parallel.

This makes it a property of a particular architecture:

  • SIMD
  • multi-core
  • GPU…

For example: in the expression (map f l1::l2::l3::[]) we could evaluate (f l1), (f l2), and (f l3) at the same time.

MapReduce: a framework for repeated (parallel) map followed by special reduce operations.

Example: HW3 similarity task

What are some limitations of this approach?

  • Data structure support: lists, arrays, trees?

  • Identifying operations that can be done in parallel

  • Writing programs to exploit degrees of parallelism

  • Managing dependencies and side effects…


the practice of dividing programs into separate threads of execution that might be interleaved in arbitrary order.

Why would you do this?

  • Take advantage of parallelism
  • Interleave i/o and computation
  • Prevent unnecessary waiting (network,GUI…)

The major difficulty with concurrent programs is side effects due to arbitrary (nondeterministic) interleaving of executions.


Ocaml supports threading via the Thread module:

  • Ask OS/runtime for a “thread” t

    let t = Thread.create
  • Give t a function to compute

    (fun () -> print_endline "boring thread");;
  • Tell os to run t

    let result = t ()

Example (Kozen notes)

let prog1 n =
  let result = ref 0 in
  let rec f i j = if j = n then () else
      let v = !result in
      Thread.delay (Random.float 1.);
      result := v + i;
      Printf.printf "Value %d\n" !result;
      flush stdout; f i (j+1) in
  ignore (Thread.create (f 1) 0);
  ignore (Thread.create (f 2) 0)

Can result ever decrease in value? [Let’s find out…]{.class=“fragment”}


In order to prevent this interference between threads, most OS/runtimes provide “locks” that can be used for mutual exclusion:

  • Mutex.create : unit -> Mutex.t creates a mutex that can be “locked” by one thread only.

  • Mutex.lock : Mutex.t -> unit tries to lock a mutex. If it’s already locked, will not return until the lock is acquired.

  • Mutex.try_lock : Mutex.t -> bool returns true if the thread acquires the lock, and false if it is already locked by another thread.

  • Mutex.unlock : Mutex.t -> unit releases the lock on a mutex.

Note: mutexes are a “cooperative” mechanism; if one thread does not use the mutex they cannot guarantee synchronization.

Applying to previous example:

let prog2 n =
  let result = ref 0 in
  let m = Mutex.create () in
  let rec f i j = if j >= n then () else begin
      Thread.delay (Random.float 1.); Mutex.lock m;
      let v = !result in
      Thread.delay (Random.float 1.);
      result := v + i;
      Printf.printf "Value %d\n" !result; flush stdout;
      Mutex.unlock m; f i (j+1) end in
  ignore (Thread.create (f 1) 0);
  ignore (Thread.create (f 2) 0)

Locks can slow down a program, or lead to deadlock when there is a cycle in the “waiting for” graph:

let locker d mlist l =
    let rec loop ls = match ls with
    | [] -> Printf.printf "Thread %s\n" l; flush stdout
    | m::t -> Mutex.lock m; Thread.delay (Random.float d);
        loop t;
        Mutex.unlock m
    in Thread.delay (Random.float d); loop mlist
let deadlocker d =
  let m = Mutex.create () in
  let n = Mutex.create () in
  ignore(Thread.create (locker d [m;n]) "1");
  ignore(Thread.create (locker d [n;m]) "2")

// reveal.js plugins