Multi-Threading : Doing Things in Parallel

Steven Zeil

Last modified: Mar 22, 2014

Parallelism - Motivation

Speed
Responsiveness
Clean design

1. Overview

Concurrency

Concurrency refers to the ability of a program to perform operations in parallel.
- Faster on hardware with multiple CPU’s, or CPU’s with parallel capability
- Cleaner designs (sometimes) even when only a single CPU available
- Allows useful work when waiting for I/O

1.1 Fundamental Ideas

Definitions

Operations are concurrent if they could be executed in parallel.
- Whether they are or not depends on hardware, compiler, etc.
Operations that must occur in a fixed sequence are sequential.
A process is a sequential calculation.
Implemented in op sys as a separate execution state
- execution counter & other registers
- activation stack
- possibly other data

Scheduler

A scheduler is

responsible for deciding
- which process gets the CPU,
- and for how long
Selection is made from among ready processes

Running Processes

A process that is running may

finish its computation
- ⇒ terminated
lose the CPU when its time slice runs out
- ⇒ ready
request a slow or unavailable resource
- ⇒ blocked

Blocked Processes

A process that is blocked on some resource

becomes ready when the resource is available
still must wait for the scheduler to select it from among all the ready processes

Processes can Interact

Communication between process may be via

messages
shared variables
- “shared” means visible to code of both processes

Synchronization

Synchronization

relates two or more threads
restricts the order in which the collection of 2 or more processes can perform events
Critical to making concurrent software safe

1.2 Parallel versus Concurrent

Concurrent software can run in many hardware configurations.

single processor
multi-processor
distributed

Single Processor

one process can run while another is waiting for I/O
time slicing
reactive systems
- e.g., user manipulates a windowed view of a lengthy calculation while that calculation is still in progress

Multi-Processor

CPU’s may
- work on distinct tasks,
- or cooperate on a common goal
Any single-CPU software designs can be mapped here

Distributed Processors

“shared” data must be replicated
much harder to control
- in many cases, the cheapest hardware solution
e.g. off-the-shelf workstations connected via a network

Looking Ahead

We’ll concentrate on the shared memory models (single and multi-processors).

2. Spawning Processes

How do we get multiple processes in one program?

Start with one, launch others
May eventually need to assemble info from each spawned process into overall solution

2.1 Processes in Unix - Fork

Usually, each time you type a command into the shell, a new process is launched to handle that command.
Can also write programs that split into multiple processes

Concurrency in C++/Unix

Languages like C and C++ lack built-in support for concurrency.

can sometimes do concurrent programming through special libraries
tend to be OS and compiler-specific

Unix Model

Unix employs a distributed process model.

processes do not share memory but communicate via messages.

Process Control

Basic operations:

fork() creates a copy of the current process, identical down to every byte in memory, except that
- the fork()call returns 0 to the new copy (“child”) and
- it returns the non-zero process ID of the child process to the original “parent”
wait(int*) suspends the process until some child process completes.

Example: Forking a Process

 int status;
 if (fork() == 0) {
    /* child 1 */
     execl("./prog1", "prog1", 0);
 } else if (fork() == 0) {
    /* child 2 */
    execl("./prog2", "prog2", 0);
 } else {
    /* parent */
    wait(&status);
    wait(&status);
 }

Process Communication

Unix processes can communicate via a pipe, a communication path that appears to the program as a pair of files, one for input, the other for output.

Reading from an empty pipe will block a process until data is available (written into the pipe by another process).

Example of a Pipe

 char str[6];
 int mypipe[2];
 int status;
 pipe (mypipe); /* establishes a pipe.         */
                /* We read from mypipe[0] and  */
                /* write to mypipe[1]          */
 
 if (fork() == 0) {     /* child */
    read(mypipe[0], str, 6);
    printf (str);
 } else { /* parent */
    write (mypipe[1], "Hello", 6);
    wait (&status);
 }
 close mypipe[0];
 close mypipe[1];

Overall

This approach to process communication is

relatively simple
relatively safe - lack of shared memory reduces the possibilities for processes to interfere with one another
- still possible though (e.g., files)
awkward for programs that need to pass complex data
“heavy weight” - each process is relatively expensive

Library vs. Language

Library-based concurrency tends to be non-portable.

A big advantage to having concurrency built in as part of a language standard
- But may lead to bad matches with some hardware configurations
Only two such languages in common use: Ada and Java

2.2 Heavy & Light Weight Processes

A Unix process includes its entire memory image

Such heavy-weight processes may consume a lot of system resource and be slow to start up
Discourages programs from spawning large numbers of child processes

Threads

A thread is a light-weight process that

Has its own execution location and activation stack
Typically shares heap and static memory with its parent

Some OS’s support this directly

Otherwise the Java runtime system implements multiple threads within a single heavy-weight OS process

3. Java Threads

Java has built-in support for threads
In fact, Java programs with GUIs already have 2 or more threads
- The initial thread starts in main().
  - It usually builds the GUI and signals when it is to be displayed.
- The event handler thread waits for input events and invokes the appropriate listeners when the events are detected.
  - repaint() calls are (usually?) handled as a simulated input event: queued up and eventually resulting in a paint().

Example: ThreadArt

As an example of the use of threads in a Java GUI, let’s add some dynamic behavior to the StringArt program of the earlier lesson on Java GUIs
The thread will
- Increment a counter that ranges from 0 to 99 and then back to zero
- Schedule a repaint() for each value of the counter

Using the counter to affect the drawing


public void drawLines(Graphics g, Dimension d)
{
    int dmin = (d.width < d.height) ? d.width : d.height;

    if (stepSize < 1)
        stepSize = 1;

    Point center = new Point();
    center.x = (double)d.width/2.0;
    center.y = (double)d.height/2.0;

    int k = Math.abs(cycleCounter - cycleLength/2);
    int theta = 60 * cycleCounter / cycleLength; 

    for (int i = 0; i < 60; ++i) {
        int radius = dmin/2;
        Point origin = ptOnCircle(6*i+thetai, radius, center);
        int j = i + stepSize;
        while (j >= 60)
            j -= 60;
        while (i != j) {
            Point destination = ptOnCircle(6*j+theta, radius, center);
            Color c = iinterpolate(colors[0], colors[1], k, cycleLength/2);i
            g.setColor(c);
            g.drawLine ((int)origin.x, (int)origin.y,
                    (int)destination.x, (int)destination.y);
            j += stepSize;
            while (j >= 60)
                j -= 60;
        }
    }
}

The counter is used to
- Compute an angle offset
  - Which is added to the angles at which all points a computed
- Interpolate between two colors

3.1 Taking the Cheap Way Out

The easiest way to animate this code is to

Add a Timer to the program
- Adds an action event to the queue every 50 milliseconds
An action listener then can increment the counter
- and request a repaint

ThreadArtByTimer.java

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.JApplet;
import javax.swing.JButton;
import javax.swing.JColorChooser;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JTextField;
import javax.swing.Timer;


/**
 * A simple example of GUI event handling in a Java application.
 * 
 * This can be run as a main program or as an applet.
 * 
 * @author zeil
 *
 */


public class ThreadArtByTimer extends JApplet {

    private boolean startedInAnApplet;

    // The Model
    private Color[] colors;
    private int stepSize = 5;

    private int cycleLength;
    private int cycleCounter;

    // The View & Controls
    private JFrame window;
    private JPanel canvas;
    private JButton colorChooser1;
    private JButton colorChooser2;
    private JTextField stepSizeIn;
    
    private Timer timer;


    private class ColorChooser implements ActionListener {
        private JButton button;
        private int colorNum;

        public ColorChooser (JButton button, int colorNum) {
            this.button = button;
            this.colorNum = colorNum;
        }

        @Override
        public void actionPerformed(ActionEvent arg0) {
            Color chosen = JColorChooser.showDialog(window, "Choose a color", colors[colorNum]);
            if (chosen != null) {
                colors[colorNum] = chosen;
                setColor (button, chosen);
                canvas.repaint();
            }
        }
    };


    /**
     * Action that slowly changes the color of the drawing 
     *
     */
    public class ColorChanger implements ActionListener {

        public ColorChanger()
        {
        }
        
        

        @Override
        public void actionPerformed(ActionEvent arg0) {
            cycleCounter = (cycleCounter + 1) % cycleLength;
            canvas.repaint();
        }
        
    }




    public ThreadArtByTimer()
    {
        startedInAnApplet = false;
        window = null;
        colors = new Color[2];
        colors[0] = Color.red;
        colors[1] = Color.blue;
        cycleLength = 100;
        cycleCounter = 0;
    }


    public static void main (String[] args)
    {
        ThreadArtByTimer instance = new ThreadArtByTimer();
        instance.createAndShowGUI();  
    }

    public void createAndShowGUI() {
        window = new JFrame();
        // set up the components
        window.getContentPane().setLayout (new BorderLayout());

        canvas = new JPanel () {
            public void paint (Graphics g) {
                super.paint(g);
                drawLines (g, getSize());
            }
        };
        canvas.setBackground(Color.white);
        window.getContentPane().add (canvas, BorderLayout.CENTER);
        canvas.setPreferredSize(new Dimension(400, 400));

        JPanel controls = new JPanel();

        colorChooser1 = new JButton("Color 1");
        controls.add (colorChooser1);
        setColor(colorChooser1, colors[0]);
        colorChooser1.addActionListener (new ColorChooser(colorChooser1, 0));

        colorChooser2 = new JButton("Color 2");
        controls.add (colorChooser2);
        setColor(colorChooser2, colors[1]);
        colorChooser2.addActionListener (new ColorChooser(colorChooser2, 1));

        stepSizeIn = new JTextField (""+stepSize, 5);
        controls.add (stepSizeIn);
        stepSizeIn.addActionListener (new ActionListener()
        {
            public void actionPerformed(ActionEvent e) {
                try {
                    Integer newSize = new Integer(stepSizeIn.getText());
                    stepSize = newSize.intValue();
                    canvas.repaint();
                } catch (Exception ex) {};
            }
        });

        window.getContentPane().add (controls, BorderLayout.SOUTH);

        window.setDefaultCloseOperation((startedInAnApplet) ? JFrame.DISPOSE_ON_CLOSE : JFrame.EXIT_ON_CLOSE);


        timer = new Timer(50, new ColorChanger());
        timer.start();

        window.pack();
        window.setVisible(true);
    }

    /**
     * Sets the background color of a button to the indicated color.
     * Changes the foreground to wither black or white, depending on
     * which will give more contrast agasint the new background.
     * 
     * @param button
     * @param color
     */
    private void setColor(JButton button, Color color) {
        button.setBackground(color);
        int brightness = color.getRed() + color.getGreen() + color.getBlue(); // max of 3*255
        if (brightness > 3*255/2) {
            // This is a fairly bright color. Use black lettering
            button.setForeground (Color.black);
        } else {
            // This is a fairly dark color. Use white lettering
            button.setForeground (Color.white);
        }
    }

    // Applet functions

    public void init() {
        startedInAnApplet = true;
    }

    public void start() {
        if (window == null)
            createAndShowGUI();
    }

    public void stop() {
    }

    public void destroy() {
    }




    int interpolate (int x, int y, int i, int steps)
    {
        return (i * x + (steps-i)*y) / steps;
    }


    Color interpolate(Color c1, Color c2, int i, int steps)
    {
        return new Color (interpolate(c1.getRed(), c2.getRed(), i, steps),
                interpolate(c1.getGreen(), c2.getGreen(), i, steps),
                interpolate(c1.getBlue(), c2.getBlue(), i, steps));
    }


    class Point {
        double x;
        double y;
    }

    Point ptOnCircle (int degrees, int radius, Point center) 
    {
        Point p = new Point();
        double theta = Math.toRadians((double)degrees);
        p.x = center.x + (double)radius * Math.cos(theta);
        p.y = center.y + (double)radius * Math.sin(theta);
        return p;
    }

    public void drawLines(Graphics g, Dimension d)
    {
        int dmin = (d.width < d.height) ? d.width : d.height;

        if (stepSize < 1)
            stepSize = 1;

        Point center = new Point();
        center.x = (double)d.width/2.0;
        center.y = (double)d.height/2.0;

        int k = Math.abs(cycleCounter - cycleLength/2);
        int theta = 60 * cycleCounter / cycleLength; 
        
        for (int i = 0; i < 60; ++i) {
            int radius = dmin/2; //interpolate(dmin/4, dmin/2, k, cycleLength/2);
            Point origin = ptOnCircle(6*i+theta, radius, center);
            int j = i + stepSize;
            while (j >= 60)
                j -= 60;
            while (i != j) {
                Point destination = ptOnCircle(6*j+theta, radius, center);
                Color c = interpolate(colors[0], colors[1], k, cycleLength/2);
                g.setColor(c);
                g.drawLine ((int)origin.x, (int)origin.y,
                        (int)destination.x, (int)destination.y);
                j += stepSize;
                while (j >= 60)
                    j -= 60;
            }
        }
    }


}

Let’s Pretend That Never Happened

Timers are easy to work with, but not always powerful enough
- Not good if the recurring event may sometimes take a long time
  - Because it runs in the event handler thread, a time- consuming listener would “lock up” the GUI
- Does not offer very flexible control options
For educational purposes, therefore, let’s look at how we might move that timed action into a separate thread
- not the event handler thread

3.2 Working with Threads

The Java Thread class

Any subclasses of the system class Thread can have a function member run() that executes as a distinct process (per object).
- Never call run() directory.
A Thread object is not actually a new thread (process) unless we start() it
- The start function
  - creates a new thread with its own activation stack, but sharing heap memory with the other threads
  - Starts that thread runnign with a call to the object’s run() function.
- The thread terminates when control returns from run().

ThreadArt

ThreadArt.java

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.JApplet;
import javax.swing.JButton;
import javax.swing.JColorChooser;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JTextField;


/**
 * A simple example of GUI event handling in a Java application.
 * 
 * This can be run as a main program or as an applet.
 * 
 * @author zeil
 *
 */


public class ThreadArt extends JApplet {

    private boolean startedInAnApplet;

    // The Model
    private Color[] colors;
    private int stepSize = 5;

    private int cycleLength;
    private int cycleCounter;
    private boolean running;

    // The View & Controls
    private JFrame window;
    private JPanel canvas;
    private JButton colorChooser1;
    private JButton colorChooser2;
    private JTextField stepSizeIn;
    
    private Animator colorChanger;


    private class ColorChooser implements ActionListener {
        private JButton button;
        private int colorNum;

        public ColorChooser (JButton button, int colorNum) {
            this.button = button;
            this.colorNum = colorNum;
        }

        @Override
        public void actionPerformed(ActionEvent arg0) {
            Color chosen = JColorChooser.showDialog(window, "Choose a color", colors[colorNum]);
            if (chosen != null) {
                colors[colorNum] = chosen;
                setColor (button, chosen);
                canvas.repaint();
            }
        }
    };


    /**
     * Thread that slowly changes the color of the drawing 
     *
     */
    public class Animator extends Thread {  ➊

        public Animator()
        {
        }
        
        
        public void run()                  ➋
        {
            running = true;
            while (running) {              ➌
                try {
                    sleep(50);             ➍
                } catch (InterruptedException e) {
                  break;
                }
                cycleCounter = (cycleCounter + 1) % cycleLength; ➎
                canvas.repaint();
            }
        }
        
    }




    public ThreadArt()
    {
        startedInAnApplet = false;
        window = null;
        colors = new Color[2];
        colors[0] = Color.red;
        colors[1] = Color.blue;
        cycleLength = 100;
        cycleCounter = 0;
        running = false;
    }


    public static void main (String[] args)
    {
        ThreadArt instance = new ThreadArt();
        instance.createAndShowGUI();  
    }

    public void createAndShowGUI() {
        window = new JFrame();
        // set up the components
        window.getContentPane().setLayout (new BorderLayout());

        canvas = new JPanel () {
            public void paint (Graphics g) {
                super.paint(g);
                drawLines (g, getSize());
            }
        };
        canvas.setBackground(Color.white);
        window.getContentPane().add (canvas, BorderLayout.CENTER);
        canvas.setPreferredSize(new Dimension(400, 400));

        JPanel controls = new JPanel();

        colorChooser1 = new JButton("Color 1");
        controls.add (colorChooser1);
        setColor(colorChooser1, colors[0]);
        colorChooser1.addActionListener (new ColorChooser(colorChooser1, 0));

        colorChooser2 = new JButton("Color 2");
        controls.add (colorChooser2);
        setColor(colorChooser2, colors[1]);
        colorChooser2.addActionListener (new ColorChooser(colorChooser2, 1));

        stepSizeIn = new JTextField (""+stepSize, 5);
        controls.add (stepSizeIn);
        stepSizeIn.addActionListener (new ActionListener()
        {
            public void actionPerformed(ActionEvent e) {
                try {
                    Integer newSize = new Integer(stepSizeIn.getText());
                    stepSize = newSize.intValue();
                    canvas.repaint();
                } catch (Exception ex) {};
            }
        });

        window.getContentPane().add (controls, BorderLayout.SOUTH);

        window.setDefaultCloseOperation((startedInAnApplet) ? JFrame.DISPOSE_ON_CLOSE : JFrame.EXIT_ON_CLOSE);


        colorChanger = new Animator();   ➏
        colorChanger.start();

        window.pack();
        window.setVisible(true);
    }

    /**
     * Sets the background color of a button to the indicated color.
     * Changes the foreground to wither black or white, depending on
     * which will give more contrast agasint the new background.
     * 
     * @param button
     * @param color
     */
    private void setColor(JButton button, Color color) {
        button.setBackground(color);
        int brightness = color.getRed() + color.getGreen() + color.getBlue(); // max of 3*255
        if (brightness > 3*255/2) {
            // This is a fairly bright color. Use black lettering
            button.setForeground (Color.black);
        } else {
            // This is a fairly dark color. Use white lettering
            button.setForeground (Color.white);
        }
    }

    // Applet functions

    public void init() {
        startedInAnApplet = true;
    }

    public void start() {
        if (window == null)
            createAndShowGUI();
    }

    public void stop() {
    }

    public void destroy() {
    }




    int interpolate (int x, int y, int i, int steps)
    {
        return (i * x + (steps-i)*y) / steps;
    }


    Color interpolate(Color c1, Color c2, int i, int steps)
    {
        return new Color (interpolate(c1.getRed(), c2.getRed(), i, steps),
                interpolate(c1.getGreen(), c2.getGreen(), i, steps),
                interpolate(c1.getBlue(), c2.getBlue(), i, steps));
    }


    class Point {
        double x;
        double y;
    }

    Point ptOnCircle (int degrees, int radius, Point center) 
    {
        Point p = new Point();
        double theta = Math.toRadians((double)degrees);
        p.x = center.x + (double)radius * Math.cos(theta);
        p.y = center.y + (double)radius * Math.sin(theta);
        return p;
    }

    public void drawLines(Graphics g, Dimension d)
    {
        int dmin = (d.width < d.height) ? d.width : d.height;

        if (stepSize < 1)
            stepSize = 1;

        Point center = new Point();
        center.x = (double)d.width/2.0;
        center.y = (double)d.height/2.0;

        int k = Math.abs(cycleCounter - cycleLength/2);
        int theta = 60 * cycleCounter / cycleLength; 
        
        for (int i = 0; i < 60; ++i) {
            int radius = dmin/2;
            Point origin = ptOnCircle(6*i+theta, radius, center);
            int j = i + stepSize;
            while (j >= 60)
                j -= 60;
            while (i != j) {
                Point destination = ptOnCircle(6*j+theta, radius, center);
                Color c = interpolate(colors[0], colors[1], k, cycleLength/2);
                g.setColor(c);
                g.drawLine ((int)origin.x, (int)origin.y,
                        (int)destination.x, (int)destination.y);
                j += stepSize;
                while (j >= 60)
                    j -= 60;
            }
        }
    }


}

➊ Here we declare a Thread subclass
➋ … and override the run() function to manage our counter
➌ Loop almost forever
➍ The sleep function blocks the thread for at the indicated number of milliseconds
➎ This is what we came here to do.

The repaint() will cause a near-future redraw of the canvas, which will use our new value of cycleCounter
➏Once the GUI has been set up, we create the new thread object and start() it.

4. Interleaving & Synchronization

In what order do calculations take place?
How Does Order Affect Correctness?

Interleavings

Given 2 processes whose code looks like:

process 1	process 2
a;	x;
b;	y;
c;	z;

Possible Orderings

First statement must be either a or x.

If it’s a, then the 2nd statement executed must be either b or x
If it’s x, then the 2nd statement executed must be either a or y

Possible Orderings (cont.)

Expanding that:

First statement must be either a or x.

If it’s a, then the 2nd statement executed must be either b or x
- If it’s b, then the 3rd must be either c or x.
- If it’s x, then the 3rd must be either b or y.
If it’s x, then the 2nd statement executed must be either a or y

and so on

Possible Orderings

abcxyz, abxcyz, abxycz, abxyzc, axbcyz, axbycz, axbyzc, axybcz, axybzc, axyzbc, xabcyz, xabycz, xabyzc, xaybcz, xaybzc, xayzbc, xyabcz, xyabzc, xyazbc, xyzabc

Although there are many possibilities, b never precedes a, c never precedes a or b, etc.

Interleavings

An interleaving of two sequences s and t is any sequence u

formed from the events of s and t such that
- the events of s retain their position relative to one another and
- the events of t do likewise.

Describing Process Execution

Describe each process as a sequence of “atomic” actions.
- What constitutes “atomic” depends on hardware, compiler, etc.
Each interleave of the two processes is a possible execution order for the program as a whole.
- (assuming no synchronization takes place)

4.1 Safety and Parallel Programs

Shared Resoruces

Processes often compete for shared resources.

I/O devices
shared variables
files

In these cases, some interleaves are dangerous.

Safety

We say that a nondeterministic program is safe if all of the answers it provides for any input are acceptable.

Example

Processes with Shared Variable
process 1	process 2
`x = x + 1;`	`x = x + 1;`

Assuming that x starts with a value of –1 before the two processes are started, what will the value of x be after both have finished?

We could get: 1

Processes with Shared Variable
process 1	process 2
`x = x + 1;`	`x = x + 1;`

process 1 fetches x (–1)
process 1 adds 1 to x and stores it (x == 0)
process 2 fetches x (0)
process 2 adds 1 to x and stores it (x == 1)

We could get: 0

Processes with Shared Variable
process 1	process 2
`x = x + 1;`	`x = x + 1;`

process 1 fetches x (–1)
process 2 fetches x (–1)
process 1 adds 1 to (it’s copy of) x and stores it (x == 0)
process 2 adds 1 to (it’s copy of) x and stores it (x == 0)

Nondeterminism

Now these two alternatives might or might not be bad.
Some concurrent programs are specifically designed for a limited amount of nondeterminism:
- The property of having multiple possible answers for the same input
But there are other possibilities as well.

We could get: 65536

Processes with Shared Variable
process 1	process 2
`x = x + 1;`	`x = x + 1;`

process 1 fetches x (–1)
process 1 adds 1 to (it’s copy of) x
process 1 begins storing x. Stores the higher two bytes (x == 0000FFFFH)
process 2 fetches x (65535)
process 1 stores remaining 2 bytes of x (x == 0)
process 2 adds 1 to (it’s copy of) x and stores it (x== 65536)

65536 : Really?

Admittedly, this requires a different level of atomicity than we might have expected, but this is not at all impossible in a distributed system.
And it’s a pretty good bet that we would not consider this an acceptable output of these two processes.
- Which implies that this code is unsafe.

Simultaneous Access to Compound Data

More realistically, any kind of compound data structure is likely to be sensitive to simultaneous update by different processes.

More specifically,

Simultaneous read-only access is probably safe.
Simultaneous updates are likely to be a problem.
Simultaneous reading and updating are often a problem as well.

Example: Simultaneous Access to a Queue

Suppose that we have a queue of customer records,

implemented as a linked list
currently holding one record

Here we show the data and possible implementations of the functions to add to the end and remove from the front.

Verify for yourself that, if either function runs without interruption, this code would work
- No matter which of the two runs first.

Queues Interrupted

Now suppose that one process tries to simultaneously add a record to the end, and another one tries to remove a record from the front.

Assume that the remove() call gets the CPU first.

And then the add() gets to run for a little while …

and then control switches back to the remove() call, which runs to the end.

And finally the add() is allowed to complete.

Our queue is now badly mangled.

4.2 Synchronization

Synchronization is a restriction of the possible interleavings of a set of processes.

(to protect against interleaves that produce undesirable results.)

Synch & Shared Resources

Synchronization often takes the form of protecting a shared resource:

Processes with Shared Variable
process 1	process 2
`seize(x);`	`seize(x);`
`x = x + 1;`	`x = x + 1;`
`release(x);`	`release(x);`

Intended meaning of seize:
- once a seize of some resource has begun, no other process can begin to seize the same resource until it has been released.

Synchronization Narrows the Options

Processes with Shared Variable
process 1	process 2
`seize(x);`	`seize(x);`
`x = x + 1;`	`x = x + 1;`
`release(x);`	`release(x);`

Now only two possible traces:
- abcxyz and xyzabc

A Bullet, but not a Silver One

Synchronization takes different forms
It can relieve dangers of shared access
- Thereby promoting safety
But it introduces new problems
- New kinds of programming errors (e.g., forgetting to release a seized resource), and
- Liveness problems

5. Liveness Properties

Synchronization promotes safety: do we get a “correct” answer?
But it may adversely affect liveness: the rate of progress toward an answer

5.1 Deadlock

In deadlock, all processes are waiting on some shared resources, with none of them able to proceed.

Example: The Dining Philosophers

N philosophers sitting at a round table with N plates of spaghetti and N forks, one between each pair of philosophers.
- Each philosopher alternately thinks and eats.
- Each needs two forks to eat.
- They put down their forks when thinking.

Simulating the Philosphers

Represent each philosopher as an independent process:

 loop
    pick up left fork;
    pick up right fork;
    eat;
    release forks;
    think
 end loop;

If all the philosophers are holding the left fork, the system deadlocks.
- And the world has a few less philosophers in it!

Demo

Try running Sun’s Demo of the dining philosophers.

Avoiding Deadlock

One way to avoid deadlock is a policy of “ordered resource allocation”.
- Number all the forks.
- Require all the philosophers to pick up the lower-numbered of the two adjacent forks first.

In general…

This works very nicely in this specialized case, but there is no general technique for avoiding deadlock in arbitrary situations.

You usually have to rely on careful analysis by the program designers.

5.2 Livelock

A system is in livelock if at least one process is not waiting, but the system makes no progress.

 loop
    pick up left fork;
    seize right fork
       if available;
    if seized then
      eat;
      release forks;
    else
      release left fork;
    end if;
    think;
  end loop;

This can get into livelock as each philosopher gets locked into a cycle of:

 pick up left fork;
 release left fork;
 pick up left fork;
 release left fork;
 pick up left fork;
 release left fork;
    ⋮

5.3 Fairness

When multiple processes are waiting for a shared resource, and that resource becomes available, a scheduler must decide which of the waiting processes gets the resource.

A scheduler is unfair if it consistently ignores certain processes.

What’s Fair?

Precise definition of fairness is difficult. Some possibilities:

Any process that wants to run will be able to do so within a finite amount of time. (“finite-progress”)
All processes awaiting a now-available resource have an equal chance of getting it.
- Implies that it is possible for some processes to wait an arbitrarily long time.

Selfish Threads

Threads can contribute to unfairness by being “selfish”:

 class MyThread extends Thread {
   ⋮
   public void run()
   {
    while (true)
      ++counter;
   }
 }

Unless the run-time system preempts this thread, it will hog the CPU.

Yielding

 class MyThread extends Thread {
   ⋮
   public void run()
   {
    while (true) {
      ++counter;
      Thread.yield();
    }
   }
 }

Java allows threads to signal that they don’t mind losing the CPU by yielding to other threads.

Thread Priority

Java also allows programs to set relative priorities for different threads:

Thread cpuHog = new Thread() {
    ⋮
};
cpuHog.setPriority(Thread.MIN_PRIORITY);
cpuHog.start();

6. Safety

Concurrent programs are often non-deterministic.

A critical section in a process is a section of code that must be treated as an atomic action if the program is to be correct.
A concurrent program is safe if its processes are mutually excluded from being in two or more critical sections at the same time.
- A fairly broad statement. Sometimes we can be more discerning by identifying groups of critical sections that can interfere with one another.
  - E.g., ones that work on a common shared resource

6.1 Mutual Exclusion Mechanisms

Semaphores

A semaphore is an ADT with two atomic operations, seize and release, and a hidden integer value.

seize()

If the semaphore value is positive, it is decremented. The calling process continues.
If the semaphore value is zero or negative, the calling process is blocked and must wait until the value becomes positive.

release()

The semaphore value is incremented. The calling process continues.
- In practice, the scheduler often selects a process blocked on this semaphore to be allowed to immediately attempt to seize it.

General Semaphores

In its most general form, a semaphore can be initialized to any value k, thereby allowing up to k processes to seize it simultaneously.
- Usually, we use binary semaphores, which are initialized to 1.
Historically, the seize and release operations were originally called p and v.

Yep, those were semaphores

process 1	process 2
`seize(x);`	`seize(x);`
`x = x + 1;`	`x = x + 1;`
`release(x);`	`release(x);`

assuming that the semaphore is not “x” itself but is something associated with or labeled as “x”.

Dining Philosophers with Semaphores

 Semaphore fork[N];
 Philosopher(i):
 loop
   int first = min(i, i+1%N);
   int second = max(i, i+1%N);
   fork[first].seize();
   fork[second].seize();
   eat;
   fork[first].release();
   fork[second].release();
   think;
 end loop;

Recap

Semaphores are relatively low-level approach to synchronization

Although available in Java, should not be used often
Monitors and synchronized sections are more elegant

Monitors

Monitors

A monitor is an ADT in which only one process at a time can execute any of its member functions.

Thus all the ADT function bodies are treated as critical sections.

A Common Pattern

A program processes data from an input source that supplies data at a highly variable rate
- Sometimes slower than the program processes data
- Sometimes faster, often in bursts
The program should not “freeze up” when data is unavailable
- may have a GUI
The data source must be checked often enough that inputs are not lost/ignored

The Consumer-Producer Pattern

Producer adds data to a queue
Consumer removes data from a queue
Each runs as a separate thread

What could go wrong?

Synchronizing the Queue

We have seen earlier that queues are likely unsafe for simultaneous access.

We avoid these simultaneous update problems by making the queue synchronized.

Monitored Queue

 class MonitoredQueue {
    private <: ... :>
    public synchronized
       void enter(Object) { ... }
    public synchronized
      Object front() { ... }
    public synchronized
      void leave() { ... }
    public synchronized
      boolean empty() { ... }
    public
       int maxSize() { ... }
  }

No two threads are allowed to be simultaneously in synchronized member functions of the same object.

Java Monitors

Monitors are the preferred synchronization technique in Java, where they are created by marking functions as “synchronized”.

Each object of type MonitoredQueue is separately monitored.

      MonitoredQueue q1 = new MonitoredQueue();
      MonitoredQueue q2 = new MonitoredQueue();

If process P1 calls q1.enter(x);, then another process P2 attempting to obtain q1.front() will be blocked until P1’s call to enter has completed.
On the other hand, if process P1 calls q1.enter(x)
- Process P2 can simultaneously execute q1.maxSize();

Monitored Statement Blocks

The synchronized member function declaration is a special case of synchronized statement blocks

Suppose we had only a regular queue and were not willing/able to change its interface.

We could still achieve a safe solution by marking the critical statements in the code as synchronized:

syncblocks.java

 class Consumer extends Thread {
    private Queue q;
 
   public void run() {
      while (true) {
          synchronized (q) {
            Customer c = q.front();
            q.leave();
          };
          ... process customer ...
        }
      }
   }
 }

 class Producer extends Thread {
    private Queue q;
 
   public void run() {
      while (true) {
        Customer c = fetchData();
        synchronized (line1) {
           q.enter(c);
        }
     }
 }

Synchronization on Objects

When we mark statements as synchronized, we have to state explicitly what object we are synchronizing on.
- When we mark member functions as synchronized, we don’t need to do so because, implicitly, the object being synchronized on is this.
Important: To actually get synchronization, blocks of statements must synchronize on the same object.

7. Direct Control of Thread State

Waiting…

We often want to make a thread inactive until certain conditions are met.

Busy Wait

This is the bad way to do it:

public void run() {
   while (true) {
      if (conditionIsMet()) {
         doSomethingUseful();
      }
   }
}

This is a busy wait

We continuously burn CPU cycles testing the condition until it becomes true
- Makes machine very sluggish

Son of Busy Wait

Only marginally better:

public void run() {
   while (true) {
      if (conditionIsMet()) {
         doSomethingUseful();
      } else {
        sleep(100); // wait 0.1 seconds
      }
   }
}

still a busy wait
still a bad idea

Controlling the Process State

A better idea is to

block the thread and then
make it ready when the condition is met.
- (has to be done by some other thread)

wait()

If we have a synchronized lock on some object x, then x.wait() will

Put the thread on a “waiting for x queue”
Make the thread ineligible for any more CPU time (blocked)
Release the lock on x

notifyAll()

If we have a synchronized lock on some object x, then x.notifyAll() will

Make all threads on the “waiting for x queue” eligible for CPU time (ready)

Example: Adding a Pause Button to ThreadArt

ThreadArt2.java

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;
import java.awt.Graphics;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;

import javax.swing.JApplet;
import javax.swing.JButton;
import javax.swing.JColorChooser;
import javax.swing.JFrame;
import javax.swing.JPanel;
import javax.swing.JTextField;

/**
 * A simple example of GUI event handling in a Java application.
 * 
 * This can be run as a main program or as an applet.
 * 
 * @author zeil
 *
 */


public class ThreadArt2 extends JApplet {

    private boolean startedInAnApplet;

    // The Model
    private Color[] colors;
    private int stepSize = 5;

    private int cycleLength;
    private int cycleCounter;
    private boolean running;
    
    // The View & Controls
    private JFrame window;
    private JPanel canvas;
    private JButton colorChooser1;
    private JButton colorChooser2;
    private JButton pause;
    private JTextField stepSizeIn;
    
    private Animator colorChanger;


    private class ColorChooser implements ActionListener {
        private JButton button;
        private int colorNum;

        public ColorChooser (JButton button, int colorNum) {
            this.button = button;
            this.colorNum = colorNum;
        }

        @Override
        public void actionPerformed(ActionEvent arg0) {
            Color chosen = JColorChooser.showDialog(window, "Choose a color", colors[colorNum]);
            if (chosen != null) {
                colors[colorNum] = chosen;
                setColor (button, chosen);
                canvas.repaint();
            }
        }
    };


    /**
     * Thread that slowly changes the color of the drawing 
     *
     */
    public class Animator extends Thread {

        public Animator()
        {
        }
        
        
        public void run()
        {
            running = true;
            while (true) {
                try {
                    sleep(50);
                } catch (InterruptedException e) {
                  break;
                }
                synchronized (this) {
                    while (!running) {
                        try {
                            wait();                           ➌
                        } catch (InterruptedException e) {
                            return;
                        }
                    }
                }
                cycleCounter = (cycleCounter + 1) % cycleLength;
                canvas.repaint();
            }
        }
        
    }




    public ThreadArt2()
    {
        startedInAnApplet = false;
        window = null;
        colors = new Color[2];
        colors[0] = Color.red;
        colors[1] = Color.blue;
        cycleLength = 100;
        cycleCounter = 0;
        running = true;
    }


    public static void main (String[] args)
    {
        ThreadArt2 instance = new ThreadArt2();
        instance.createAndShowGUI();  
    }

    public void createAndShowGUI() {
        window = new JFrame();
        // set up the components
        window.getContentPane().setLayout (new BorderLayout());

        canvas = new JPanel () {
            public void paint (Graphics g) {
                super.paint(g);
                drawLines (g, getSize());
            }
        };
        canvas.setBackground(Color.white);
        window.getContentPane().add (canvas, BorderLayout.CENTER);
        canvas.setPreferredSize(new Dimension(400, 400));

        JPanel controls = new JPanel();

        colorChooser1 = new JButton("Color 1");
        controls.add (colorChooser1);
        setColor(colorChooser1, colors[0]);
        colorChooser1.addActionListener (new ColorChooser(colorChooser1, 0));

        colorChooser2 = new JButton("Color 2");
        controls.add (colorChooser2);
        setColor(colorChooser2, colors[1]);
        colorChooser2.addActionListener (new ColorChooser(colorChooser2, 1));

        stepSizeIn = new JTextField (""+stepSize, 5);
        controls.add (stepSizeIn);
        stepSizeIn.addActionListener (new ActionListener()
        {
            public void actionPerformed(ActionEvent e) {
                try {
                    Integer newSize = new Integer(stepSizeIn.getText());
                    stepSize = newSize.intValue();
                    canvas.repaint();
                } catch (Exception ex) {};
            }
        });

        pause = new JButton("Pause");             ➊
        controls.add (pause);
        pause.addActionListener(new ActionListener() {
            
            @Override
            public void actionPerformed(ActionEvent e) {
                if (running) {
                    running = false;              ➋
                    pause.setText("Resume");
                    pause.repaint();
                } else {
                    synchronized (colorChanger) {
                        running = true;           ➍
                        pause.setText("Pause");
                        pause.repaint();
                        colorChanger.notifyAll();
                    }
                }
            }
        });
        
        
        window.getContentPane().add (controls, BorderLayout.SOUTH);

        window.setDefaultCloseOperation((startedInAnApplet) ? JFrame.DISPOSE_ON_CLOSE : JFrame.EXIT_ON_CLOSE);


        colorChanger = new Animator();
        colorChanger.start();

        window.pack();
        window.setVisible(true);
    }

    /**
     * Sets the background color of a button to the indicated color.
     * Changes the foreground to wither black or white, depending on
     * which will give more contrast agasint the new background.
     * 
     * @param button
     * @param color
     */
    private void setColor(JButton button, Color color) {
        button.setBackground(color);
        int brightness = color.getRed() + color.getGreen() + color.getBlue(); // max of 3*255
        if (brightness > 3*255/2) {
            // This is a fairly bright color. Use black lettering
            button.setForeground (Color.black);
        } else {
            // This is a fairly dark color. Use white lettering
            button.setForeground (Color.white);
        }
    }

    // Applet functions

    public void init() {
        startedInAnApplet = true;
    }

    public void start() {
        if (window == null)
            createAndShowGUI();
    }

    public void stop() {
    }

    public void destroy() {
    }




    int interpolate (int x, int y, int i, int steps)
    {
        return (i * x + (steps-i)*y) / steps;
    }


    Color interpolate(Color c1, Color c2, int i, int steps)
    {
        return new Color (interpolate(c1.getRed(), c2.getRed(), i, steps),
                interpolate(c1.getGreen(), c2.getGreen(), i, steps),
                interpolate(c1.getBlue(), c2.getBlue(), i, steps));
    }


    class Point {
        double x;
        double y;
    }

    Point ptOnCircle (int degrees, int radius, Point center) 
    {
        Point p = new Point();
        double theta = Math.toRadians((double)degrees);
        p.x = center.x + (double)radius * Math.cos(theta);
        p.y = center.y + (double)radius * Math.sin(theta);
        return p;
    }

    public void drawLines(Graphics g, Dimension d)
    {
        int dmin = (d.width < d.height) ? d.width : d.height;

        if (stepSize < 1)
            stepSize = 1;

        Point center = new Point();
        center.x = (double)d.width/2.0;
        center.y = (double)d.height/2.0;

        int k = Math.abs(cycleCounter - cycleLength/2);
        int theta = 60 * cycleCounter / cycleLength; 
        
        for (int i = 0; i < 60; ++i) {
            int radius = dmin/2; //interpolate(dmin/4, dmin/2, k, cycleLength/2);
            Point origin = ptOnCircle(6*i+theta, radius, center);
            int j = i + stepSize;
            while (j >= 60)
                j -= 60;
            while (i != j) {
                Point destination = ptOnCircle(6*j+theta, radius, center);
                Color c = interpolate(colors[0], colors[1], k, cycleLength/2);
                g.setColor(c);
                g.drawLine ((int)origin.x, (int)origin.y,
                        (int)destination.x, (int)destination.y);
                j += stepSize;
                while (j >= 60)
                    j -= 60;
            }
        }
    }


}

➊ A new button
➋ that sets running to false
➌ which causes the animation thread to block itself, putting it a queue associated with the object colorChanger

How does the thread ever get awakened?
➍ Next time we click the button, we set running to true and notify all threads (there should only be one) waiting on colorChanger to make themselves ready.

8. Summing Up

Making Things Parallel

Divide the tasks into appropriate threads
Examine the threads for shared variables (and other resources)
- Consider whether simultaneous access to any of these shared resources could be unsafe.
Add synchronization to prevent unsafe simultaneous access
Analyze the synchronization for possible liveness problems