Pushdown Automata

CS390, Spring 2024

Last modified: Mar 9, 2023
Contents:

Abstract

Pushdown automata (PDAs) can be thought of as combining an NFA “control-unit” with a “memory” in the form of an infinite stack. PDAs are more powerful than FAs, being able to recognize languages that FAs cannot. In fact, the set of languages that can be recognized by PDAs are the context-free languages of the previous module.

1 The Automaton

1.1 Nondeterministic PDAs

Suppose we couple an NFA (with $\epsilon$ transitions) with a stack:

The moves of this automaton are controlled at each step by

The last is the big change in the way that state transitions are determined.

Like a Mealy machine, we will allow our NFA to produce output at each transition. These outputs are written to the stack after popping the top symbol:

So on each move, we

  1. Change state, and
  2. Rewrite the top of the stack.
 

Definition: A Pushdown Automaton (PDA) is given as $(Q, \Sigma, \Gamma, \delta, q_0, Z, F)$ where

  1. Q is a finite set of states.
  2. $\Sigma$ is the alphabet of input symbols
  3. $\Gamma$ is an alphabet of stack symbols
  4. $\delta$ is a transition function mapping $Q \times \Sigma \times \Gamma \rightarrow Q \times \Gamma^*$
  1. $q_0 \in Q$ is the start state
  2. $Z \in \Gamma$ is the starting symbol on the stack
  3. $F \subseteq Q$ is a set of final or accepting states.

Compare this to the formal definition of a finite state automaton. What jumps out at you?

Example 1: A PDA for $0^n1^n$

One thing worth pointing out is that the output of $\delta$ is a set. Because the FA controller for a PDA is allowed to be non-deterministic, it is quite possible for the transition on a new (input, stack symbol) will actually be to multiple states. We’ll take more advantage of that in later examples.

1.2 Transition Diagrams

We can show PDAs this in a transition diagram similar to the ones we used for FAs.

Instead of labeling each transition with just one symbol (the input), we have to label it with three components $a, X / \alpha$ where

That is, the arc label tells what input is used, and also gives the old and new tops of the stack.

Example 2: A Transition Diagram for $0^n1^n$

 

The diagram here illustrates those conventions.

When a stack symbol is matched by a transition rule, it is also popped from the stack. Sometimes we do want to “consume” that stack symbol. Other times, we may want to leave it in place, or even push something else on top of it.

  • In the rule $\epsilon,Z/Z$, we match the Z on the stack, popping it, but the /Z means that we replace it by another Z. The next effect is that we simply left the Z where it was on the stack.
  • In the rule $0,Z/XZ$, if we see a 0 in the input and if see a Z on the top of the stack, we pop the matched Z from the top of the stack, and then replace it by XZ. The characters are pushed in reverse order, so first the Z is pushed back onto the bottom of the stack, and then the X is pushed on top of it.
  • In the rule $1,X/\epsilon$, if we see a 1 in the input and if we see an ‘X’ on the top of the stack, we pop the matched X and replace it by $\epsilon$, the empty string. In other words, we don’t replace the popped symbol by anything at all.

You should be able to see that $\epsilon$ can play multiple roles in transition labels,.

This first example of a PDA made limited use of non-determinism. Our next example makes much heavier use of it.

Example 3: A PDA for $ww^R$

One of the “obvious” things we would expect a machine with a stack to be good at would be reversing strings or, because our automata are about recognizing strings in a language, recognizing when a string has been reversed.

Let’s construct a PDA for the language $ww^R$ where $w$ is any string in $\{0, 1\}^*$ and $w^R$ is the reverse of that string.

 

Here is the transition diagram for that PDA.

Look at the transitions from $q_0$ to itself. If the input (first element in the triple label) is 0, then no matter what symbol is on top of the stack (the $\epsilon$ in the second position), we push a 0 (the third position) onto the stack. Similarly, if the input is 1, then no matter what symbol is on top of the stack, we push a 1 onto the stack. So basically, we are using $q_0$ to store up the characters of $w$ on the stack.

  • You won’t see the $\epsilon$ used for the stack symbol in your textbook.

    It stands for all of the productions that we would obtain by replacing $\epsilon$ by each of the possible stack symbols, and modifying the “push string” in the third position to push that symbol back on.

    For example, the transition from state $q_0$ to $q_0$ on input “0” is really a shorthand for:

    \[ \begin{align} \delta(q_0,0,Z) &= \{ (q_0, 0Z) \} \\ \delta(q_0,0,0) &= \{ (q_0, 00) \} \\ \delta(q_0,0,1) &= \{ (q_0, 01) \} \\ \end{align} \]

     

    Here, I have written the same set of transitions without any $\epsilon$s in the stack element position.

Look at the transitions from $q_1$ to itself. If the input is 0 and the top symbol on the stack is 0, we “return” to $q_1$ after replacing that 0 on the top of the stack by the empty string – in other words we pop the matched 0 from the stack. Similarly, if the input is 1 and the top symbol on the stack is 1, we “return” to $q_1$ after popping the matched symbol from the stack. (If the input is zero and the symbol is 1, or vice versa, there is no transition from this state, not even back to $q_1$.) So, the state $q_1$ can be seen as doing the actual matching of the symbols in $w^R$.

The “trick” in the design of this PDA is knowing when to stop storing and to start checking for the reversed string. For example, ‘0110’ need to be accepted because ‘01’, reversed, is ‘10’. But ‘0110110110’ should also be accepted, because ‘01101’ reversed is ‘10110’. But if we go from $q_0$ to $q_1$ after seeing ‘01’, and then start popping characters, and it turns out that the string is going to continue as ‘0110110110’ instead of just as ‘0110’, we will have lost the chance to record the characters we need to detect the longer of those two strings.

The answer to this problem is that we don’t actually try to guess when to start reversing the string. Instead, we exploit non-determinism to simultaneously reverse all prefixes of the input. So, if the input is actually going to be ‘0110’, we will actually simultaneously explore the possibility that the string is going to be ’’, ‘00’, ‘0110’, ‘011110’, and ‘01100110’.

We accomplish this by adding the transition from $q_0$ to $q_1$. That transition triggers on every state transition (including the start of the automaton). So, every time we push a character onto the stack in $q_0$, we simultaneously return to $q_0$ to push some more input and go to $q_1$ to start detecting reversed input.

That leaves the question of knowing when we are done. Whenever we are in $q_1$, pop a matched symbol, and wind up looking at the start symbol $Z$ on the stack, we have detected a $ww^R$ string, and immediately go to a final state r. (Of course, we can’t discount the possibility that more input remains, but if we encounter more input, there is no legal transition from r, so we would leave r on that input.

To complete the example, here is the transition function:

  • $\delta(q_0, 0, \epsilon) = \{(q_0, 0)\}$
  • $\delta(q_0, 1, \epsilon) = \{(q_0, 1)\}$
  • $\delta(q_0, \epsilon, \epsilon) = \{(q_1, \epsilon)\}$
  • $\delta(q_1, 0, 0) = \{(q_1, \epsilon)\}$

  • $\delta(q_1, 1, 1) = \{(q_1, \epsilon)\}$
  • $\delta(q_1, \epsilon, Z) = \{(q_2, \epsilon)\}$

1.3 Running a PDA

As noted earlier, because the FA controller for a PDA is allowed to be non-deterministic, it is quite possible for the transition on a new (input, stack symbol) will actually be to multiple states. Even messier, the multiple parallel transitions may manipulate the stack in different ways, leading to different stack contents for each of those parallel states.

An instantaneous description of a PDA state (“configuration”) is a description of the PDA as a triple: $(q,w,\gamma)$, where

 

For example, if we wanted to run this PDA on the input “0011”, we would describe the starting state as $(q,“0011”,“Z”)$. After processing the first character of input, we would describe the resulting machine as $(q,“011”, “0Z”)$.

 

We will use the “turnstile” symbol $\vdash$ to denote a “move” or transition of a PDA. So, for example

\[ (q,“0011”,“Z”) \vdash (q,“011”, “0Z”) \]

We could indicate a series of transitions

\[ \begin{align} (q,“0011”,“Z”) & \vdash (q,“011”, “0Z”) \\ & \vdash (q,“11”, “00Z”) \\ & \vdash (p,“1”, “0Z”) \\ & \vdash (p,“1”, “Z”) \\ & \vdash (r, \epsilon, \epsilon) \\ \end{align} \]

We use $\vdash^*$ to indicate a series of zero or more transitions. Examples would include

\[ \begin{align} (q,“0011”,“Z”) & \vdash^* (q,“0011”,“Z”) \\ (q,“0011”,“Z”) & \vdash^* (p,“1”, “Z”) \\ (q,“11”, “00Z”) & \vdash^* (r, \epsilon, \epsilon) \\ \end{align} \]

The $\vdash^*$ provides a useful way to discuss what can and cannot be recognized by a PDA.

It doesn’t really help a lot with the fundamental problem of tracking the multitude of states that can arise simultaneously during a derivation. See Example 6.4 in your text for an attempt to do this. You can see that the solution attempted there is, at best, a textual hack.

This is a place where a program like Automat comes in handy.

1.3.1 Accepting an Input String

Historically, there have been two different ways of indicating that a PDA was to accept an input string:

  1. Accept a string that leaves the FA controller in a final/accepting state.

    This is the technique I have used in the examples above and that we will continue to use throughout this course.

  2. Ignore the FA state and accept any string that leaves the stack empty (including having popped the starting stack symbol from the stack).

It’s fairly easy to show that these two methods are equivalent – you can easily transform a PDA that uses one of the methods into a PDA that would accept the exact same language using the other method. The proof is in your textbook. (If you have not encountered it yet, you might want to think about how you would do these transformations before reading the textbook’s approach.)

1.4 Deterministic PDAs

A deterministic PDA is a PDA that is never simultaneously in two or more states, no matter what input it is given.

Both of our example PDAs in this lesson have been non-deterministic.

Deterministic PDAs are of great practical importance, as discussed later, but not of particularly great theoretical importance.

2 Equivalence of PDAs and CFGs

We can prove that PDAs accept exactly the context free languages by showing that we can convert any CFG into a PDA and any PDA into a CFG,

2.1 Every CFG can be Converted to a PDA

Suppose we have a CFG. We have previously described the process of generating a string by left-most derivation as

  1. Begin with a string consisting only of the start symbol.
  2. Pick the leftmost variable occurrence in the current string.
  3. Pick any production that has that variable on the left of the $\rightarrow$.
  4. Replace the chosen occurrence of that variable by the right-hand side of the chosen production.
  5. Repeat steps 2-4 until the string contains only terminals.

We can turn this into a parsing algorithm to recognize if a string is in the language of that grammar by making just a few changes:

  1. Begin with an input string $s$ and a derived string $\alpha$ consisting only of the start symbol.
  2. If $\alpha$ begins with one or more terminals, then check to see if the input string $s$ begins with the same terminals. If not, stop the algorithm. If they match, remove those matching terminals from the start of both strings. Go to step 6.
  3. Let $A$ denote the variable that begins $\alpha$.
  4. Pick any production that has $A$ on the left of the $\rightarrow$.
  5. Remove $A$ from the front of $\alpha$ and prepend the right-hand side of the chosen production.
  6. If both $s$ and $\alpha$ are empty, we have successfully parsed the string. If only $\alpha$ is empty, the original input string is not in the language. If $\alpha$ is nto empty, go back to step 2.

The “catch” to this procedure is step 4, where we pick a production. There might be many productions that have $A$ on the left, and if we pick the wrong one our parse will fail to match strings that it should.

But non-determinism means that we don’t really need to make a choice. We can, in parallel, choose every production with $A$ on the left, and try to carry out the remainder of the parsing in parallel.

With that in mind, we can make an intuitive argument (the formal proof is in the text) that we can map these steps of the above procedure onto a PDA:

  1. Begin with an input string $s$ and a derived string $\alpha$ consisting only of the start symbol.

    • We will store $\alpha$ on the PDA stack. In this step, we use the start symbol of the grammar as the start symbol on the stack.

  2. If $\alpha$ begins with one or more terminals, then check to see if the input string $s$ begins with the same terminals. If not, stop the algorithm. If they match, remove those matching terminals from the start of both strings. Go to step 6.

    • This suggest a set of transitions $\delta(p,a,a) = \{(p,\epsilon)\}$, $\forall a \in \Sigma$.

  3. Let $A$ denote the variable that begins $\alpha$.

  4. Pick any production that has $A$ on the left of the $\rightarrow$.
    • Again, we’ll pick all of them and let the non-determinism run rampant.

  5. Remove $A$ from the front of $\alpha$ and prepend the right-hand side of the chosen production.

    • Remember that a PDA transition can write any number of symbols to the stack. So, for a CFG production $A \rightarrow X Y Z\ldots$, we would get a PDA transition of the form $\delta(q,\epsilon,A) = \{(r, X Y Z\ldots) \}$. (Remember, we push the strings in reverse order of the way we write them.)

  6. If both $s$ and $\alpha$ are empty, we have successfully parsed the string. If only $\alpha$ is empty, the original input string is not in the language. If $\alpha$ is not empty, go back to step 2.

Your textbook connects all of these elements together.

2.2 Every PDA can be Converted to a CFG

This is, IMO, less obvious. We will create a grammar in which our variable names have the form $[pXq]$ where $p$ and $q$ are states in the PDA and $X$ is a stack symbol.

Now, in a grammar, each variable represents a smaller language in its right. The variable $[pXq]$ is supposed to denote the language of all strings $w$ for which

\[ (p,wz,X\alpha) \vdash^* (q,z,\alpha) \]

i.e., a set of strings that could be consumed when taking us from state $p$ to state $q$, during which time the single symbol $X$ would be popped from the stack.

If we are able to construct a grammar for which those variables actually fulfill that promise, then $[q_0Z_0q_i]$ (for all states $q_i$) would be the language of strings that take us from the starting state of the PDA to any other state, while popping the initial stack symbol $Z_0$ (which means that we have emptied the stack). So that would be the language recognized by a PDA that accepts upon emptying its stack.

Example 4: Constructing a Grammar for $ww^R$

 

Your text gives the construction procedure and proves it in Theorem 6.14. I won’t type all that out here, but to help make it clear why this works, let’s apply the construction procedure to this PDA.

However, I’m going to do something a bit odd. Before we construct the grammar, I’m going to do a sample derivation using it. We can do that because of the definition of the $[qXp]$ variable names.

We’re going to derive “0110”.

Looking at the PDA, it empties its stack (removing the starting stack element $Z$) when making the transition from $p$ to $r$. So we need to start in $q$, end up in $r$, and pop $Z$ on the way. The set of strings that would do that is, by definition, $[qZr]$. So we will want a derivation to start:

\[ \begin{align} S & \Rightarrow [qZr] \\ \end{align} \]

So we are predicting that the grammar we generate will include a production $S \rightarrow [qZr]$.

Now, we expect the first 0 to be "processed by state q. The PDA will push a 0 onto the stack as it consumes that first 0 in the input. That means that, once we have gone past that 0 in the input, we will need to find, in the remaining input, a string that pops that 0 from the stack, leaving us in state $p$, then a string that pops the $Z$ while taking us to state $r$:

\[ \begin{align} S & \Rightarrow [qZr] \\ & \Rightarrow 0 [q0p][pZr] \\ \end{align} \]

So we are predicting that the grammar we generate will include a production $[qZr] \rightarrow 0 [q0p] [pZr]$.

Next up in the input is the first ‘1’. Being intelligent beings who can actually plan ahead, we know that we want this input to be processed in state $q$, pushing a ‘1’ onto the stack, and then immediately take the $\epsilon$-transition to state $p$ so we can start matching and popping the second half of the input.

\[ \begin{align} S & \Rightarrow [qZr] \\ & \Rightarrow 0 [q0p][pZr] \\ & \Rightarrow 0 1 [p1p][p0p][pZr] \\ \end{align} \]

So we will be looking for a string that takes us from state $p$ to state $p$ while popping a 1, then from $p$ to $p$ while popping a $0$, then from $p$ to $r$ popping a $Z$.

So we are predicting that the grammar we generate will include a production $[q0p] \rightarrow 1 [q1p] [p0p]$.

Looking at the derivation, and comparing to the PDA, you can start to see what happens with this style of grammar construction.

The variables in the leftmost derivation are encoding the contents that the PDA stack would have once it has recognized all of the terminal symbols to the left of the first variable.

That’s the key insight that motivates the use of this construction to provide that PDAs can be converted to CFGs.

Continuing our derivation, we can fulfill the goal of $[p1p]$ very easily, by simply deriving a ‘1’.

\[ \begin{align} S & \Rightarrow [qZr] \\ & \Rightarrow 0 [q0p][pZr] \\ & \Rightarrow 0 1 [p1p][p0p][pZr] \\ & \Rightarrow 0 1 1 [p0p][pZr] \\ \end{align} \]

We are predicting a production $[p1p] \rightarrow 1$.

We can similarly fulfill the goal of $[p0p]$ by deriving a ‘0’.

\[ \begin{align} S & \Rightarrow [qZr] \\ & \Rightarrow 0 [q0p][pZr] \\ & \Rightarrow 0 1 [p1p][p0p][pZr] \\ & \Rightarrow 0 1 1 [p0p][pZr] \\ & \Rightarrow 0 1 1 0 [pZr] \\ \end{align} \]

We are predicting a production $[p0p] \rightarrow 0$.

Finally, we can fulfill the goal of $[pZr]$ by deriving an empty string..

\[ \begin{align} S & \Rightarrow [qZr] \\ & \Rightarrow 0 [q0p][pZr] \\ & \Rightarrow 0 1 [p1p][p0p][pZr] \\ & \Rightarrow 0 1 1 [p0p][pZr] \\ & \Rightarrow 0 1 1 0 [pZr] \\ & \Rightarrow 0 1 1 0 \\ \end{align} \]

Our derivation is done. We have predicted that we will have the following productions in the grammar:

\[ \begin{align} S & \rightarrow [qZr] \\ [qZr] & \rightarrow 0 [q0p] [pZr] \\ [q0p] & \rightarrow 1 [q1p] [p0p] \\ [p1p] & \rightarrow 1 \\ [p0p] & \rightarrow 0 \\ [pZr] & \rightarrow \epsilon \\ \end{align} \]

That’s no to say that these will be the only productions in our grammar. They are simply the ones we expect to use when deriving “0110”. Other strings in the language may require other productions.

OK, now that we have an idea what we are looking for, let’s construct the grammar.

 

As noted earlier, the use of $\lambda$ in the stack element position to denote “don’t care” is not used in the textbook, and this construction will not work with that shortcut. So we will work from this expanded version of the PDA.

a) For all states $p$, $G$ has the production $S \rightarrow [qZp]$.

So we start with the productions

\[ \begin{align} S &\rightarrow [qZq] \\ S &\rightarrow [qZp] \\ S &\rightarrow [qZr] \\ \end{align} \]

Now, in fact, we know that there is no possible way to pop $Z$ from the stack and then wind up in states $p$ or $q$, so we know that the languages $[qZq]$ and $[qZp]$ are empty. We could actually drop those productions without affecting the language accepted by the CFG we are going to generate.

b) For each transition $\delta(q,a,X) = \{ \ldots, (r, Y_1Y_2\ldots Y_k) \}$, and for all lists of states $r_1, r_2, \ldots, r_k$, G has a production $[qXr_k] \rightarrow a[rY_1r_1][r_1Y_1r_2]\ldots[r_{k-1}Y_kr_k]$.

So let’s look at our transitions, one at a time.

$\delta(q,1,0) = (q,10)$
There are two symbols being pushed onto the stack, so $k=2$. That means that we need to consider all lists of states of length 2: $[q, q]$, $[q, p]$, $[q, r]$, $[p, q]$, $[p, p]$, $[p, r]$, $[r, q]$, $[r, p]$, $[r, r]$,

So we get new productions

list of states new production
$q, q$ $[q0q] \rightarrow 1 [q1q] [q0q]$
$q, p$ $[q0p] \rightarrow 1 [q1q] [q0p]$
$q, r$ $[q0r] \rightarrow 1 [q1q] [q0r]$
$p, q$ $[q0q] \rightarrow 1 [q1p] [p0q]$
$p, p$ $[q0p] \rightarrow 1 [q1p] [p0p]$
$p, r$ $[q0r] \rightarrow 1 [q1p] [p0r]$
$r, q$ $[q0q] \rightarrow 1 [q1r] [r0q]$
$r, p$ $[q0p] \rightarrow 1 [q1r] [r0p]$
$r, r$ $[q0r] \rightarrow 1 [q1r] [r0r]$
$\delta(q,1,1) = (q,11)$
There are two symbols being pushed onto the stack, so $k=2$.

So we get new productions

list of states new production
$q, q$ $[q1q] \rightarrow 1 [q1q] [q1q]$
$q, p$ $[q1p] \rightarrow 1 [q1q] [q1p]$
$q, r$ $[q1r] \rightarrow 1 [q1q] [q1r]$
$p, q$ $[q1q] \rightarrow 1 [q1p] [p1q]$
$p, p$ $[q1p] \rightarrow 1 [q1p] [p1p]$
$p, r$ $[q1r] \rightarrow 1 [q1p] [p1r]$
$r, q$ $[q1q] \rightarrow 1 [q1r] [r1q]$
$r, p$ $[q1p] \rightarrow 1 [q1r] [r1p]$
$r, r$ $[q1r] \rightarrow 1 [q1r] [r1r]$
$\delta(q,1,Z) = (q,1Z)$
There are two symbols being pushed onto the stack, so $k=2$.

So we get new productions

list of states new production
$q, q$ $[qZq] \rightarrow 1 [q1q] [qZq]$
$q, p$ $[qZp] \rightarrow 1 [q1q] [qZp]$
$q, r$ $[qZr] \rightarrow 1 [q1q] [qZr]$
$p, q$ $[qZq] \rightarrow 1 [q1p] [pZq]$
$p, p$ $[qZp] \rightarrow 1 [q1p] [pZp]$
$p, r$ $[qZr] \rightarrow 1 [q1p] [pZr]$
$r, q$ $[qZq] \rightarrow 1 [q1r] [rZq]$
$r, p$ $[qZp] \rightarrow 1 [q1r] [rZp]$
$r, r$ $[qZr] \rightarrow 1 [q1r] [rZr]$
$\delta(q,0,0) = (q,00)$
There are two symbols being pushed onto the stack, so $k=2$.

So we get new productions

list of states new production
$q, q$ $[q0q] \rightarrow 0 [q0q] [q0q]$
$q, p$ $[q0p] \rightarrow 0 [q0q] [q0p]$
$q, r$ $[q0r] \rightarrow 0 [q0q] [q0r]$
$p, q$ $[q0q] \rightarrow 0 [q0p] [p0q]$
$p, p$ $[q0p] \rightarrow 0 [q0p] [p0p]$
$p, r$ $[q0r] \rightarrow 0 [q0p] [p0r]$
$r, q$ $[q0q] \rightarrow 0 [q0r] [r0q]$
$r, p$ $[q0p] \rightarrow 0 [q0r] [r0p]$
$r, r$ $[q0r] \rightarrow 0 [q0r] [r0r]$
$\delta(q,0,1) = (q,01)$
There are two symbols being pushed onto the stack, so $k=2$.

So we get new productions

list of states new production
$q, q$ $[q1q] \rightarrow 0 [q0q] [q1q]$
$q, p$ $[q1p] \rightarrow 0 [q0q] [q1p]$
$q, r$ $[q1r] \rightarrow 0 [q0q] [q1r]$
$p, q$ $[q1q] \rightarrow 0 [q0p] [p1q]$
$p, p$ $[q1p] \rightarrow 0 [q0p] [p1p]$
$p, r$ $[q1r] \rightarrow 0 [q0p] [p1r]$
$r, q$ $[q1q] \rightarrow 0 [q0r] [r1q]$
$r, p$ $[q1p] \rightarrow 0 [q0r] [r1p]$
$r, r$ $[q1r] \rightarrow 0 [q0r] [r1r]$
$\delta(q,0,Z) = (q,0Z)$
There are two symbols being pushed onto the stack, so $k=2$.

So we get new productions

list of states new production
$q, q$ $[qZq] \rightarrow 0 [q0q] [qZq]$
$q, p$ $[qZp] \rightarrow 0 [q0q] [qZp]$
$q, r$ $[qZr] \rightarrow 0 [q0q] [qZr]$
$p, q$ $[qZq] \rightarrow 0 [q0p] [pZq]$
$p, p$ $[qZp] \rightarrow 0 [q0p] [pZp]$
$p, r$ $[qZr] \rightarrow 0 [q0p] [pZr]$
$r, q$ $[qZq] \rightarrow 0 [q0r] [rZq]$
$r, p$ $[qZp] \rightarrow 0 [q0r] [rZp]$
$r, r$ $[qZr] \rightarrow 0 [q0r] [rZr]$
$\delta(q,\epsilon,1) = (p,1)$
This time, $k=1$, and we have only three possible lists of states of length 1.
list of states new production
$q$ $[q1q] \rightarrow [p1q]$
$p$ $[q1p] \rightarrow [p1p]$
$r$ $[q1r] \rightarrow [p1r]$
$\delta(q,\epsilon,0) = (p,0)$
$k=1$
list of states new production
$q$ $[q0q] \rightarrow [p0q]$
$p$ $[q0p] \rightarrow [p0p]$
$r$ $[q0r] \rightarrow [p0r]$
$\delta(q,\epsilon,Z) = (p,Z)$
$k=1$
list of states new production
$q$ $[qZq] \rightarrow [pZq]$
$p$ $[qZp] \rightarrow [pZp]$
$r$ $[qZr] \rightarrow [pZr]$
$\delta(p,1,1) = (p,\epsilon)$
$k=0$

$[p1p] \rightarrow 1$

$\delta(p,0,0) = (p,\epsilon)$
$k=0$

$[p0p] \rightarrow 0$

$\delta(\epsilon,Z,\epsilon) = r,\epsilon)$
$k=0$

$[pZr] \rightarrow \epsilon$

And we’re done. The total grammar is

\[ \begin{align} S &\rightarrow [qZq] \\ S &\rightarrow [qZp] \\ \color{red}{S} &\color{red}{\rightarrow [qZr]} \\ [q0q] &\rightarrow 1 [q1q] [q0q] \\ [q0p] &\rightarrow 1 [q1q] [q0p] \\ [q0r] &\rightarrow 1 [q1q] [q0r] \\ [q0q] &\rightarrow 1 [q1p] [p0q] \\ \color{red}{[q0p]} &\color{red}{\rightarrow 1 [q1p] [p0p]} \\ [q0r] &\rightarrow 1 [q1p] [p0r] \\ [q0q] &\rightarrow 1 [q1r] [r0q] \\ [q0p] &\rightarrow 1 [q1r] [r0p] \\ [q0r] &\rightarrow 1 [q1r] [r0r] \\ [q1q] &\rightarrow 1 [q1q] [q1q] \\ [q1p] &\rightarrow 1 [q1q] [q1p] \\ [q1r] &\rightarrow 1 [q1q] [q1r] \\ [q1q] &\rightarrow 1 [q1p] [p1q] \\ [q1p] &\rightarrow 1 [q1p] [p1p] \\ [q1r] &\rightarrow 1 [q1p] [p1r] \\ [q1q] &\rightarrow 1 [q1r] [r1q] \\ [q1p] &\rightarrow 1 [q1r] [r1p] \\ [q1r] &\rightarrow 1 [q1r] [r1r] \\ [qZq] &\rightarrow 1 [q1q] [qZq] \\ [qZp] &\rightarrow 1 [q1q] [qZp] \\ [qZr] &\rightarrow 1 [q1q] [qZr] \\ [qZq] &\rightarrow 1 [q1p] [pZq] \\ [qZp] &\rightarrow 1 [q1p] [pZp] \\ [qZr] &\rightarrow 1 [q1p] [pZr] \\ [qZq] &\rightarrow 1 [q1r] [rZq] \\ [qZp] &\rightarrow 1 [q1r] [rZp] \\ [qZr] &\rightarrow 1 [q1r] [rZr] \\ [q0q] &\rightarrow 0 [q0q] [q0q] \\ [q0p] &\rightarrow 0 [q0q] [q0p] \\ [q0r] &\rightarrow 0 [q0q] [q0r] \\ [q0q] &\rightarrow 0 [q0p] [p0q] \\ [q0p] &\rightarrow 0 [q0p] [p0p] \\ [q0r] &\rightarrow 0 [q0p] [p0r] \\ [q0q] &\rightarrow 0 [q0r] [r0q] \\ [q0p] &\rightarrow 0 [q0r] [r0p] \\ [q0r] &\rightarrow 0 [q0r] [r0r] \\ [q1q] &\rightarrow 0 [q0q] [q1q] \\ [q1p] &\rightarrow 0 [q0q] [q1p] \\ [q1r] &\rightarrow 0 [q0q] [q1r] \\ [q1q] &\rightarrow 0 [q0p] [p1q] \\ [q1p] &\rightarrow 0 [q0p] [p1p] \\ [q1r] &\rightarrow 0 [q0p] [p1r] \\ [q1q] &\rightarrow 0 [q0r] [r1q] \\ [q1p] &\rightarrow 0 [q0r] [r1p] \\ [q1r] &\rightarrow 0 [q0r] [r1r] \\ [qZq] &\rightarrow 0 [q0q] [qZq] \\ [qZp] &\rightarrow 0 [q0q] [qZp] \\ [qZr] &\rightarrow 0 [q0q] [qZr] \\ [qZq] &\rightarrow 0 [q0p] [pZq] \\ [qZp] &\rightarrow 0 [q0p] [pZp] \\ \color{red}{[qZr]} &\color{red}{\rightarrow 0 [q0p] [pZr]} \\ [qZq] &\rightarrow 0 [q0r] [rZq] \\ [qZp] &\rightarrow 0 [q0r] [rZp] \\ [qZr] &\rightarrow 0 [q0r] [rZr] \\ [q1q] &\rightarrow [p1q] \\ [q1p] &\rightarrow [p1p] \\ [q1r] &\rightarrow [p1r] \\ [q0q] &\rightarrow [p0q] \\ [q0p] &\rightarrow [p0p] \\ [q0r] &\rightarrow [p0r] \\ [qZq] &\rightarrow [pZq] \\ [qZp] &\rightarrow [pZp] \\ [qZr] &\rightarrow [pZr] \\ \color{red}{[p1p]} &\color{red}{\rightarrow 1} \\ \color{red}{[p0p]} &\color{red}{\rightarrow 0} \\ \color{red}{[pZr]} &\color{red}{\rightarrow \epsilon} \\ \end{align} \]

Not exactly pretty. In fact, that’s pretty darn ugly. But it appears plausible, at least. I have marked in red the productions that we predicted would need to be included to derive the string “0110”.

We can be pretty certain that this grammar is

  • highly ambiguous,

    • Productions like $[q0q] \rightarrow [q0q]$ are kind of a dead give-away.
  • may contain some variables representing an empty language (in which case they and their productions could be dropped),

    • For example, the only transition that can empty the stack is the one from $p$ to $r$, and there is no way to get from $r$ back to either of the other states. So there are no strings in $[qZq]$ because there is no way to return back toe $q$ after having popped $Z$ from the stack.
  • or both.

Even though the grammar is huge, the derivations are fairly short (once you find them) because every step in this derivation adds an input character and/or removes a variable.

In fact, it is possible to give a grammar for $ww^R$, where $w \in \{0, 1\}^*$, in just 3 productions. Can you do so?

Click to reveal answer.

3 Applications: Common Parsing Algorithms

The development of CFGs helped turn compiler development from one of the major challenges of early computer science to a common activity that can be tackled by relatively small teams with a high degree of success.

Part of this is due to the more widely spread understanding of common parsing algorithms, so that people don’t have to work as hard to actually get the code written. Part of it is because programming language designers have learned how to design their languages to make them easy to parse.

Here is an example of a parsing algorithm, called “predictive parsing”, adapted from Principles of Compiler Design by Aho, Sethi, and Ullman, a classic text in the field.

Predictive parsing is driven by a parse table. This table, called M in the algorithm, is a two-dimensional table indexed by a non-terminal symbol (variable) in a CFG and an input symbol:

k = M[X,a]

means that if the top of the stack is X and we see input a coming up, then we want to derive using the $k^{th}$ production of the grammar.

parse (String w, Table M)
{
    ip = 0;
    stack = emptyStack;
    push S (starting symbol of grammar) onto stack
    repeat {
        X = stack.top();
        a = w[ip];
        if (X is a terminal) {
            if (X == a) {
                stack.pop();
                ++ip;
            } else {
                syntax-error();
            }
        } else { // X is a non-terminal
            if (M[X,a] >= 0) {
               pop X from the stack;
               R[] = symbols form right hand side of kth production
               push R[n-1], R[N-1], ..., R[0] onto stack;
            } else {
                syntax-error();
            }
        }
    } until stack is empty
}

Looking at this algorithm,

So this algorithm appears to have its roots in a PDA, but without non-determinism. This is why the comment has been made that most programming languages fall into the class of languages that can be recognized by deterministic PDAs.

So how do we actually get tables smart enough to always choose the proper production rather than exploring several in parallel? Well, that’s really a topic for a compilers’ course, but it’s fair to say that the algorithms for computing the parse tables are more complicated and more time consuming than the parsing algorithms that use them.