CS390 Outline

Overview

This module introduces you to the course organization, policies, and mechanics. We’ll review the structure of the course website and give you an opportunity to get set up for the semester to come.

We’ll take a brief look at the major themes and areas of emphasis that you can expect to hear more about through the coming semester.

Objectives

• Identify the requirements, protocols, and policies for the course.

• Use the selected technology tools for communication and collaboration.

• Discuss the course themes and their relation to software development.

Relevance

An understanding of the tools used in an online course is fundamental in becoming a successful online learner. It is also important to identify the expectations for participation, assignment submission, and the time management skills required in the online format.

Overview

This module reviews a number of topics in discrete mathematics that form the basis for the core content of this course.

With one exception (the definition of a “language”), nothing in this section should seem completely unfamiliar to you, and this module will not provide a complete instruction in these topics. rather, this is an opportunity to review and refresh your memory and, if need be, to learn whether you need to return to your textbooks from earlier courses to seek more detailed explanations.

The concept of a “language” used in computer science theory is simpler and more general than what we generally think of when we talk of “programming languages” or, for that matter, “natural languages” like English. In particular, the languages studied in this course focus on the “form” of sentences rather than on their meaning. But in both programming languages and natural languages, a malformed sentence may fail to communicate its intended meaning or even communicate an incorrect or unintended meaning.

Objectives

• Understand the basic definitions and concepts of logic, sets, mathematical functions and relations.

• Read and write meaningful statements in the mathematical terminology common to those subjects.

• Understand the idea of a “language” and the use of recursion in defining languages.

• Read and write proofs of simple theorems related to the above-named mathematical subjects, including proofs by induction.

Relevance

This course will eventually focus on formal models of languages and of the machines (automata) that can process those languages. Our definitions of these languages and automata will be rooted in the more basic discrete structures reviewed in this module.

Overview

A finite automaton (FA) is a mathematical model of a machine that switches among a limited number of different states each time it is fed a new piece of input.

This is the most basic and the least powerful of the automata that we will examine this semester. It is, however, still capable of performing a variety of useful tasks, and its very simplicity makes it something that many people can work with intuitively.

many uses of finite automata are simpler if we allow the FA to be in multiple states simultaneously. This leads us to the idea of non-determinism. We will distinguish between deterministic FAs (DFAs) and non-deterministic FAs (NFAs) and will explore how to convert from one to the other.

Objectives

• Understand the basic definitions and concepts of finite automata and how they can be used to recognize languages.

• Construct FAs for some simple languages and determine what strings a given FA will accept.

• Understand the concept of nondeterminism and how (NFAs) capture that idea.

• Construct NFAs for some simple languages and determine what languages an NFA will accept.

• Convert NFAs to DFAs.

Relevance

Despite their simplicity, finite automata can process a number of useful languages. Because of their simplicity, programmers find FAs intuitive enough that they are often used to model high-level behaviors in large systems and in complicated programs (even though FAs are not powerful enough to model program behaviors in general.)

Overview

For every interesting group of automata, there is a corresponding set of languages that can be recognized by those automata. In this module, we examine the regular languages, the set of languages recognized by FAs. We look at regular expressions, a common notation for describing such languages, and one that should already be familiar to most students.

We will prove that every regular expression can be expressed as a FA, and that every FA can be described by a regular expression.

Objectives

• Read and write regular expressions for desired regular languages.

• Construct NFAs corresponding to a regular expression and regular expressions describing a given NFA.

Relevance

Regular expressions are pervasive in programming. They are a common tool in string searching, and students should recall their introduction in CS252 for that purpose. They are popular enough to be included in the standard libraries for both C++ and Java.

Regular expressions are also widely employed in programming language compilers. The first phase of a typical compiler (the “scanner” or “lexical analyzer”) compresses the input by reducing a string of characters to a string of tokens. Those tokens are typically recognized by a FA, which may have thousands of states. Compiler developers use automated tools to generate these FAs from a collection of regular expressions.

Overview

In the previous modules, we have seen two very different ways to describe regular languages: by giving a FA or by a regular expression.

In this section we focus on the important properties of the languages themselves:

• closure properties, which say how we can build new regular languages from combinations of existing ones, and

• decision properties, which allow us to decide ask questions about regular languages

Objectives

• Apply the pumping lemma to determine what languages can be recognized by finite automata.

• Understand command ways of combining regular languages without losing their regularity.

• Modify a FA to minimize the number of states.

Relevance

The properties discussed in this section speak a lot to the suitability of FAs for a variety of practical tasks. They tell us much of what can and cannot be processed by a FA, what kinds of questions we can answer about regular languages, and how to prepare an FA for efficient processing.

Overview

This module introduces the idea of “grammars”, a powerful tool for recursively describing a language. Grammars allow us to derive sentences in a language or to “parse” a string to see if it is a sentence in our language.

We will then explore the idea of a “context-free grammar” and the set of languages, the “context-free languages” described by those grammars.

The recursive structure inherent in grammars means that if we can parse a sentence, we learn about the “structure” of that sentence as revealed in its derivation tree. In practical problems, that structure often tells us something of the “meaning” of the sentence. The discovery of that meaning can be compromised, however, if the grammar is “ambiguous” in its structure for that sentence.

Objectives

• Understand the basic definitions and concepts of grammars, derivation, and parsing.

• Use a grammar to derive sentences in a language.

• Understand the basic definition of context-free grammars.

• Read and write context-free grammars for some languages.

• Use a context-free grammar to parse sentences and obtain a derivation tree.

• Recognize ambiguity and understand the problems that it can entail.

Relevance

Context-free grammars (CFGs) are a primary tool in compiler design. Efficient parsing algorithms are known for (a subset of) the context-free languages, and only slower algorithms for parsing more general lantauges than context-free. Therefore programming language designers make a conscious effort to make sure that their languages are context-free or nearly so.

Parsing is the heart of programming-language compilation because programming languages reveal a substantial part of their meaning through the structure of each sentence.

Overview

Pushdown automata (PDAs) can be thought of as combining an FA “control-unit” with a “memory” in the form of an infinite stack. PDAs are more powerful than FAs, being able to recognize languages that FAs cannot. In fact, the set of languages that can be recognized by PDAs are the context-free languages of the previous module.

Objectives

• Understand the basic definitions and concepts of pushdown automata.

• Understand the distinctions between deterministic and non-deterministic PDAs.

• Create a PDA corresponding to a CFG, and vice-versa.

Relevance

Efficient parsing algorithms are known for (a subset of) the context-free languages. These algorithms combine a large FA and a stack, in effect mirroring the structure of a PDA. Compiler writers often use automated tools to generate that FA from a context free grammar.

Overview

This module explores decision properties for context-free languages.

The most important of these is the problem of deciding whether a language is context-free. A key tool in recognizing which languages are context-free is the pumping lemma, similar to the one introduced earlier for regular languages.

We will also examine common closure properties for CFLs.

Objectives

• Understand the pumping lemma for context-free languages.

• Apply the pumping lemma to determine if a language is context-free.

• Understand common decision problems involving context-free languages.

• Know the common transformations under which CFLs are closed.

Relevance

Not all languages are context-free, but given the importance of the context-free languages, deciding whether a given language is context-free or not is important. This module explores that decision problem.

Overview

A Turing machine combines a finite state controller with a memory in the form of an infinite tape that can move back and forth beneath a read-write head.

Turing machines are not only more powerful than PDAs, they are believed to be a general model of computation in that computation that can be expressed as an algorithm can be implemented by a Turing machine.

Objectives

• Understand the concept and structure of a Turing machine.

• Apply a Turing machine to a string to see if it is a sentence in the language denoted by that machine.

• Construct Turing machines for some languages.

• Understand the Church-Turing thesis and its relation to computing.

• Understand the idea and significance of universal Turing machines.

Relevance

Turing machines are believed to be a general model of computation. Anything we can write as an algorithm can be done on a Turing machine, and anything a Turing machine can do can be described as an algorithm.

There’s a certain element of faith in this thesis. We need to agree first on what we mean by an “algorithm”. But this thesis is, in many ways, at the very heart of computer science theory.

Overview

The set of languages recognizable via Turing machines is the “recursively enumerable” languages. This modules explores these andand the process for enumerating the sentences in them.

There are things that can be computed and things that cannot be (algorithmicly) computed. By implication, there are decision problems (boolean functions) that are undecidable by any computing device.

This module explores some of the widely known undecidable problems, including the famous halting problem – will a given program ever stop or will it get stuck in an infinite loop?

Objectives

• Understand the basic definitions and grammar forms for recursively enumerable languages.

• Understand the concept of undecidability.

• Know some of the more common undecidable problems.

• Prove decidability or undecidability for some problems.

Relevance

We have spent a lot of time taking about the things that can be computed. By implication, there are some things that cannot be computed.

This is, in some ways, the pay-off of this entire course. In this module we come up against the fundamental limits of what we can possibly do with computers.

One might expect that the set of uncomputable functions or undecidable problems might be rare or arcane, or perhaps be mired in “soft” concepts that call for subjective judgment of the kind reserved to human intelligence. But actually undecidable problems tend to pop up the moment we start asking practical questions about our programs. To paraphrase Lewis Carroll, a software engineer may sometimes have to solve as many as six undecidable problems before breakfast.

Overview

What can Turing machines and the Church-Turing thesis tell us about real programming languages? Are some programming languages more “powerful” than others? Or is the choice of programming language more a matter of style and convenience?

In this module we examine some of the practical implications of automata theory to real programming languages.

Objectives

• Understand the concept of a “Turing-complete” programming language.

• Know the common programming language features that contribute to Turing-completeness.

• Know how to prove that a programming language is Turing-complete.

Relevance

Turing machines are believed to be a general model of computation. Anything we can write as an algorithm can be done on a Turing machine, and anything a Turing machine can do can be described as an algorithm.

What that thesis does not say, however, is whether any of our modern programming languages are capable of expressing all algorithms.