CS390, Fall 2021

CS390 Outline

Fall 2021

Welcome to CS390.

  • Each module consists of a series of activities.
    • Not every assigned activity requires you to submit something for grading. Nonetheless, you are expected to do them all.

    • Graded activities are due by the end of the day (Eastern Time) of the day they are due.

    • If no due date is specified, you are supposed to complete the assigned activity by the end of the final day allotted for that entire module.

      In other words, unless I say otherwise, quizzes are due by 11:59:59PM on the final day of each module.

KEYS TO SUCCESS IN THIS COURSE:

  1. READ THE SYLLABUS

    The syllabus lays out the basic course policies. It tells you what you need to do to earn a passing grade. It tells you when you need to have done that by. It tells you how to get in touch with me if you run into problems.

  2. HAVE A SCHEDULE

    You have the freedom to schedule your own time in this course, but you DO need to set up a schedule. Don’t forget that this course exists and that you are registered for it. Don’t think you can repeatedly set it aside for weeks at a time and make up the time later.

  3. IF YOU DON"T UNDERSTAND SOMETHING, ASK QUESTIONS

    In a web course, my role as Instructor changes from “lecturer” to “tutor”. You can ask questions in the course Forums. You can send me email. You can also contact me during office hours. You’ll find more information on these options in the syllabus and other documents on the Course Policies page.

    Some people are too shy to ask questions. Some are too proud to ask questions. My advice to both groups is to get over it! Part of being educated is knowing how to exploit your available information resources. In this course, I am one of those resources.

Upcoming Events
1 Getting Started 08/29/2021 - 09/04/2021

Overview

This module introduces you to the course organization, policies, and mechanics. We’ll review the structure of the course website and give you an opportunity to get set up for the semester to come.

We’ll take a brief look at the major themes and areas of emphasis that you can expect to hear more about through the coming semester.

Objectives

  • Identify the requirements, protocols, and policies for the course.

  • Use the selected technology tools for communication and collaboration.

  • Discuss the course themes and their relation to software development.

Relevance

An understanding of the tools used in an online course is fundamental in becoming a successful online learner. It is also important to identify the expectations for participation, assignment submission, and the time management skills required in the online format.

Activities
  1. video Watch Welcome to CS390

  2. lecture Read: CS 390 Syllabus - Fall 2021

  3. lecture Read: Communications

  4. event Attend: Orientation session 08/30/2021, 7:30PM EDT

  5. reading Read Hopcroft, ch 1.1

  6. lecture Read: Typing Mathematics

  7. selfassess Take the self-assessment: Typing Mathematics in Blackboard (Except where a specific link is provided, self-assessments will be found in Blackboard)

2 Mathematics Review 09/05/2021 - 09/11/2021

Overview

This module reviews a number of topics in discrete mathematics that form the basis for the core content of this course.

With one exception (the definition of a “language”), nothing in this section should seem completely unfamiliar to you, and this module will not provide a complete instruction in these topics. rather, this is an opportunity to review and refresh your memory and, if need be, to learn whether you need to return to your textbooks from earlier courses to seek more detailed explanations.

The concept of a “language” used in computer science theory is simpler and more general than what we generally think of when we talk of “programming languages” or, for that matter, “natural languages” like English. In particular, the languages studied in this course focus on the “form” of sentences rather than on their meaning. But in both programming languages and natural languages, a malformed sentence may fail to communicate its intended meaning or even communicate an incorrect or unintended meaning.

Objectives

  • Understand the basic definitions and concepts of logic, sets, mathematical functions and relations.

  • Read and write meaningful statements in the mathematical terminology common to those subjects.

  • Understand the idea of a “language” and the use of recursion in defining languages.

  • Read and write proofs of simple theorems related to the above-named mathematical subjects, including proofs by induction.

Relevance

This course will eventually focus on formal models of languages and of the machines (automata) that can process those languages. Our definitions of these languages and automata will be rooted in the more basic discrete structures reviewed in this module.

Activities
  1. reading Read Hopcroft, 1

  2. lecture Read: Methods of Proof

  3. lecture Read: When Writing Your Own Proofs...
  4. lecture Read: Strings and Languages

  5. lecture Read: Math Symbols used in Chapter 1

  6. selfassess Take the self-assessment: Mathematics Review

  7. quiz Take the quiz: Mathematics Review (All quizzes will be found in Blackboard) Due: 09/11/2021

3 Deterministic Finite Automata 09/12/2021 - 09/18/2021

Overview

A finite automaton (FA) is a mathematical model of a machine that switches among a limited number of different states each time it is fed a new piece of input.

This is the most basic and the least powerful of the automata that we will examine this semester. It is, however, still capable of performing a variety of useful tasks, and its very simplicity makes it something that many people can work with intuitively.

Objectives

  • Understand the basic definitions and concepts of finite automata and how they can be used to recognize languages.

  • Construct FAs for some simple languages and determine what strings a given FA will accept.

Relevance

Despite their simplicity, finite automata can process a number of useful languages. Because of their simplicity, programmers find FAs intuitive enough that they are often used to model high-level behaviors in large systems and in complicated programs (even though FAs are not powerful enough to model program behaviors in general.)

Activities
  1. reading Read Hopcroft, 2.1-2.2

  2. lecture Read: Finite State Automata

  3. lab Do: Finite State Automata: Examples

  4. selfassess Take the self-assessment: DFAs

  5. quiz Take the quiz: DFSAs Due: 09/18/2021

4 Nondeterministic Finite Automata 09/19/2021 - 09/25/2021

Overview

Many uses of finite automata are simpler if we allow the FA to be in multiple states simultaneously. This leads us to the idea of non-determinism. We will distinguish between deterministic FAs (DFAs) and non-deterministic FAs (NFAs) and will explore how to convert from one to the other.

Objectives

  • Understand the concept of nondeterminism and how (NFAs) capture that idea.

  • Construct NFAs for some simple languages and determine what languages an NFA will accept.

  • Convert NFAs to DFAs.

Relevance

NDFAs serve as a first model of parallelism in computing. Despite what may appear to be a complication in the model, the NDFA is often easier to write and easier to manipulate than the deterministic variety.

Activities
  1. reading Read Hopcroft, 2.3-2.6

  2. lecture Read: Nondeterministic Finite Automata

  3. lab Do: Nondeterministic Finite State Automata: Examples

  4. selfassess Take the self-assessment: NFAs

  5. quiz Take the quiz: NFSAs Due: 09/25/2021

5 Regular Expressions 09/26/2021 - 10/02/2021

Overview

For every interesting group of automata, there is a corresponding set of languages that can be recognized by those automata. In this module, we examine the regular languages, the set of languages recognized by FAs. We look at regular expressions, a common notation for describing such languages, and one that should already be familiar to most students.

We will prove that every regular expression can be expressed as a FA, and that every FA can be described by a regular expression.

Objectives

  • Read and write regular expressions for desired regular languages.

  • Construct NFAs corresponding to a regular expression and regular expressions describing a given NFA.

Relevance

Regular expressions are pervasive in programming. They are a common tool in string searching, and students should recall their introduction in CS252 for that purpose. They are popular enough to be included in the standard libraries for both C++ and Java.

Regular expressions are also widely employed in programming language compilers. The first phase of a typical compiler (the “scanner” or “lexical analyzer”) compresses the input by reducing a string of characters to a string of tokens. Those tokens are typically recognized by a FA, which may have thousands of states. Compiler developers use automated tools to generate these FAs from a collection of regular expressions.

Activities
  1. reading Read Hopcroft, 3

  2. lecture Read: Regular Expressions

  3. lab Do: Regular Expressions: Examples

  4. selfassess Take the self-assessment: Regular Expressions

  5. quiz Take the quiz: Regular Expressions Due: 10/02/2021

6 Regular Languages 10/03/2021 - 10/08/2021

Overview

In the previous modules, we have seen two very different ways to describe regular languages: by giving a FA or by a regular expression.

In this section we focus on the important properties of the languages themselves:

  • closure properties, which say how we can build new regular languages from combinations of existing ones, and

  • decision properties, which allow us to decide ask questions about regular languages

Objectives

  • Apply the pumping lemma to determine what languages can be recognized by finite automata.

  • Understand command ways of combining regular languages without losing their regularity.

  • Modify a FA to minimize the number of states.

Relevance

The properties discussed in this section speak a lot to the suitability of FAs for a variety of practical tasks. They tell us much of what can and cannot be processed by a FA, what kinds of questions we can answer about regular languages, and how to prepare an FA for efficient processing.

Activities
  1. reading Read Hopcroft, 4

  2. lecture Read: Properties of Regular Languages

  3. lecture Read: Regular Languages - Sample Problems

  4. selfassess Take the self-assessment: Regular Languages

  5. quiz Take the quiz: Regular Languages 2021-10-09

  6. exam Take the exam: End of material for midterm exam

7 Context-Free Languages 10/13/2021 - 10/23/2021

Overview

This module introduces the idea of “grammars”, a powerful tool for recursively describing a language. Grammars allow us to derive sentences in a language or to “parse” a string to see if it is a sentence in our language.

We will then explore the idea of a “context-free grammar” and the set of languages, the “context-free languages” described by those grammars.

The recursive structure inherent in grammars means that if we can parse a sentence, we learn about the “structure” of that sentence as revealed in its derivation tree. In practical problems, that structure often tells us something of the “meaning” of the sentence. The discovery of that meaning can be compromised, however, if the grammar is “ambiguous” in its structure for that sentence.

Objectives

  • Understand the basic definitions and concepts of grammars, derivation, and parsing.

  • Use a grammar to derive sentences in a language.

  • Understand the basic definition of context-free grammars.

  • Read and write context-free grammars for some languages.

  • Use a context-free grammar to parse sentences and obtain a derivation tree.

  • Recognize ambiguity and understand the problems that it can entail.

Relevance

Context-free grammars (CFGs) are a primary tool in compiler design. Efficient parsing algorithms are known for (a subset of) the context-free languages, and only slower algorithms for parsing more general languages than context-free. Therefore programming language designers make a conscious effort to make sure that their languages are context-free or nearly so.

Parsing is the heart of programming-language compilation because programming languages reveal a substantial part of their meaning through the structure of each sentence.

Activities
  1. reading Read Hopcroft, ch 5

  2. lecture Read: Context-Free Grammars and Languages

  3. lab Do: Context-Free Grammars -- Examples

  4. selfassess Take the self-assessment: Context Free Grammars

  5. quiz Take the quiz: Context Free Grammars Due: 10/23/2021

8 Push-Down Automata 10/24/2021 - 10/30/2021

Overview

Pushdown automata (PDAs) can be thought of as combining an FA “control-unit” with a “memory” in the form of an infinite stack. PDAs are more powerful than FAs, being able to recognize languages that FAs cannot. In fact, the set of languages that can be recognized by PDAs are the context-free languages of the previous module.

Objectives

  • Understand the basic definitions and concepts of pushdown automata.

  • Understand the distinctions between deterministic and non-deterministic PDAs.

  • Create a PDA corresponding to a CFG, and vice-versa.

Relevance

Efficient parsing algorithms are known for (a subset of) the context-free languages. These algorithms combine a large FA and a stack, in effect mirroring the structure of a PDA. Compiler writers often use automated tools to generate that FA from a context free grammar.

Activities
  1. reading Read Hopcroft, ch 6

  2. lecture Read: Pushdown Automata

  3. lab Do: Pushdown Automata: Examples

  4. selfassess Take the self-assessment: Push-Down Automata

  5. quiz Take the quiz: Push-Down Automata quiz Due: 10/30/2021

9 Properties of Context-Free Languages 10/31/2021 - 11/06/2021

Overview

This module explores decision properties for context-free languages.

The most important of these is the problem of deciding whether a language is context-free. A key tool in recognizing which languages are context-free is the pumping lemma, similar to the one introduced earlier for regular languages.

We will also examine common closure properties for CFLs.

Objectives

  • Understand the pumping lemma for context-free languages.

  • Apply the pumping lemma to determine if a language is context-free.

  • Understand common decision problems involving context-free languages.

  • Know the common transformations under which CFLs are closed.

Relevance

Not all languages are context-free, but given the importance of the context-free languages, deciding whether a given language is context-free or not is important. This module explores that decision problem.

Activities
  1. reading Read Hopcroft, ch 7

  2. lecture Read: Properties of Context-Free Languages

  3. lecture Read: Properties of Context-Free Languages - Examples

  4. selfassess Take the self-assessment: CFLs

  5. quiz Take the quiz: CFL quiz Due: 11/06/2021

10 Turing Machines 11/07/2021 - 11/16/2021

Overview

A Turing machine combines a finite state controller with a memory in the form of an infinite tape that can move back and forth beneath a read-write head.

Turing machines are not only more powerful than PDAs, they are believed to be a general model of computation in that computation that can be expressed as an algorithm can be implemented by a Turing machine.

Objectives

  • Understand the concept and structure of a Turing machine.

  • Apply a Turing machine to a string to see if it is a sentence in the language denoted by that machine.

  • Construct Turing machines for some languages.

  • Understand the Church-Turing thesis and its relation to computing.

  • Understand the idea and significance of universal Turing machines.

Relevance

Turing machines are believed to be a general model of computation. Anything we can write as an algorithm can be done on a Turing machine, and anything a Turing machine can do can be described as an algorithm.

There’s a certain element of faith in this thesis. We need to agree first on what we mean by an “algorithm”. But this thesis is, in many ways, at the very heart of computer science theory.

Activities
  1. reading Read Hopcroft, ch 8

  2. lecture Read: Turing machines

  3. lab Do: Turing Machines: Examples

  4. selfassess Take the self-assessment: Turing machines

  5. quiz Take the quiz: Turing Machines Due: 11/16/2021

11 Undecidable Problems 11/17/2021 - 12/04/2021

Overview

The set of languages recognizable via Turing machines is the “recursively enumerable” languages. This modules explores these andand the process for enumerating the sentences in them.

There are things that can be computed and things that cannot be (algorithmically) computed. By implication, there are decision problems (boolean functions) that are undecidable by any computing device.

This module explores some of the widely known undecidable problems, including the famous halting problem – will a given program ever stop or will it get stuck in an infinite loop?

Objectives

  • Understand the basic definitions and grammar forms for recursively enumerable languages.

  • Understand the concept of undecidability.

  • Know some of the more common undecidable problems.

  • Prove decidability or undecidability for some problems.

Relevance

We have spent a lot of time taking about the things that can be computed. By implication, there are some things that cannot be computed.

This is, in some ways, the pay-off of this entire course. In this module we come up against the fundamental limits of what we can possibly do with computers.

One might expect that the set of uncomputable functions or undecidable problems might be rare or arcane, or perhaps be mired in “soft” concepts that call for subjective judgment of the kind reserved to human intelligence. But actually undecidable problems tend to pop up the moment we start asking practical questions about our programs. To paraphrase Lewis Carroll, a software engineer may sometimes have to solve as many as six undecidable problems before breakfast.

Activities
  1. reading Read Hopcroft, ch 9

  2. lecture Read: Undecidable Problems

  3. lecture Read: (Un)Decidability -- Sample Problems

  1. selfassess Take the self-assessment: Undecidable problems

  2. quiz Take the quiz: Undecidable Problems Due: 12/04/2021

12 The Church-Turing Thesis and Real Programming Languages 12/05/2021 - 12/10/2021

Overview

What can Turing machines and the Church-Turing thesis tell us about real programming languages? Are some programming languages more “powerful” than others? Or is the choice of programming language more a matter of style and convenience?

In this module we examine some of the practical implications of automata theory to real programming languages.

Objectives

  • Understand the concept of a “Turing-complete” programming language.

  • Know the common programming language features that contribute to Turing-completeness.

  • Know how to prove that a programming language is Turing-complete.

Relevance

Turing machines are believed to be a general model of computation. Anything we can write as an algorithm can be done on a Turing machine, and anything a Turing machine can do can be described as an algorithm.

What that thesis does not say, however, is whether any of our modern programming languages are capable of expressing all algorithms.

Activities
  1. lecture Read: Turing Completeness
13 Special Dates
Activities
  1. event Attend: Fall Holiday - no classes 10/09/2021 - 10/12/2021
  2. exam Take the exam: Midterm Exam, on Blackboard 10/14/2021 - 10/15/2021
  3. event Attend: Thanksgiving Break - no classes 11/24/2021 - 11/28/2021
  4. exam Take the exam: Final Exam (on Blackboard) 12/14/2021 - 12/15/2021
  5. event Attend: Last day of classes 12/10/2021
Symbol Key
lecture Lecture Notes
video Video
slides Slides
reading Textbook readings
lab Lab

All times in this schedule are given in Eastern Time.