Introduction

Introduction to Theoretical Computer Science



Today computers are used everywhere: banks, hospitals, schools, airline companies, gas stations, grocery stores, in our cars, in home appliances, PCs, etc., etc. Some are used to crunch numbers, some are used to process images, some are used to process other non-numeric data and some are used to control operations of various devices. They can reason, they can prove many mathematical theorems, they can beat chess experts in their game, they can solve some very complex problems, they can understand our languages, they can answer our questions and of course they can crunch numbers much much faster than us.
Let us for a moment call what computers do computation for convenience, though some of the things computers do such as controling appliances, answering our questions etc. don't fall into our traditional sense of computation. Then these computers seem to be able to compute an awfully lot of things if not everything. But are they capable of computing anything ? Are there things computers can not do ? If there are things computers can not do, what are they ? And why ? If there aren't things computers can not do, then how can we tell ? What do we exactly mean by computation ?
Unfortunately there are many things computers can not do. Computers can not solve certain types of problems. For example no computer can tell in general whether or not a given computer program stops after a finite amount of time on a given input. They can not solve some other types of problems fast enough even though they can solve them in some finite amount of time. For example take the traveling salesman problem: a salesman is given a road map with distances between cities and wants to find a shortest round trip route that visits all the cities on the map exactly once. At the moment the so called traveling salesman problem requires an extremely large amount of time to solve. No one has been able to find a reasonably fast algorithm to solve it and the consensus is that it is not likely that anyone can find such an algorithm.

I have just given you an example of the problems that computers could not solve. How do we know that that is the case ? Are there other problems like that ? How can we tell whther or not a given problem can be solved and solved fast enough ?

The main objective of this course is to answer those questions, that is to study limitations of computers and computation. We are going to investigate limitations of computers and computations by studying the essence of compuers and computations rather than all the variations of computer and computation. This essence is a device called Turing machine. It was first conceived of by Alan Turing in early 20-th century. It is a very simple device but remarkably, every task modern computers perform can also be accomplished by Turing machines. Though it has not been proven, it is generally believed (Church's thesis) that any "computation" humans do can be done by Turing machines and that "computation" is the computation performed by Turing machines.
Thus by studying Turing machines we can learn capabilities hence limitatgions of computers.

Before proceeding to the study of Turing machines and their computations in this course, we study a simpler type of computing device called finite automata. Finite automata are very similar to Turing machines but a few restrictions are imposed on them. Consequently they are less capable than Turing machines but then their operations are simpler. So they provide a good introduction to our study of Turing machines. In addition finite automata can model a large number of systems used in practice. Thus they are a powerful tool to design and study those systems with.

We call a set of strings (of symbols) a language. Finite automata process strings. More specifically they answer the question whether or not a given string belongs to a language. We say finite automata recognize languages. It turns out that solving a problem can be viewed as recognizing a language. Thus when a finite automaton is processing strings, it can actually be solving a problem.
The languages that are recognized by finite automata are called regular languages. Since so many systems in practice can be described by regular languages, we are also going to study regular languages in detail as well as finite automata. We are going to learn their properties, ways to describe them and how to use them to model many of the real life systems.

Turing machines also recognize languages. The languages Turing machines recognize are called Type 0 (or phrase structure) languages (regular languages are Type 3) and they are more complex than regular languages. These two type of languages belong to a hierarchy of four languages called Homsky hierarchy. The other two are context-free languages and context-sensitive languages. Those four languages are together called formal languages.

After briefly studying context-free languges, which are also heavily used in practice, we go to Turing machines. There we learn how computers can be simulated by Turing machines and what it means that a Turing machine recognizes (decides) a language, which is the key to the unsolvability of problem by computers.
Then with Turing machines we investigate limitations of computers and computations. In particular we are going to see a few problems that can not be solved by Turing machines hence by computers and how we can tell that they are unsolvable.

Our last topic is time complexities of various problems. Among the solvable problems there are problems that can be solved within a reasonable amount of time and there are problems that are known to require a finite but very large amount of time to solve. We are going to see some of those which take a large amount of time. Unfortunately there is nothing we can do to speed them up. The time needed to solve such a problem increases at least exponentially with the size of the problem as long as we use Turing machines (hence computers). Increasing the processor speed does not help much for such problems. If the computation time is 2n where n is the size of the problem, then even if the processor speed increased 1,000 times it can handle only ten or so more larger problem sizes. For example for the traveling salesman problem if 100 cities were too many to solve fast enough, then with the increase in the processor speed of 1,000 times 110 cities would already be too many.
The time complexity issues are investigated using Turing machines so that the results apply to all computers.

Let us start with review of mathematics.




Next -- Review of Mathematics --- Logic

Back to Study Schedule

Back to Table of Contents