Introduction to Theoretical Computer Science
Today computers are used everywhere: banks, hospitals, schools, airline companies,
gas stations, grocery stores, in our cars, in home appliances, PCs, etc., etc.
Some are used to crunch numbers, some are used to process images, some are used to process
other non-numeric data and some are used to control operations of various devices.
They can reason, they can prove many mathematical theorems, they can beat chess experts
in their game, they can solve some very
complex problems, they can understand our
languages, they can answer our questions and of course they can crunch numbers much much
faster than us.
Let us for a moment call what computers do computation for convenience, though some of
computers do such as controling
appliances, answering our questions etc. don't fall into our traditional sense of
Then these computers seem to be able to compute an awfully lot of things if not everything.
But are they capable of computing anything ?
Are there things computers can not do ?
If there are things computers can not do, what are they ? And why ? If there aren't things
computers can not do, then how can we tell ? What do we exactly mean by computation ?
Unfortunately there are many things computers can not do. Computers can not solve certain
types of problems. For example no computer can tell in general whether or not a given
computer program stops after a finite amount of time on a given input. They can not
solve some other types of problems fast enough even though they can solve them in some
amount of time. For example take the traveling salesman problem: a salesman is given a road
map with distances between
cities and wants to find a shortest round trip route that visits all the cities on the map
At the moment the so called
traveling salesman problem requires an extremely large amount of time to solve. No one
has been able to find a reasonably fast algorithm to solve it and the consensus is that
it is not likely that anyone
can find such an algorithm.
I have just given you an example of the problems that computers could not solve.
How do we know that that is the case ? Are there other problems like that ?
How can we tell whther or not
a given problem can be solved and solved fast enough ?
The main objective of this course is to answer those questions, that is to study limitations
of computers and computation.
We are going to investigate limitations of computers and computations by studying
of compuers and computations rather than all the variations of computer and computation.
This essence is a device called Turing machine. It was first conceived of by Alan Turing
in early 20-th century. It is a very simple device but remarkably, every task modern
computers perform can also be accomplished by Turing machines. Though it has not been
proven, it is generally believed (Church's thesis) that any "computation" humans do can be
done by Turing machines
and that "computation" is the computation performed by Turing machines.
Thus by studying Turing machines we can learn capabilities hence limitatgions of computers.
Before proceeding to the study of Turing machines and their computations in this course,
a simpler type of computing device called finite automata. Finite automata are very similar
to Turing machines but a few restrictions are imposed on them. Consequently they are less
capable than Turing machines but then their operations are simpler. So they provide a good
introduction to our study of Turing machines. In addition finite automata can model a large
number of systems used in practice. Thus they are a powerful tool to design and study those
We call a set of strings (of symbols) a language. Finite automata process strings.
More specifically they answer the question whether
or not a given string belongs to a language. We say finite automata recognize languages.
It turns out that solving a problem
can be viewed as recognizing a language. Thus when a finite automaton is processing
strings, it can actually be solving a problem.
The languages that are recognized by finite automata are called regular languages.
Since so many systems in practice can be described by regular languages, we are also going
to study regular languages in detail as well as finite automata. We are going to learn
their properties, ways to describe them
and how to use them to model many of the real life systems.
Turing machines also recognize languages. The languages Turing machines recognize are
called Type 0 (or phrase structure) languages (regular languages are Type 3) and they are
more complex than regular languages.
These two type of languages belong to a hierarchy of four languages called Homsky hierarchy.
The other two are context-free languages and context-sensitive languages. Those four
are together called formal languages.
After briefly studying context-free languges, which are also heavily used in practice,
we go to Turing machines. There we learn how computers can be simulated by Turing machines
and what it means that a Turing machine recognizes (decides) a language, which is the key
to the unsolvability of problem by computers.
Then with Turing machines we investigate limitations of computers and computations.
In particular we are going to see a few problems that can not be solved by Turing machines
hence by computers and how we can tell that they are unsolvable.
Our last topic is time complexities of various problems. Among the solvable problems
there are problems that can be solved within a reasonable amount of time and there are
problems that are known to require a finite but very large amount of time to solve.
We are going to
see some of those which take a large amount of time. Unfortunately there is nothing we can
do to speed them up. The time needed to solve such a problem increases at least
with the size of the problem as long as we use Turing machines (hence computers).
Increasing the processor speed does not help much for such problems. If the computation time
where n is the size of the problem, then even if the processor speed increased 1,000 times
it can handle only ten or so more larger problem sizes. For example for the traveling
problem if 100 cities were too many to solve fast enough, then with the increase in
speed of 1,000 times 110 cities would already be too many.
The time complexity issues are investigated using Turing machines so that the results
apply to all computers.
Let us start with review of mathematics.
Next -- Review of Mathematics --- Logic
Back to Study Schedule
Back to Table of Contents