Build Management with Make
Steven Zeil
When you begin to develop projects that involve multiple files that need to be compiled or otherwise processed, keeping them all up-to-date can be a problem. Even more of a problem is passing them on to someone else (e.g., your instructor) and expecting them to know what to do to build your project from the source code.
The Unix program make
is designed to simplify such project build management. In a makefile, you record the steps necessary to build both the final file (e.g., your executable program) and each intermediate file (e.g., the .o
files produced by compiling a single C++ source code file).
We say that a file
file1
depends upon a second filefile2
if thefile2
is used as input to some command used to producefile1
.
When the make
program is run, it then checks to be sure that all of the needed files exist, and that each needed file has been updated more recently than all of the files it depends upon.
1 Makefiles
The key bits of information in a makefile, therefore are
- For each file, a list of other files it depends upon, and
- The command used to produce the dependent file from the files it depends upon.
A makefile may also include various macros/abbreviations designed to simplify the task of dealing with many instances of the same commands or files.
Suppose that we are engaged in a project to produce a Java program, to be stored in a Jar file named pie.jar
.
pie.jar
will be created by compiling filesPie.java
to produce a compiled filePie.class
, andPieSlicer.java
to produce a compiled filePieSlicer.class
, andPieView.java
to produce a compiled filePieView.class
.class
files intopie.jar
.- The
.jar
file will be created using ajar
command. - The
.class
files will be created using ajavac
command.
By the definition above, we can say that
pie.jar
depends onPie.class
,PieSlicer.class
, andPieView.class
.
However, that’s not all of the dependencies to be considered. If we look through the Java source code, we can observe that PieSlicer.java
contains mentions of the Pie
class and of the PieView
class, and that PieView.java
contains mentions of the Pie
class.
Remember that when a file of Java source code is being compiled, if that source code mentions another class, e.g., Pie
, then the compiler looks for an already compiled Pie.class
file and reads that for information about that class.
So we conclude that:
PieSlicer.class
depends onPieSlicer.java
,Pie.class
andPieView.class
, andPieView.class
depends onPieView.java
, andPie.class
.Pie.class
depends onPie.java
.
Here is a makefile for this project. This file should reside in the project directory, and should be called “Makefile
” or “makefile
”.
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
PieSlicer.class: PieSlicer.java PieView.class Pie.class
javac -g PieSlicer.java
Pie.class: Pie.java
javac -g Pie.java
PieView.class: PieView.java Pie.class
javac -g PieView.java
1.1 Rules
The key information in a makefile consists of a variety of rules for producing “target” files. Rules have the form:
targetFile: inputFile1 inputFile2 ...
command1
command2
⋮
-
Each target rule begins with a single line containing the name of the file to produce, a colon, and then a list of all files that serve as inputs to the commands that produce the file.
-
Following that are any number of command lines that give the Unix commands to actually produce the file.
Each command line starts with a “Tab” character (invisible in this listing).
Let’s take a good look at some of the rules in our earlier example:
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
This rule says that the file pie.jar
is created by the command
jar cfe pie.jar PieSlicer *.class
and that the inputs to those commands are the files Pie.class
, PieSlicer.class
, and PieView.class
.
(A subtle point about compiling Java code. When you compile a source file foo.java
, you will get a new foo.class
file, but you may also get additional .class
files with names like foo$bar.class
. We need all of these class files to go into the jar. That is why the jar
command uses the wildcard *.class
.)
PieView.class: PieView.java Pie.class
javac -g PieView.java
This rule says that the file PieView.class
is produced by running the command
javac -g PieView.java
and that the inputs to this command are PieView.java
, and Pie.class
.
All of the rules in our sample make file need only a single Unix command to generate the target file. That’s common, but not required.
If we wanted to, we could have written the first rule as
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe wrongName.jar PieSlicer *.class
mv wrongName.jar pie.jar
It would be a bit silly, but it’s legal.
The introduction of the now-familiar mv
command also highlights something else about make files. You can put pretty much any Unix command into a make file – you aren’t limited to “compilation” commands. Generally, you won’t want to put interactive commands (commands that pause and wait for human input, e.g. editors) into a make file. It’s not illegal to do so, but since the point of a make file is automate a procedure, interactive commands would usually interfere with the automation.
To write a makefile rule, it is important that you understand the commands that you need to run and know what all their actual inputs are, not just the inputs that are explicitly listed in the command line.
1.2 Running make
Suppose that, with just the above Makefile
and the various source code files in your directory, you issued the command
make pie.jar
make
reads the Makefile
and finds the rule for creating the file pie.jar
:
pie.jar:➀ Pie.class PieSlicer.class PieView.class ➁
jar cfe pie.jar PieSlicer *.class ➂
-
➀ We (and the make command) know that this rule tells how to create the file
pie.jar`` because
pie.jar` is listed as the target, to the left of the colon.You and I know, from the initial discussion, that pie.jar is actually a program, but make does not know that and doesn’t care about that. make regards its task to be the creation of files, and it does not care what is in those files.
-
➁ From the dependency list to the right of the colon, make discovers that, in order to create
pie.jar
, it will first need up-to-date copies ofPie.class
,PieSlicer.class
, andPieView.class
. -
➂ make also learns that, once it has up-to-date copies of those class files, that it can then create pie.jar by running the command
jar cfe pie.jar PieSlicer *.class
In this case, make should realize that it cannot execute the jar
command immediately because it does not have up-to-date copies of the required class files. In fact, none of these files exists. Therefore make sets out to create them, by looking for the appropriate rules for each of them.
It will find the rules
PieSlicer.class: PieSlicer.java PieView.class Pie.class
javac -g PieSlicer.java
Pie.class: Pie.java
javac -g Pie.java
PieView.class: PieView.java Pie.class
javac -g PieView.java
and will consider running the javac
commands to create the .class
files.
However, looking at the dependencies, make will decide that it cannot immediately compile PieSlicer.java
because that needs the files PieView.class
and Pie.class
, which do not exist yet. And it can’t immediately compile PieView.java
because that needs the file Pie.class
. So make eventually concludes that the only sensible order of commands is
javac -g Pie.java
javac -g PieView.java
javac -g PieSlicer.java
1.2.1 The order of rules in a makefile…
…does not matter. make
uses the information we have provided on the dependencies to figure out what order the commands need to be run in.
1.2.2 …with one exception…
If we run the command
make
without naming a target, make
will try to generate the target of the very first rule in the make file.
By convention, we always put the “main” target of our process in the first rule of the make file.
1.2.3 Trying It Out
Example 1: Try This:
In an
xterm
, set up a directory for this exercise:mkdir ~/playing/withMake cd ~/playing/withMake
Copy the files from
~cs252/Assignments/makePie
into this directory.cp ~cs252/Assignments/makePie/* .
Use
ls
andmore
to read through the files that you have copied. You’ll see a copy of the makefile we have just been looking at, together with a collection of Java source code files.Give the command
make Pie.class ls
Look closely at the commands run by
make
.
make
has done exactly what we asked it to do.Give the command
make pie.jar
Look closely at the commands being run. Notice that
make
does not bother recompilingPie.java
. It can se that it already has an up-to-date copy ofPie.class
.Do an
ls
and take note of the new files that have appeared.Run the program with the command
java -jar pie.jar
Give the command
make pie.jar
Look closely at the commands being run, or, more precisely, the commands that are not run.
make
is smart enough to realize that it doesn’t need to do anything to produce files that are already up-to-date.Give the command
rm pie.jar make pie.jar
Look closely at the commands being run and the commands that are not run.
make
realizes that it still has up-to-date versions of the.class
files and does not need to recompile the source code.Give the commands
rm *.class *.jar ls make ls
Because we removed the jar and its class components,
make
rebuilds everything for us.How did it know we wanted to get a new
pie.jar
? We did not give a target to the make command, so by default it build the target of the first rule in our make file.Give the command
make doesNotExist.dat
Take note of the response that you get.
This is a pretty common response from
make
and, to me, seems pretty self-explanatory. But some people seem to be very mystified when they see this response.You can “test” a makefile without actually performing the commands with the
-n
option.Run:
rm *.jar *.class ls make -n ls
Notice that, although
make
listed the commands to build your Java jar, it has not actually executed any of them.
1.3 make Tries to be Smart
One of the most important behaviors of make
is that it is smart enough to only do as much work as is needed. That may not seem important to you right now, working with programs that can be compiled in a second or less, but large programs may have hundreds of source code files that take many minutes or even hours to compile.
If you are trying to modify or debug a program like that, you are probably only modifying one or two files at a time. Typically the only steps needed to rebuild the program will be to compile the changed files and then link all of the .o
files to produce the final executable.
With a proper makefile, make
will be smart enough to do that. That’s because each file has a “last modified date” (which you can view using ls -l
). make
looks at the “last modified” dates on files to see if they have been changed since the last time that make
was run.
Example 2: Try This: Small Changes
Return to your make project:
cd ~/playing/withMake
Give the command
make
just to be sure that the program is already built.
Edit
PieView.java
by adding a Java comment.Give the command
make
again. Look closely at the commands being run.
Notice that
make
is smart enough to re-compile the file we have edited, but knows that it doesn’t have to recompilePie.java
because it is unaffected by (does not depend on)PieView
.
make
does recompilePieSlicer.java
because we told it thatPieSlicer
depends onPieView
. Now, as it happens, all that you did was to add a comment, which would not really change the compilation ofPieSlicer
, butmake
doesn’t know that the change you made was trivial. It only looks at the date and time at which each file was last modified.When testing out makefiles, we can simulate making changes to source code files via the
touch
command.touch
does not actually change the contents of the file, but it does reset the “last modified” time on the file to make it appear that the file has been changed.Give the commands:
make touch PieView.java make
You should see, before the
touch
, ‘make’ does not see a need to recompile anything, but, after thetouch
,make
carries out the same steps as when you actually edited the file.Let’s try a different modification. Simulate the modification of
PieSlicer.java
:touch PieSlicer.java make
Do the steps performed by
make
seem reasonable?
2 More Advanced Makefiles
2.1 Symbols
Thinking ahead, we might realize that we won’t always want to compile with the flags “-g
”.
We can make our makefile more flexible by gathering things that might need to be changed later into a symbol. make
allows us to define symbols like this
SymbolName=string
and later use that symbol like this: $(SymbolName)
.
So we could modify our makefile as follows:
# Compiler settings
JAVAFLAGS=-g
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
PieSlicer.class: PieSlicer.java PieView.class Pie.class
javac ${JAVAFLAGS} PieSlicer.java
Pie.class: Pie.java
javac ${JAVAFLAGS} Pie.java
PieView.class: PieView.java Pie.class
javac ${JAVAFLAGS} PieView.java
2.2 Default (Pattern) Rules
Make files can be simplified by introducing default rules for forming one kind of file from another. Here is an equivalent make file that defines appropriate default rules:
# Compiler settings
JAVAFLAGS=-g
# Pattern for compiling Java code
%.class: %.java
javac ${JAVAFLAGS} $*.java
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
Pie.class: Pie.java
PieView.class: PieView.java Pie.class
PieSlicer.class: PieSlicer.java PieView.class Pie.class
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
Notice, first, that the individual javac
commands have been removed from the rules for generating the various class files. Instead, make
will use the command given in the new pattern rule (highlighted).
The pattern rule uses “%
” as a wildcard character that stands for any string of characters.
So the target pattern %.class
will match any class file.
The same “%
” can appear once in the dependencies list. So the dependency pattern %.java
will match any Java file with the same base file name as a desired class file.
The $*
in the rule inserts whatever string was matched by the “%
”.
So, to generate PieSlicer.class
, make
will
- Notice that
PieSlicer.class
depends onPieSlicer.java
. - See that this matches the pattern (
%
matching “PieSlicer”). - Check to be sure that the other dependencies (
Pie.class
andPieView.class
) are up-to-date and, if not, will generate those. - Take the command from the pattern rule, repalce the
$*
by “PieSlicer”, and issue the commandjavac -g PieSlicer.java
2.3 Artificial Targets
An artificial target in a rule is a “file name” used as a target in a rule but that is never actually created by the commands in the rule. These can be used as a kind of shorthand for commands that we want to run frequently and so set up in the makefile.
There are some very commonly used artificial targets. These are just conventions. They don’t happen automatically, but most people who design makefiles set them up to work this way.
- all
- The target
all
compiles and builds everything (except, sometimes, documentation). So one of the more common ways to invoke make is to saymake all
The
all
target rule, if it is present, is always given as the first rule in the makefile. So, in most makefiles,make
will also compile and build everything.
- clean
- The command
make clean
is often used to clean up a directory, deleting all the temporary files produced bymake all
, leaving only the original files (e.g., the source code) from which everything else can be rebuilt later, if desired.
- install
- In programs that must be “installed” by placing them in special directories, this target controls the commands necessary to do that installation.
A common sequence for building and installing new Unix software is therefore:
make make install make clean
- test
- Less common, runs a test suite to see if the program built successfully.
- doc
- (or
docs
) May be used to build program documentation, user manuals, etc.
We can bring our sample makefile into conformity with these conventions as follows:
# Compiler settings
JAVAFLAGS=-g
# Pattern for compiling Java code
%.class: %.java
javac ${JAVAFLAGS} $*.java
all: pie.jar
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
Pie.class: Pie.java
PieView.class: PieView.java Pie.class
PieSlicer.class: PieSlicer.java PieView.class Pie.class
pie.jar: Pie.class PieSlicer.class PieView.class
jar cfe pie.jar PieSlicer *.class
clean:
rm *.jar *.class
Example 3: Try This:
Edit the
makefile
from the prior Try This to match the listing above.Don’t forget to use tab characters and not spaces in front of the indented commands.
In that
makeTry
directory, give the commands:make ls make clean ls make all ls make clean ls make ls
observing which commands are issued and which files are created at each step.
3 Creating a Makefile
When you are faced with the task of writing your own makefiles, the best way to start is to
-
Make a list of all files that will be involved in your project. This includes the file(s) that constitute the overall “goal” of the project, the files that you yourself will create “manually” using
emacs
or some other text editor, and all the intermediate files that will be produced by the build commands. Do not include the makefile itself in this list.If you’re not sure what those intermediate files will be, write out a list of the commands that you would use to build the project if you were typing those commands, one at a time, into a command shell. Avoid compilation commands that perform multiple compile and link steps via a single command. Think instead of compiling each source code file separately and then separately linking the object files together. Every file mentioned in any of these commands should probably be in your list.
-
Now, divide the files in your list into two groups.
The provided group lists the files that you create via a text editor or other interactive program or that you receive from outside the project and don’t need to create at all.
The constructed group lists those files that will be created by running a non-interactive Unix command using other files (from either or both groups) as input.
- How can you tell if a command is interactive or not?
-
It’s not enough to look and see if the file contains text, or even if it looks like program source code. By now you are familiar with lots of commands (e.g., grep and sed) that are used to generate text and that, with the aid of redirection, can put that text into a file. You’ve even seen examples of using such commands to alter and produce program source code. In fact, as you move to more advanced forms of programming, you will find that programs and commands that generate part of your program’s source code are quite common.
-
The safest way to tell if a command is interactive or not is to learn just what that command does. Another possibility is to try running the command and see if it actually stops and asks you for any additional input via the keyboard or mouse.
- Most of the commands that we have studied in this course are not interactive, e.g.,
cp
,mv
,ls
,javac
– these do not pause and wait for additional input from you before completing your work. - The commands that we have seen that are definitely interactive would be the editors (
nano
,emacs
,vim
) andsftp
. - Commands that read from standard in might or might not be interactive, depending on how you use them in a particular project. For example, the command
sed s/e/x/g > substituted.txt
would be interactive, because it will pause and wait for you to type the input text from the keyboard. On the other hand, the command
sed s/e/x/g < original.txt > substituted.txt
would not be interactive, because it will take all of its input from the file
original.txt
.
- Most of the commands that we have studied in this course are not interactive, e.g.,
-
Finally, if you have no instructions at all on how to generate a particular text file, then odds are you are supposed to maintain that text via an (interactive) editor such as emacs.
-
- How can you tell if a command is interactive or not?
-
For each file in the constructed_ group, write a makefile rule with that file as the target, the Unix commands used to produce it as the command part of the rule, and any files that will be read by those commands listed as the dependencies of the rule.
-
If the goal of the project is to produce a single file (e.g., a program executable), the rule with that file as the target should appear first. The other rules can occur in any order.
-
-
(Optional) Add symbols, artificial targets, and other refinements as desired to simplify your makefile simpler or to make it more convenient to work with.
3.1 Creating a makefile: Example
Suppose that we are engaged in a project to produce a Java program, to be stored in a Jar file named pie.jar
.
pie.jar
will be created by compiling filesand combining all of these
Pie.java
to produce a compiled filePie.class
, andPieSlicer.java
to produce a compiled filePieSlicer.class
, andPieView.java
to produce a compiled filePieView.class
.class
files intopie.jar
.- The
.jar
file will be created using ajar
command.- The
.class
files will be created using ajavac
command.
Let’s step through our procedure for creating the makefile.
-
Make a list of all files that will be involved in your project.
The files involved will be:
pie.jar
,Pie.java
,PieSlicer.class
,PieSlicer.java
,Pie.class
,PieView.java
, andPieView.class
. -
Now, divide the files in your list into two groups, provided and constructed.
The
.class
files and the mainpie.jar
are all produced non-interactively.The rest of the files are all program source code, typically (though not always) produced via an editor. So we get
-
provided:
Pie.java
,PieSlicer.java
,PieView.java
-
constructed:
pie.jar
,PieSlicer.class
,Pie.class
, andPieView.class
.
-
-
For each file in the constructed group, write a makefile rule…
Because we have 4 files in the constructed group, we need to write 4 rules. For example, we will need a rule for
PieView.class
. That rule will havePieView.class
as its target, the command used to produce it (javac -g PieView.java
) in the command part of the rule, and any files that will be read by thejavac
command in the inputs/dependencies part of the rule. Obviously,PieView.java
is one of those inputs. But we will need to check to see ifPieView.class
depends on any ofthe other Java files.grep
commands are very useful in searching for dependencies. For example, the commandsgrep 'Pie ' *.java grep 'PieView' *.java grep 'PieSlicer' *.java
show us that the class
Pie
is used in all three Java files, the classPieView
is used inPieView.java
andPieSlicer.java
, but that the classPieSlicer
is only used inPieSlicer.java
.With that information we can write the rule for
PieView.class
:PieView.class: PieView.java Pie.class javac -g PieView.java
Continuing on like that, we eventually get a full set of six rules:
PieSlicer.class: PieSlicer.java PieView.class Pie.class javac -g PieSlicer.java Pie.class: Pie.java javac -g Pie.java PieView.class: PieView.java Pie.class javac -g PieView.java pie.jar: Pie.class PieSlicer.class PieView.class jar cfe pie.jar PieSlicer *.class
which, except possibly for the rule ordering, is the example that we saw at the beginning of this lesson.
-
Add symbols, artificial targets, etc.,.
Remember that, if you invoke
make
without giving an explicit target, then by default it builds the first target. In this case, that would beprogA
.We would probably prefer that, as a default, it built both programs in our project. So the single most useful thing we can add would be an artificial “all” target to encouraging building both programs:
all: pie.jar PieSlicer.class: PieSlicer.java PieView.class Pie.class javac -g PieSlicer.java Pie.class: Pie.java javac -g Pie.java PieView.class: PieView.java Pie.class javac -g PieView.java pie.jar: Pie.class PieSlicer.class PieView.class jar cfe pie.jar PieSlicer *.class
The next most useful thing would be a target to clean up afterwards:
all: pie.jar clean: rm *.jar *.class PieSlicer.class: PieSlicer.java PieView.class Pie.class javac -g PieSlicer.java Pie.class: Pie.java javac -g Pie.java PieView.class: PieView.java Pie.class javac -g PieView.java pie.jar: Pie.class PieSlicer.class PieView.class jar cfe pie.jar PieSlicer *.class
4 Is It Worth It?
Now, creating a makefile may seem like a lot of trouble the first time that you want to compile your program. The payoff comes while you are testing and debugging, and find yourself making changes to two or three files and then needing to recompile. Which files do you really need to recompile? It can be hard to remember some times, and the errors resulting from an incorrect guess may be hard to understand. make
eliminates this problem (as well as just being easier to type than a whole series of recompilation commands).
- This is why, when you give the
M-x compile
command inemacs
, the default compilation command is “make” rather than a direct use of any particular compiler. - It’s also why the
vim
command to do compilation is:make
.
That said, make
is something of a Swiss Army knife of build tools. You can put any Unix command into a make file, so you can use it to automate almost any process that does not reuqire human interaction.
There are other build tools out there, specifically oriented towards compiling code, sometimes towards compiling code of a specific programming languages. Because these have a more limited score, they can be easier to set up than a make file.
We’ll look briefly at a couple of these in the next lesson.