Build Management with Make

Steven Zeil

Last modified: Dec 19, 2023
Contents:

When you begin to develop projects that involve multiple files that need to be compiled or otherwise processed, keeping them all up-to-date can be a problem. Even more of a problem is passing them on to someone else (e.g., your instructor) and expecting them to know what to do to build your project from the source code.

The Unix program make is designed to simplify such project build management. In a makefile, you record the steps necessary to build both the final file (e.g., your executable program) and each intermediate file (e.g., the .o files produced by compiling a single C++ source code file).

We say that a file file1 depends upon a second file file2 if the file2 is used as input to some command used to produce file1.

When the make program is run, it then checks to be sure that all of the needed files exist, and that each needed file has been updated more recently than all of the files it depends upon.

1 Makefiles

The key bits of information in a makefile, therefore are

A makefile may also include various macros/abbreviations designed to simplify the task of dealing with many instances of the same commands or files.

Suppose that we are engaged in a project to produce a Java program, to be stored in a Jar file named pie.jar.

By the definition above, we can say that

However, that’s not all of the dependencies to be considered. If we look through the Java source code, we can observe that PieSlicer.java contains mentions of the Pie class and of the PieView class, and that PieView.java contains mentions of the Pie class.

Remember that when a file of Java source code is being compiled, if that source code mentions another class, e.g., Pie, then the compiler looks for an already compiled Pie.class file and reads that for information about that class.

So we conclude that:

 

Here is a makefile for this project. This file should reside in the project directory, and should be called “Makefile” or “makefile”.

pie.jar: Pie.class PieSlicer.class PieView.class
        jar cfe pie.jar PieSlicer *.class

PieSlicer.class: PieSlicer.java PieView.class Pie.class
        javac -g PieSlicer.java

Pie.class: Pie.java
        javac -g Pie.java

PieView.class: PieView.java Pie.class
        javac -g PieView.java

1.1 Rules

The key information in a makefile consists of a variety of rules for producing “target” files. Rules have the form:

targetFile: inputFile1 inputFile2 ...
    command1  
    command2  
      ⋮
 
  1. Each target rule begins with a single line containing the name of the file to produce, a colon, and then a list of all files that serve as inputs to the commands that produce the file.

  2. Following that are any number of command lines that give the Unix commands to actually produce the file.

    Each command line starts with a “Tab” character (invisible in this listing).

Let’s take a good look at some of the rules in our earlier example:

pie.jar: Pie.class PieSlicer.class PieView.class
        jar cfe pie.jar PieSlicer *.class

This rule says that the file pie.jar is created by the command

jar cfe pie.jar PieSlicer *.class

and that the inputs to those commands are the files Pie.class, PieSlicer.class, and PieView.class.

(A subtle point about compiling Java code. When you compile a source file foo.java, you will get a new foo.class file, but you may also get additional .class files with names like foo$bar.class. We need all of these class files to go into the jar. That is why the jar command uses the wildcard *.class.)


PieView.class: PieView.java Pie.class
        javac -g PieView.java

This rule says that the file PieView.class is produced by running the command

javac -g PieView.java

and that the inputs to this command are PieView.java, and Pie.class.

All of the rules in our sample make file need only a single Unix command to generate the target file. That’s common, but not required.

If we wanted to, we could have written the first rule as

pie.jar: Pie.class PieSlicer.class PieView.class
        jar cfe wrongName.jar PieSlicer *.class
        mv wrongName.jar pie.jar

It would be a bit silly, but it’s legal.

The introduction of the now-familiar mv command also highlights something else about make files. You can put pretty much any Unix command into a make file – you aren’t limited to “compilation” commands. Generally, you won’t want to put interactive commands (commands that pause and wait for human input, e.g. editors) into a make file. It’s not illegal to do so, but since the point of a make file is automate a procedure, interactive commands would usually interfere with the automation.

To write a makefile rule, it is important that you understand the commands that you need to run and know what all their actual inputs are, not just the inputs that are explicitly listed in the command line.

1.2 Running make

Suppose that, with just the above Makefile and the various source code files in your directory, you issued the command

make pie.jar

make reads the Makefile and finds the rule for creating the file pie.jar:

pie.jar:➀ Pie.class PieSlicer.class PieView.class ➁
        jar cfe pie.jar PieSlicer *.class         ➂

In this case, make should realize that it cannot execute the jar command immediately because it does not have up-to-date copies of the required class files. In fact, none of these files exists. Therefore make sets out to create them, by looking for the appropriate rules for each of them.

It will find the rules

PieSlicer.class: PieSlicer.java PieView.class Pie.class
        javac -g PieSlicer.java

Pie.class: Pie.java
        javac -g Pie.java

PieView.class: PieView.java Pie.class
        javac -g PieView.java

and will consider running the javac commands to create the .class files.

However, looking at the dependencies, make will decide that it cannot immediately compile PieSlicer.java because that needs the files PieView.class and Pie.class, which do not exist yet. And it can’t immediately compile PieView.java because that needs the file Pie.class. So make eventually concludes that the only sensible order of commands is

javac -g Pie.java
javac -g PieView.java
javac -g PieSlicer.java

1.2.1 The order of rules in a makefile…

…does not matter. make uses the information we have provided on the dependencies to figure out what order the commands need to be run in.

1.2.2 …with one exception…

If we run the command

make

without naming a target, make will try to generate the target of the very first rule in the make file.

By convention, we always put the “main” target of our process in the first rule of the make file.

1.2.3 Trying It Out

Example 1: Try This:
  1. In an xterm, set up a directory for this exercise:

    mkdir ~/playing/withMake
    cd ~/playing/withMake
    
  2. Copy the files from ~cs252/Assignments/makePie into this directory.

    cp ~cs252/Assignments/makePie/* .
    
  3. Use ls and more to read through the files that you have copied. You’ll see a copy of the makefile we have just been looking at, together with a collection of Java source code files.

  4. Give the command

    make Pie.class
    ls
    

    Look closely at the commands run by make.

    make has done exactly what we asked it to do.

  5. Give the command

    make pie.jar
    

    Look closely at the commands being run. Notice that make does not bother recompiling Pie.java. It can se that it already has an up-to-date copy of Pie.class.

    Do an ls and take note of the new files that have appeared.

    Run the program with the command

    java -jar pie.jar
    
  6. Give the command

    make pie.jar
    

    Look closely at the commands being run, or, more precisely, the commands that are not run.

    make is smart enough to realize that it doesn’t need to do anything to produce files that are already up-to-date.

  7. Give the command

    rm pie.jar
    make pie.jar
    

    Look closely at the commands being run and the commands that are not run.

    make realizes that it still has up-to-date versions of the .class files and does not need to recompile the source code.

  8. Give the commands

    rm *.class *.jar
    ls
    make
    ls
    

    Because we removed the jar and its class components, make rebuilds everything for us.

    How did it know we wanted to get a new pie.jar? We did not give a target to the make command, so by default it build the target of the first rule in our make file.

  9. Give the command

    make  doesNotExist.dat
    

    Take note of the response that you get.

    This is a pretty common response from make and, to me, seems pretty self-explanatory. But some people seem to be very mystified when they see this response.

  10. You can “test” a makefile without actually performing the commands with the -n option.

    Run:

    rm *.jar *.class
    ls
    make -n
    ls
    

    Notice that, although make listed the commands to build your Java jar, it has not actually executed any of them.

1.3 make Tries to be Smart

One of the most important behaviors of make is that it is smart enough to only do as much work as is needed. That may not seem important to you right now, working with programs that can be compiled in a second or less, but large programs may have hundreds of source code files that take many minutes or even hours to compile.

If you are trying to modify or debug a program like that, you are probably only modifying one or two files at a time. Typically the only steps needed to rebuild the program will be to compile the changed files and then link all of the .o files to produce the final executable.

With a proper makefile, make will be smart enough to do that. That’s because each file has a “last modified date” (which you can view using ls -l). make looks at the “last modified” dates on files to see if they have been changed since the last time that make was run.

Example 2: Try This: Small Changes
  1. Return to your make project:

    cd ~/playing/withMake
    
  2. Give the command

    make
    

    just to be sure that the program is already built.

  3. Edit PieView.java by adding a Java comment.

    Give the command

    make
    

    again. Look closely at the commands being run.

    Notice that make is smart enough to re-compile the file we have edited, but knows that it doesn’t have to recompile Pie.java because it is unaffected by (does not depend on) PieView.

    make does recompile PieSlicer.java because we told it that PieSlicer depends on PieView. Now, as it happens, all that you did was to add a comment, which would not really change the compilation of PieSlicer, but make doesn’t know that the change you made was trivial. It only looks at the date and time at which each file was last modified.

  4. When testing out makefiles, we can simulate making changes to source code files via the touch command. touch does not actually change the contents of the file, but it does reset the “last modified” time on the file to make it appear that the file has been changed.

    Give the commands:

    make
    touch PieView.java
    make
    

    You should see, before the touch, ‘make’ does not see a need to recompile anything, but, after the touch, make carries out the same steps as when you actually edited the file.

  5. Let’s try a different modification. Simulate the modification of PieSlicer.java:

     touch PieSlicer.java
     make
    

    Do the steps performed by make seem reasonable?

2 More Advanced Makefiles

2.1 Symbols

Thinking ahead, we might realize that we won’t always want to compile with the flags “-g”.

We can make our makefile more flexible by gathering things that might need to be changed later into a symbol. make allows us to define symbols like this

SymbolName=string

and later use that symbol like this: $(SymbolName).

So we could modify our makefile as follows:

# Compiler settings
JAVAFLAGS=-g

pie.jar: Pie.class PieSlicer.class PieView.class
        jar cfe pie.jar PieSlicer *.class

PieSlicer.class: PieSlicer.java PieView.class Pie.class
        javac ${JAVAFLAGS} PieSlicer.java

Pie.class: Pie.java
        javac ${JAVAFLAGS} Pie.java

PieView.class: PieView.java Pie.class
        javac ${JAVAFLAGS} PieView.java

2.2 Default (Pattern) Rules

Make files can be simplified by introducing default rules for forming one kind of file from another. Here is an equivalent make file that defines appropriate default rules:

# Compiler settings
JAVAFLAGS=-g

# Pattern for compiling Java code
%.class: %.java
	javac ${JAVAFLAGS} $*.java

pie.jar: Pie.class PieSlicer.class PieView.class
	jar cfe pie.jar PieSlicer *.class

Pie.class: Pie.java

PieView.class: PieView.java Pie.class

PieSlicer.class: PieSlicer.java PieView.class Pie.class

pie.jar: Pie.class PieSlicer.class PieView.class
        jar cfe pie.jar PieSlicer *.class

Notice, first, that the individual javac commands have been removed from the rules for generating the various class files. Instead, make will use the command given in the new pattern rule (highlighted).

The pattern rule uses “%” as a wildcard character that stands for any string of characters.

So the target pattern %.class will match any class file.

The same “%” can appear once in the dependencies list. So the dependency pattern %.java will match any Java file with the same base file name as a desired class file.

The $* in the rule inserts whatever string was matched by the “%”.

So, to generate PieSlicer.class, make will

2.3 Artificial Targets

An artificial target in a rule is a “file name” used as a target in a rule but that is never actually created by the commands in the rule. These can be used as a kind of shorthand for commands that we want to run frequently and so set up in the makefile.

There are some very commonly used artificial targets. These are just conventions. They don’t happen automatically, but most people who design makefiles set them up to work this way.

all
The target all compiles and builds everything (except, sometimes, documentation). So one of the more common ways to invoke make is to say
make all

The all target rule, if it is present, is always given as the first rule in the makefile. So, in most makefiles,

make

will also compile and build everything.

clean
The command make clean is often used to clean up a directory, deleting all the temporary files produced by make all, leaving only the original files (e.g., the source code) from which everything else can be rebuilt later, if desired.
install
In programs that must be “installed” by placing them in special directories, this target controls the commands necessary to do that installation.

A common sequence for building and installing new Unix software is therefore:

make
make install
make clean
test
Less common, runs a test suite to see if the program built successfully.
doc
(or docs) May be used to build program documentation, user manuals, etc.

We can bring our sample makefile into conformity with these conventions as follows:

# Compiler settings
JAVAFLAGS=-g

# Pattern for compiling Java code
%.class: %.java
	javac ${JAVAFLAGS} $*.java

all: pie.jar

pie.jar: Pie.class PieSlicer.class PieView.class
	jar cfe pie.jar PieSlicer *.class

Pie.class: Pie.java

PieView.class: PieView.java Pie.class

PieSlicer.class: PieSlicer.java PieView.class Pie.class

pie.jar: Pie.class PieSlicer.class PieView.class
        jar cfe pie.jar PieSlicer *.class

clean:
        rm *.jar *.class
Example 3: Try This:
  1. Edit the makefile from the prior Try This to match the listing above.

    Don’t forget to use tab characters and not spaces in front of the indented commands.

  2. In that makeTry directory, give the commands:

    make
    ls
    make clean
    ls
    make all
    ls
    make clean
    ls
    make
    ls
    

    observing which commands are issued and which files are created at each step.

3 Creating a Makefile

When you are faced with the task of writing your own makefiles, the best way to start is to

  1. Make a list of all files that will be involved in your project. This includes the file(s) that constitute the overall “goal” of the project, the files that you yourself will create “manually” using emacs or some other text editor, and all the intermediate files that will be produced by the build commands. Do not include the makefile itself in this list.

    If you’re not sure what those intermediate files will be, write out a list of the commands that you would use to build the project if you were typing those commands, one at a time, into a command shell. Avoid compilation commands that perform multiple compile and link steps via a single command. Think instead of compiling each source code file separately and then separately linking the object files together. Every file mentioned in any of these commands should probably be in your list.

  2. Now, divide the files in your list into two groups.

    The provided group lists the files that you create via a text editor or other interactive program or that you receive from outside the project and don’t need to create at all.

    The constructed group lists those files that will be created by running a non-interactive Unix command using other files (from either or both groups) as input.

    • How can you tell if a command is interactive or not?
      • It’s not enough to look and see if the file contains text, or even if it looks like program source code. By now you are familiar with lots of commands (e.g., grep and sed) that are used to generate text and that, with the aid of redirection, can put that text into a file. You’ve even seen examples of using such commands to alter and produce program source code. In fact, as you move to more advanced forms of programming, you will find that programs and commands that generate part of your program’s source code are quite common.

      • The safest way to tell if a command is interactive or not is to learn just what that command does. Another possibility is to try running the command and see if it actually stops and asks you for any additional input via the keyboard or mouse.

        • Most of the commands that we have studied in this course are not interactive, e.g., cp, mv, ls, javac – these do not pause and wait for additional input from you before completing your work.
        • The commands that we have seen that are definitely interactive would be the editors (nano, emacs, vim) and sftp.
        • Commands that read from standard in might or might not be interactive, depending on how you use them in a particular project. For example, the command

          sed s/e/x/g > substituted.txt

          would be interactive, because it will pause and wait for you to type the input text from the keyboard. On the other hand, the command

          sed s/e/x/g < original.txt > substituted.txt

          would not be interactive, because it will take all of its input from the file original.txt.

      • Finally, if you have no instructions at all on how to generate a particular text file, then odds are you are supposed to maintain that text via an (interactive) editor such as emacs.

  3. For each file in the constructed_ group, write a makefile rule with that file as the target, the Unix commands used to produce it as the command part of the rule, and any files that will be read by those commands listed as the dependencies of the rule.

  4. (Optional) Add symbols, artificial targets, and other refinements as desired to simplify your makefile simpler or to make it more convenient to work with.

3.1 Creating a makefile: Example

Suppose that we are engaged in a project to produce a Java program, to be stored in a Jar file named pie.jar.

  • pie.jar will be created by compiling files
    • Pie.java to produce a compiled file Pie.class, and
    • PieSlicer.java to produce a compiled file PieSlicer.class, and
    • PieView.java to produce a compiled file PieView.class
    and combining all of these .class files into pie.jar.
  • The .jar file will be created using a jar command.
  • The .class files will be created using a javac command.

Let’s step through our procedure for creating the makefile.

  1. Make a list of all files that will be involved in your project.

    The files involved will be: pie.jar, Pie.java, PieSlicer.class, PieSlicer.java, Pie.class, PieView.java, and PieView.class.

  2. Now, divide the files in your list into two groups, provided and constructed.

    The .class files and the main pie.jar are all produced non-interactively.

    The rest of the files are all program source code, typically (though not always) produced via an editor. So we get

    • provided: Pie.java, PieSlicer.java, PieView.java

    • constructed: pie.jar, PieSlicer.class, Pie.class, and PieView.class.

  3. For each file in the constructed group, write a makefile rule…

    Because we have 4 files in the constructed group, we need to write 4 rules. For example, we will need a rule for PieView.class. That rule will have PieView.class as its target, the command used to produce it (javac -g PieView.java) in the command part of the rule, and any files that will be read by the javac command in the inputs/dependencies part of the rule. Obviously, PieView.java is one of those inputs. But we will need to check to see if PieView.class depends on any ofthe other Java files.

    grep commands are very useful in searching for dependencies. For example, the commands

    grep 'Pie ' *.java
    grep 'PieView' *.java
    grep 'PieSlicer' *.java
    

    show us that the class Pie is used in all three Java files, the class PieView is used in PieView.java and PieSlicer.java, but that the class PieSlicer is only used in PieSlicer.java.

    With that information we can write the rule for PieView.class:

    PieView.class: PieView.java Pie.class
            javac -g PieView.java
    

    Continuing on like that, we eventually get a full set of six rules:

    PieSlicer.class: PieSlicer.java PieView.class Pie.class
            javac -g PieSlicer.java
    
    Pie.class: Pie.java
            javac -g Pie.java
    
    PieView.class: PieView.java Pie.class
            javac -g PieView.java
    
    pie.jar: Pie.class PieSlicer.class PieView.class
            jar cfe pie.jar PieSlicer *.class
    
    

    which, except possibly for the rule ordering, is the example that we saw at the beginning of this lesson.

  4. Add symbols, artificial targets, etc.,.

    Remember that, if you invoke make without giving an explicit target, then by default it builds the first target. In this case, that would be progA.

    We would probably prefer that, as a default, it built both programs in our project. So the single most useful thing we can add would be an artificial “all” target to encouraging building both programs:

    all: pie.jar
    
    PieSlicer.class: PieSlicer.java PieView.class Pie.class
            javac -g PieSlicer.java
    
    Pie.class: Pie.java
            javac -g Pie.java
    
    PieView.class: PieView.java Pie.class
            javac -g PieView.java
    
    pie.jar: Pie.class PieSlicer.class PieView.class
            jar cfe pie.jar PieSlicer *.class
    
    

    The next most useful thing would be a target to clean up afterwards:

    all: pie.jar
    
    clean:
           rm *.jar *.class
    
    PieSlicer.class: PieSlicer.java PieView.class Pie.class
            javac -g PieSlicer.java
    
    Pie.class: Pie.java
            javac -g Pie.java
    
    PieView.class: PieView.java Pie.class
            javac -g PieView.java
    
    pie.jar: Pie.class PieSlicer.class PieView.class
            jar cfe pie.jar PieSlicer *.class
    
    

4 Is It Worth It?

Now, creating a makefile may seem like a lot of trouble the first time that you want to compile your program. The payoff comes while you are testing and debugging, and find yourself making changes to two or three files and then needing to recompile. Which files do you really need to recompile? It can be hard to remember some times, and the errors resulting from an incorrect guess may be hard to understand. make eliminates this problem (as well as just being easier to type than a whole series of recompilation commands).

That said, make is something of a Swiss Army knife of build tools. You can put any Unix command into a make file, so you can use it to automate almost any process that does not reuqire human interaction.

There are other build tools out there, specifically oriented towards compiling code, sometimes towards compiling code of a specific programming languages. Because these have a more limited score, they can be easier to set up than a make file.

We’ll look briefly at a couple of these in the next lesson.