1.3. Project Management with Make

When you begin to develop projects that involve multiple files that need to be compiled or otherwise processed, keeping them all up-to-date can be a problem. Even more of a problem is passing them on to someone else (e.g., your instructor) and expecting them to know what to do to build your project from the source code.

The Unix program make is designed to simplify such project management. In a makefile, you record the steps necessary to build both the final file (e.g., your executable program) and each intermediate file (e.g., the .o files produced by compiling a single source code file).

We say that a file file1 depends upon a second file file2 if the file2 is used as input to some command used to produce file1.

When the make program is run, it then checks to be sure that all of the needed files exist, and that each needed file has been updated more recently than all of the files it depends upon. The key bits of information in a makefile, therefore are

A makefile may also include various macros/abbreviations designed to simplify the task of dealing with many instances of the same commands or files.

Suppose that we are engaged in a project to produce 2 programs, progA and progB. progA is produced by compiling files utilities.c, progA1.cpp, and progA2.cpp and linking together the resulting .o files. Program progB is produced by compiling file utilities.c and progB1.cpp and linking together the resulting .o files. All of the .c and .cpp files have #include statements for a file utilities.h. Also, both of the .cpp files have an #include statement for a file progA1.h.

Here is a makefile for this project. This file should reside in the project directory, and should be called Makefile or makefile .

progA: utilities.o progA1.o progA2.o
        g++ -g -DDEBUG utilities.o progA1.o progA2.o
        mv a.out progA
  
progB: utilities.o progB1.o
        g++ -g -DDEBUG utilities.o progB1.o
        mv a.out progB
  
utilities.o: utilities.c utilities.h
        g++ -g -DDEBUG -c utilities.c
  
progA1.o: progA1.cpp utilities.h progA1.h
        g++ -g -DDEBUG -c progA1.cpp
  
progA2.o: progA2.cpp utilities.h progA1.h
        g++ -g -DDEBUG -c progA2.cpp
  
progB1.o: progB1.cpp
        g++ -g -DDEBUG -c progB1.cpp

The key information in a makefile consists of a variety of rules for producing target files. Each target rule begins with a single line containing the name of the file to produce, a colon, and then a list of all files that serve as inputs to the commands that produce the file. Following that are any number of command lines that give the Unix commands to actually produce the file. Each command line starts with a Tab character (invisible in this listing).[43]

Suppose that, with just this Makefile and the various source code files in your directory, you issued the command

make progB

make reads the Makefile and finds the rule for creating the file progB:

progB:1 utilities.o progB1.o2
        g++ -g -DDEBUG utilities.o progB1.o 3
        mv a.out progB

1

We (and the make command) know that this rule tells how to create the file progB because progB is listed as the target, to the left of the colon. You and I know, from the initial discussion, that progB is actually a program, but make does not know that and doesn't care about that. make regards its task to be the creation of files, and it does not care what is in those files.

2

From the dependency list to the right of the colon, make discovers that, in order to create progB, it will first need up-to-date copies of utilities.o and progB1.o.

3

make also learns that, once it has up-to-date copies of utilities.o and progB1.o, that it can then create progB by running the commands

g++ -g -DDEBUG utilities.o progB1.o
mv a.out progB

You can put any command lines into the commands section of a make rule that you can actually execute in Unix. This includes both basic commands that we have studied previously (e.g., mv), invocations of the compiler (e.g., g++), or any other program (including ones that you yourself might have written).

In this case, make should realize that it cannot execute these two commands immediately because it does not have up-to-date copies of utilities.o and progB1.o. In fact, neither of these files exists. Therefore make sets out to create them, by looking for the appropriate rules for each of them.

utilities.o depends upon utilities.c and utilities.h. Since these files exist and do not themselves depend upon anything else, make will issue the command to create utilities.o from them. This command is the standard command for making a .o file from a .c file:

   gcc -g -DDEBUG -c utilities.c

Next make looks at progB1.o. It depends upon progB1.cpp which exists and does not depend upon anything else. So make uses the standard command for C++ files:

   g++ -g -DDEBUG -c progB1.cpp

Now that both .o files have been created, make proceeds to build its main target, progB, using the command lines provided for that purpose:

   g++ -g -DDEBUG utilities.o progB1.o

and the progB program has been created.

Now suppose that we immediately give the command

make progA

(or just make , since by default make builds the first target when none is explicitly given). Then the following commands would be performed:

   g++ -g -DDEBUG -c progA1.cpp
   g++ -g -DDEBUG -c progA2.cpp
   g++ -g -DDEBUG utilities.o progA1.o progA2.o
   mv a.out progA

Note that utilities.c is not recompiled, because make would notice that utilities.o already exists and was created more recently than the last time when either utilities.c or utilities.h was changed.

If you want to test your makefile without actually performing the commands, add a -n option to your command (e.g., make -n progB) and make will simply list the commands it would issue without actually doing any of them.

Symbols

Thinking ahead, we might realize that we won't always want to compile with the flags -g -DDEBUG (the significance of which will be introduced in the debugging section).

We can make our makefile more flexible by gathering things that might need to be changed later into a symbol. make allows us to define symbols like this

SymbolName=string

and later use that symbol like this: $(SymbolName).

So we could modify our makefile as follows:

# Macro definitions for "standard" language compilations
# 
#  First, define special compilation flags. These may change when
#  we're done testing and debugging.
CPPFLAGS=-g -DDEBUG
# 

# 
# Targets:
# 
progA: utilities.o progA1.o progA2.o
        g++ $(CPPFLAGS) utilities.o progA1.o progA2.o
        mv a.out progA
  
progB: utilities.o progB1.o
        g++ $(CPPFLAGS) utilities.o progB1.o
        mv a.out progB
  
utilities.o: utilities.c utilities.h
        g++ $(CPPFLAGS) -c utilities.c
  
progA1.o: progA1.cpp utilities.h progA1.h
        g++ $(CPPFLAGS) -c progA1.cpp
  
progA2.o: progA2.cpp utilities.h progA1.h
        g++ $(CPPFLAGS) -c progA2.cpp
  
progB1.o: progB1.cpp
        g++ $(CPPFLAGS) -c progB1.cpp

Default Rules

Makefiles can be simplified by introducing default rules for forming one kind of file from another. Here is an equivalent makefile that defines appropriate default rules:

# Macro definitions for "standard" language compilations
# 
#  First, define special compilation flags. These may change when
#  we're done testing and debugging.
CPPFLAGS=-g -DDEBUG
# 
#  The following is "boilerplate" to set up the standard compilation
#  commands:
.SUFFIXES:
.SUFFIXES: .cpp .c .cpp .h .o
.c.o: ; gcc $(CPPFLAGS) -c $*.c
.cpp.o: ; g++ $(CPPFLAGS) -c $*.cpp
# 
# Targets:
# 
progA: utilities.o progA1.o progA2.o
        g++ $(CPPFLAGS) utilities.o progA1.o progA2.o
        mv a.out progA
  
progB: utilities.o progB1.o
        g++ $(CPPFLAGS) utilities.o progB1.o
        mv a.out progB
  
utilities.o: utilities.c utilities.h
  
progA1.o: progA1.cpp utilities.h progA1.h
  
progA2.o: progA2.cpp utilities.h progA1.h
  
progB1.o: progB1.cpp

In the SUFFIXES area, standard commands are defined for producing a .o file from a .c or .cpp file. Of course these standard commands simply invoke the C or C++ compilers. Command lines are not needed if the standard commands from the Suffixes area can be used to build the desired file.

Common Conventions

So far, we have talked about using make exclusively with compilation. But a makefile can control almost any sequence of operations that build one kind of file out of others.

Certain conventions have arisen that you will find useful in designing your own makefiles and in using the makefiles of others. Most of these involve certain artificial targets that you can use when issuing the make command.

These are just conventions. They don't happen automatically, but most people who design makefiles set them up to work this way.

all

The target all compiles and builds everything (except, sometimes, documentation). So one of the more common ways to invoke make is to say

make all

The all target rule, if it is present, is always given as the first rule in the makefile. The make command, if not given any target, always goes to the first target rule. So, in most makefiles,

make

will also compile and build everything.

clean

The command make clean is often used to clean up a directory, deleting all the temporary files produced by make all, leaving only the original files (e.g., the source code) from which everything else can be rebuilt later, if desired.

install

In programs that must be installed by placing them in special directories, this target controls the commands necessary to do that installation.

A common sequence for building and installing new Unix software is therefore:

make
make install
make clean

test

Less common, runs a test suite to see if the program built successfully.

doc

(or docs) May be used to build program documentation, user manuals, etc.

We can bring our sample makefile into conformity with these conventions as follows:

# Macro definitions for "standard" language compilations
# 
#  First, define special compilation flags. These may change when
#  we're done testing and debugging.
CPPFLAGS=-g -DDEBUG
# 
#  The following is "boilerplate" to set up the standard compilation
#  commands:
.SUFFIXES:
.SUFFIXES: .cpp .c .cpp .h .o
.c.o: ; gcc $(CPPFLAGS) -c $*.c
.cpp.o: ; g++ $(CPPFLAGS) -c $*.cpp
# 
# Targets:
# 
all: progA progB

clean:
	rm progA progB *.o

progA: utilities.o progA1.o progA2.o
        g++ $(CPPFLAGS) utilities.o progA1.o progA2.o
        mv a.out progA
  
progB: utilities.o progB1.o
        g++ $(CPPFLAGS) utilities.o progB1.o
        mv a.out progB
  
utilities.o: utilities.c utilities.h
  
progA1.o: progA1.cpp utilities.h progA1.h
  
progA2.o: progA2.cpp utilities.h progA1.h
  
progB1.o: progB1.cpp

Creating a Makefile

When you are faced with the task of writing your own makefiles, the best way to start is to

  1. Make a list of all files that will be involved in your project. This includes the file(s) that constitute the overall goal of the project, the files that you yourself will create manually using emacs or some other text editor, and all the intermediate files that will be produced by the build commands.

    If you're not sure what those intermediate files will be, write out a list of the commands that you would use to build the project if you were typing those commands, one at a time, into a command shell. Avoid compilation commands that perform multiple compile and link steps via a single command. Think instead of compiling each source code file separately and then separately linking the object files together. Every file mentioned in any of these commands should probably be in your list.

  2. Now, divide the files in your list into two groups: the first group is the files that you create via a text editor or other interactive program. The second group are those files that will be created by running a non-interactive Unix command using other files (from either or both groups) as input.

  3. For each file in the second group, write a makefile rule with that file as the target, the Unix command used to produce it as the command part of the rule, and any files that will be read by that command listed as the inputs to the rule.

  4. Add symbols, artificial targets, and other refinements as desired to simplify your makefile simpler or to make it more convenient to work with.

Creating a makefile: Example

Suppose that we are engaged in a project to produce 2 programs, progA and progB. progA is produced by compiling files utilities.c, progA1.cpp, and progA2.cpp and linking together the resulting .o files. Program progB is produced by compiling file utilities.c and progB1.cpp and linking together the resulting .o files. All of the .c and .cpp files have #include statements for a file utilities.h. Also, both of the .cpp files have an #include statement for a file progA1.h.

Let's step through our procedure for creating the makefile.

  1. Make a list of all files that will be involved in your project.

    The files involved will be: progA, progB, utilities.c, progA1.cpp, progA2.cpp, utilities.o, progA1.o, progA2.o, progB1.cpp, progB1.cpp, utilities.h, progA1.h.

  2. Now, divide the files in your list into two groups: files created interactively and files created via non-interactive commands.

    The .o files and the main programs are all produced non-interactively. The rest of the files are all program source code, typically (though not always) produced via an editor. So we get

    • Group 1: utilities.c, progA1.cpp, progA2.cpp, progB1.cpp, progA1.h, utilities.h.

    • Group 2: progA, progB, utilities.o, progA1.o, progA2.o.

  3. For each file in the second group, write a makefile rule...

    Since we have 5 files in group 2, we need to write 5 rules. For example, we will need a rule for progA1.o. That rule will have progA1.o as its target, the command used to produce it (g++ -g -c progA1.cpp) in the command part of the rule, and any files that will be read by the g++ command in the inputs/dependencies part of the rule. Obviously, progA1.cpp is one of those inputs. But the description above tells us that progA1.cpp includes progA1.h and utilities.h, so those will also be read when we compile and need to listed in the rule:

    progA1.o: progA1.cpp utilities.h progA1.h
            g++ -g -DDEBUG -c progA1.cpp
    

    Continuing on like that, we eventually get a full set of 5 rules:

    progA: utilities.o progA1.o progA2.o
            g++ -g -DDEBUG utilities.o progA1.o progA2.o
            mv a.out progA
      
    progB: utilities.o progB1.o
            g++ -g -DDEBUG utilities.o progB1.o
            mv a.out progB
      
    utilities.o: utilities.c utilities.h
            g++ -g -DDEBUG -c utilities.c
      
    progA1.o: progA1.cpp utilities.h progA1.h
            g++ -g -DDEBUG -c progA1.cpp
      
    progA2.o: progA2.cpp utilities.h progA1.h
            g++ -g -DDEBUG -c progA2.cpp
      
    progB1.o: progB1.cpp
            g++ -g -DDEBUG -c progB1.cpp
    

    which is the example that we saw at the beginning of this lesson.

  4. Add symbols, artificial targets, etc.,.

    Remember that, if you invoke make without giving an explicit target, then by default it builds the first target. In this case, that would be progA.

    We would probably prefer that, as a default, it built both programs in our project. So the single most useful thing we can add would be an artificial "all" target to encouraging building both programs:

    all: progA progB
    
    progA: utilities.o progA1.o progA2.o
            g++ -g -DDEBUG utilities.o progA1.o progA2.o
            mv a.out progA
      
    progB: utilities.o progB1.o
            g++ -g -DDEBUG utilities.o progB1.o
            mv a.out progB
      
    utilities.o: utilities.c utilities.h
            g++ -g -DDEBUG -c utilities.c
      
    progA1.o: progA1.cpp utilities.h progA1.h
            g++ -g -DDEBUG -c progA1.cpp
      
    progA2.o: progA2.cpp utilities.h progA1.h
            g++ -g -DDEBUG -c progA2.cpp
      
    progB1.o: progB1.cpp
            g++ -g -DDEBUG -c progB1.cpp
    

Is It Worth It?

Now, creating a makefile may seem like a lot of trouble the first time that you want to compile your program. The payoff comes while you are testing and debugging, and find yourself making changes to two or three files and then needing to recompile. Which files do you really need to recompile? It can be hard to remember some times, and the errors resulting from an incorrect guess may be hard to understand. make eliminates this problem (as well as just being easier to type than a whole series of recompilation commands). (This is why, when you give the M-x compile command in emacs, the default compilation command is make rather than a direct use of any particular compiler.)

Most of the details of generating a makefile can be automated. The gcc/g++ compiler, for example, will actually write out the makefile rules that would determine when a given .c or .cpp file needs to be recompiled. I've used this idea in this self-constructing Makefile. To use it, copy it into your working directory where you keep the source code files for any single program. Your copy must be named Makefile. Edit your copy of the file to supply the appropriate program name, list of source code files needed for that program, and to indicate whether the final step (linking) should be done with the C (gcc) or C++ (g++) compiler.

Now you can compile your program by saying make, and clean up afterwards with make clean.

As you continue to work with your code, just remember to keep the OBJS list in the Makefile up to date.



[43] A common mistake in preparing makefiles is to use ordinary spaces instead of a tab character in front of these command lines. The usual result of this mistake is the error message

  Makefile:N *** missing separator
where N is the approximate line number where the error occurs. The "separator" here refers to the fact that make expects each rule to be separated from the others by one or more empty lines. A line that starts with a space (instead of a tab) is assumed to be a new rule. Since command lines are not separated from the rest of the rule, a command line starting with a blank instead of a tab appears to make as a new rule starting up without an empty line separating it from the previous rule.