1 Structural Architecture of a Development Project

Let’s talk about how development projects are typically organized into files, directories, etc.

1.1 Projects and Sub-projects

A project consists of one or more more sub-projects.

• Even if we conceive of a project as a single entity, it’s worth treating it as having a single sub-project
  • A hedge against changing our minds later.
  • Sets a standard for consistent tool use and setup

What Constitutes a Sub-project?

A sub-project is generally defined as the code and data that yields a single deliverable.

Examples of deliverables include

• an executable program
• a library
  • Java .jar or
  • C/C++ .a, .lib, .so, or .dll
• a reference manual

1.2 The Project Directory

Typically contains

• Top-level project documentation.
• Overall project build & configuration information
• One sub-directory per sub-project
• Version control and configuration management information

1.2.1 Example: AlgAE

Top-level directory contains:

• README.md, LICENSE.md
• build.xml: builds the sub-projects in the correct order
• .git, .gitignore, ivysettings.xml (version control and configuration management)
• Directories:
  • algae-client-server
  • algae-cppserver
  • algae-reference-manual
  • demos
  • Reports

  All but the last are sub-projects.

  C/C++ projects might add directories that will (after the project is built) contain the various sub-projects’ deliverables:

  • bin
  • lib

1.3 Sub-Project Directory (Java)

Contains:

• sub-project build & configuration
• source code directory
• project and test data
• target directory for deliverables and other build projects
  • When source code is compiled, output is placed here rather than in the source code directory

1.3.1 Apache Project Directories

The Apache Foundation hosts many open source projects, which organize their projects & sub-projects like this:

src/     # anything supplied/edited by the programmers
target/  # initially empty, holds products of the compilation/build

The src/ directory is split into separate directories for the "real’ code and for the test code.

src/
|  main/   # things that contribute directly to the deliverable
|  test/   # things used for testing but not delivered
target/

“Deliverables” are usually an archive of some kind.

1.3.2 Android/Gradle Project Directories

1.4 Sub-Project Directory (C/C++)

Much more variation exists. One possibility is:

include/ # header files
|
src/ # compilation units (.c and .cpp files)
|
bin/ # executables and .o files produced by compiling src/
|
lib/ # libraries produced by combining object files

• Sometimes .o files are placed in a separate obj/ directory.
• Sometimes executables and libraries are copied directly to a project-level bin/ or lib/ directory.

1.4.1 Android-ish structure

Increasingly common is this approach, inspired by the Apache/Android Java styles:

src/
|  main/
|  |  cpp/       # C++ source code. .cpp files and local headers
|  |  headers/   # Header (.h) files that need to be visible to main code and
|  |             #   to tests.
|  |  public/    # For library projects, the header files that will be exported
|  |             #   as part of the delivered library.
|  test/
|  |  cpp/       # Unit test code
|  |  data/      # test data
build/
|  exe/         # Executables
|  |  main/     #   - from main/cpp
|  |  test/     #   - from test/cpp
|  lib/         # libraries constructed from object code
|  |  main/     #   - from obl/main
|  obj/         # Compiled object code
|  |  main/     #   - from main/cpp
|  |  test/     #   - from test/cpp
|  tmp/         # Work area for general temporary files

2 First-Generation – Dependency-Based

• Boxes are files.

• Arrows denote dependencies. “A depends on B” means that if B is missing or changed, then A must be (re)generated.

• Labels on arrows indicate the program used to generate the file at the base of the arrow.

Analysis of such a graph facilitates

• efficiency - easy to tell what needs to be rebuilt after a change

• incrementality - can determine required build step for any file, not just the “final” one

make is the canonical example of a build manager of this type.

2.1 make

make is a command/program that enacts builds according to a dependency graph expressed in a makefile.

• make devised by Dr. Stuart Feldman of Bel Labs in 1977

• It has long been a standard component of *nix systems

  • GNU make is a popular moden variant

2.2 makefiles

At its heart, a makefile is a collection of rules.

2.2.1 Rules

• A rule describes how to build a single file of the project.

  Each rule indicates

  • The target file to be constructed
  • The dependencies: the other files in this project from which the target is constructed.
  • The commands that must be executed to construct the target from its dependencies.

• Rules may appear in any order

  • Except that the first rule’s target is the default built by make when no explicit target is specified in the command line.

The Components of a Rule

• A rule has the form

  target: dependencies
  		commands
  

where

• target is the target file,

• dependencies is a space-separated list of files on which the target is dependent

• commands is a set of zero or more commands, one per line, each preceded by a Tab character.

Rule Examples

codeAnnotation.jar: code2HTML.class CppJavaScanner.class
        jar tvf codeAnnotation.jar code2HTML.class CppJavaScanner.class

CppJavaScanner.class: CppJavaScanner.java
        javac CppJavaScanner.java

code2HTML.class: code2HTML.java CppJavaScanner.java
        javac code2HTML.java

CppJavaScanner.java: code2html.flex
        java -cp JFlex.jar JFlex.Main code2html.flex

Pros & Cons

• • Widely available.
• • Make rules are invariably OS-specific.
• • Not every build step produces a single file.
• • Not every build step “consumes” files.
• • Arcane syntax.

3 Second Generation – Task-Based

• Ellipses are tasks (activities). Each task can involve multiple steps.

• Arrows denote success dependencies. “A depends on B” means that A will be run after B and only if task B finished successfully.

This approach facilitates

• ease of setup: usually less detailed than a full file-based dependency graph

• incrementality - can request any intermediate step

ant is based on this approach.

3.1 ant

• ant devised by James Davidson of Sun, contributed to Apache project (along with what would eventually become TomCat), released in 2000

• Quickly became a standard tool for Java projects

  • slower to move into other arenas

3.1.1 ant Features

• Task-based
• OS independent
• Commands are written in XML, not a scripting language
• ant itself is implemented in Java
• The command set can be extended via java classes

3.1.2 Targets

At its heart, a build file is a collection of targets.

• A target is an XML element and, as attributes, has a name and, optionally,

  • a list of dependencies
  • a condition
  • a human-readable description

• The target can contain multiple tasks, which contain the actual “commands” to get things done.

Example of Targets

3.2.1 Motivations for Maven

Grew out of an observation that many supposedly cooperative, related Apache projects had inconsistent and incompatible ant build structures.

3.2.2 pom.xml

The build file for maven is also in XML:

3.3 Maven and 3rd Party Libraries

One of the most important innovations introduced by Maven.

3.3.1 Dependencies

• pom.xml files have a dependencies section. e.g.,

    <dependencies>
  	<groupId>junit</groupId>
  	<artifactId>junit</artifactId>
  	<version>4.10</version>
  	<scope>test</scope>
    </dependencies>
  
  

• This indicates that our project requires the junit package.

• Could also say [4.10,] to get versions 4.10 or greater

3.3.2 Fetching Dependencies

• Maven does a transitive search over the dependencies for a project

  • Tries to find a mutually compatible set of versions
    • Helps if you give it some flexibility

• Maven then downloads the required libraries automatically

  • Downloaded libraries are cached (e.g., ~/.m2)

3.3.3 Maven Repositories

3.3.4 Transitive Dependencies

How does Maven know whether junit itself depends on other libraries?

• Near the top of the junit 4.10 page, click to “View” the POM file:

  Near the bottom, you will see

    <dependencies>
  	<groupId>org.hamcrest</groupId>
  	<artifactId>hamcrest-core</artifactId>
  	<version>1.3</version>
  	<scope>compile</scope>
    </dependencies>
  
  

• This is the same kind of info that we put into our own pom.xml file

  • And is, presumably, taken from the pom.xml that the JUnit team used to maintain their builds.
  • Publishing the dependency information along with the libraries leads to an accumulated base of information on library dependencies.

3.3.5 Too Good of an Idea to Let Go

ant got jealous…

• Eventually ant aquired a plugin, Ivy, that allowed it to handle 3rd party libraries in an almost identical fashion.
• Including searching the same repositories established for use by Maven.

4 Third Generation Build Managers

Combine

• task orientation
• OS independence
• archetypes to provide trivial defaults
• easy creation of custom tasks
• 3rd-party library management

For this we, will look to gradle.