Managing Third-Party Libraries

Steven J Zeil

Last modified: Mar 31, 2019
Contents:

Abstract

Modern build managers provide support for

so that all of these can be handled quickly and efficiently as part of the normal build.

1 Issues

Baselines may include multiple 3rd-party libraries.

 


Example: Report Accumulator

A project of mine, this page lists the 3rd party libraries required by the project.

2 Basic Concepts

2.1 Repositories

A repository is a collection of packages (usually libraries) and metadata about packages

2.1.1 Common Java Repositories

The Java world is dominated by

2.1.2 Local Repositories

It’s common for development organizations to host their own repository.

Local repositories can be

2.2 Identifying Packages

Packages are typically identified by

2.3 Scopes

Not every library that your project needs is needed all of the time.

So build managers allow you to import libraries in selected scopes which indicate when the library is actually needed.

This affects

3 Adding Libraries to a Java Project in Different Build Managers

3.1 Maven

3.1.1 Specifying a Dependency

3.1.2 Fetching Dependencies

When the build is run (e.g. mvn package):

3.1.3 Transitive Dependencies

How does Maven know whether junit itself depends on other libraries?

3.1.4 Choosing Repositories


3.1.5 Example: specifying a repository

In pom.xml:

  <repositories>
     <repository>
        <snapshots>
           <enabled>false</enabled>
        </snapshots>
        <id>central</id>
           <name>libs-release</name>
           <url>http://archive350.cs.odu.edu:8080/artifactory/libs-release</url>
    </repository>
  </repositories>

3.2 Ivy (ant)

YAAP1 , Ivy adds the dependency management features pioneered by Maven to ant

3.2.1 Getting Started with Ivy

The Ivy project pages include some “magic” ant code to * download the latest version of Ivy * add a resolve-ivy task to be invoked before you compile your code * This task will download any 3rd-party dependencies and add them to your compilation classpaths.

3.2.2 Specifying our Desired Libraries

We list the libraries our project needs in a separate ivy.xml file

<ivy-module version="2.0">
  <dependencies>
    <dependency org="de.jflex" name="jflex" rev="1.4.3"/>
    <dependency org="junit" name="junit" rev="4.10"/>
  </dependencies>
</ivy-module>

This should look familiar to the section from the Maven pom:

  <dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.10</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
  

3.2.3 Invoking Ivy

Once Ivy is installed, we can invoke it:

build.xml2.listing
<project name="codeAnnotation" basedir="." default="build"
	 xmlns:ivy="antlib:org.apache.ivy.ant">
     ⋮

  <target name="resolve-ivy" depends="install-ivy">
    <ivy:resolve/>                         ➀
    <ivy:cachepath pathid="ivy.path" />  ➁
    <taskdef classpath="JFlex.jar" 
	     classname="JFlex.anttask.JFlexTask" name="jflex" /> ➂
  </target>

  <target name="generateSource" depends="resolve-ivy">
    <mkdir dir="target/gen/java"/>
    <jflex file="src/main/jflex/code2html.flex"    ➃
	   destdir="target/gen/java"/>
    <jflex file="src/main/jflex/code2tex.flex"
	   destdir="target/gen/java"/>

        ⋮

3.2.4 Ivy & Repositories

An example of an ivysettings.xml file:

ivysettings.xml.listing
<?xml version="1.0" encoding="UTF-8"?>
<ivy-settings>
  <settings defaultResolver="default" />
 <resolvers>
    <chain name="default">   ➀
      <ibiblio name="central" m2compatible="true"/> ➁
    </chain>
  </resolvers>
</ivy-settings>

3.2.5 Ivy & Eclipse

The developers of Ivy provide an Eclipse plugin, IvyDE.

3.3 Gradle

As is often the case, gradle tends to provide simpler syntax to do much the same things as Maven and Ant.

3.3.1 Selecting libraries in Gradle

In build.gradle:

apply plugin: 'java'


⋮

repositories {
    jcenter()   ➀

    // Use my own CS dept repo
    ivy {      ➁
        url 'https://secweb.cs.odu.edu/~zeil/ivyrepo'
    }
}



dependencies { ➂
    compile 'net.sf.saxon:saxon-dom:8.7+'
    testCompile 'junit:junit:4.12'
    testRuntime 'edu.odu.cs.zeil:reportAccumulator:1.1+'

}

Here you see both the choice of libraries and of repositories , .

The dependencies indicate

4 Eclipse and Library Management

Each of the build/library managers we have discussed have support in Eclipse

4.1 Eclipse and Maven

4.2 Eclipse and Ant/Ivy

4.3 Eclipse and Gradle

Eclipse has various Gradle plugins available.

The most popular seems to be the BuildShip package, available from the Gradle Marketplace (in the Eclipse Help menu).

5 Complicating Factors

5.1 Plug-ins

Plug-ins to build managers modify the build process itself

Consequently, plug-ins generally need to be fetched and processed at the very beginning of a build, much earlier than most 3rd-party libraries.

5.1.1 Separate but similar

Plug-ins to build managers are generally handled separately from other items, though the syntax is often similar.

5.2 Publishing

Publishing artifacts to a repository is also supported by maven, Ant/Ivy, & Gradle.


Example: in Gradle build.xml

publishing {
    publications {
        ivyJava(IvyPublication) {
            organisation 'edu.odu.cs.zeil'
            module 'cowem-plugin'
            revision project.version
            descriptor.status = 'integration'   // milestone, release
            descriptor.branch = 'v1'

            from components.java
        }
    }

    repositories {
        ivy {
            name 'ivyRepo'
            url 'sftp://atria.cs.odu.edu:22/home/zeil/secure_html/ivyrepo'
            // Readable via https://www.cs.odu.edu/~zeil/ivyrepo
            credentials {
                ⋮
            }
        }
    }
}

5.3 Login Credentials

Providing login credentials to a build is problematic.

A common work-around is to store a few build “properties” in a file in the user’s home directory, not kept as part of the package version control.


Example: in Gradle build.xml

// Credentials are loaded from ~/.gradle/gradle.properties
if(project.hasProperty("ivyRepoUser")){        ➀
    ext.ivyRepoUser = "$ivyRepoUser";
} else {
    ext.ivyRepoUser = "user";
}
if(project.hasProperty("ivyRepoPass")){
    ext.ivyRepoPass = "$ivyRepoPass";
} else {
    ext.ivyRepoPass = "password";
}

⋮

publishing {
    ⋮
            credentials {                    ➁
                username project.ivyRepoUser
                password project.ivyRepoPass
            }
    ⋮
}

6 What about C++?

Sadly, the situation for C++ is somewhat more complicated.

6.1 What’s Holding Us Up?

We Don’t Have Anything Equivalent to Maven Central for C++

There’s no recognized central place to publish and find open source C++ libraries.


We Don’t Have Anything Equivalent to Maven/Ivy/Gradle Dependency Management for C++

There are no build managers capable of processing dependencies for C++ while simultaneously factoring in the platform-specific requirements.


Which Came First, the Chicken or the Egg?

6.2 On the Horizon


1: Yet Another Apache Project