Abstract

In continuous integration, the practices of version control, automated building, automated configuration, and automated testing are combined so that, as changes are checked in to the version control repository, the system is automatically rebuilt, tested, reports generated, and the results posted to a project website.

• Integration problems caught early and fixed fast
  • avoids “integration hell”
• Immediate testing of all changes
• Emphasis on frequent check-ins encourages modularity
• Visible code quality metrics motivate developers.

2.1.2 Disadvantages

• Initial setup effort to set up
• Level of sophistication required of team to put build, configuration mgmt., testing, reporting, into an automated build

2.2.1 The Continuous Integration Server

 

A continuous integration server is a network-accessible machine that

• Can be told of development projects under way, including

  • location & access info to version control (VC) repository
  • which branch(es) to watch
  • how to build the project
  • what reports are produced by the build

• Monitors, in some fashion, the VC repository for commits

• When a commit (to a monitored branch) takes place, the CI server notifies one or more runners.

GitHub provides an integrated CI service called “Actions”.

• “Actions” on the main project page shows recent runs from commits.

  • Example

    Click on links with green checkmarks and red X’s to see successful and failed builds.

    Then click on “Build” to see the details.

1. In the root directory of your project, create a .github/workflows/ directory.

   This can hold one or more action scripts, written in YAML.

2. Create a file with a .yml extension in that directory describing your desired CI actions.

   This will have a mixture of metadata describing how you want this run and run blocks containing scripted commands.

3. Commit and push.

An example of .github/workflows/gradle.yml

# This workflow will build a Java project with Gradle

name: CI - build and test

on:                                
  push:
    branches: [ main, CI-setup ]

jobs:
  build:                           

    runs-on: ubuntu-latest         

    steps:                         
      ⋮

• name: Pick any name you like. these will appear in the log of the actions performed.
• on: Describes the circumstances under which this CI script is activated.
  • push: indicates that we want to perform these action when commits are pushed to selected branches.
    • branches: A list ([ ]) of branch names on pushes will cause this to be run. Often this will only occur on pushes to main, but CI-setup was the branch I used to test out this CI script.
• jobs: You can have one or more of these. Jobs are run in parallel (if runners are available to do so). By default, they do not share any information, although that can be changed.
  • build: “build” is the name I chose to give my job. It’s components are:

    • runs-on: This describes the runner I want to use.

      In this case, I have opted for a GitHub-supplied runner, one that provides the latest version of Ubuntu Linux.

    • steps: the scripted steps that I actually want performed.

      We’ll look at these in detail next.

      ⋮
    steps:                         
    - uses: actions/checkout@v4                   ➀
      with:
        fetch-depth: 0
    - name: Set up JDK 11                         ➁
      uses: actions/setup-java@v2        
      with:
        java-version: '11'
        distribution: 'adopt'
    - name: Grant execute permission for gradlew  ➂
      run: chmod +x gradlew              
    - name: Build and test the plugin             ➃
      working-directory: ./cowem-plugin
      timeout-minutes: 20
      run: ../gradlew build

• ➀ Almost every job will start with this. It is a predefined action that causes your repository to be cloned into the working area and set to the appropriate branch.

  The fetch-depth parameter causes all remote branches to be tracked in the new repository instead of just the main branch. That’s not always necessary, but would be useful when combining this with the branch-based Github mechanism for deploying reports to a Github website. (We would still need to use git commands to create a local branch tracking the remote one, but this makes sure that the remote one is available.)

• ➁ Sets up the Java 11 compiler on the Ubuntu Linux runner.

• ➂ A bit of insurance. If you or any of your teammates are working in Windows, there’s a good change that the executable permission will not be set on the gradlew script. So we make sure that gradlew is executable.

• ➃ Because this project has subprojects within the project root directory, we set the working directory to one of the subprojects and then run Gradle.

In an earlier lesson, we saw that GitHub uses a potentially complicated series of git actions to publish web content in a separate repository branch (usually, gh-pages).

Not surprisingly, they’ve packaged this up into a pre-defined action that can be invoked in the steps instructions:

- name: Deploy
        uses: JamesIves/github-pages-deploy-action@v4.3.4  ➀
        with:
          branch: gh-pages                                 ➁
          folder: build/reports                            ➂

• ➀ The name of the deployment action.
• ➁ The branch being used to hold the website.
• ➂ The directory containing the website content that you want to deploy.

Run this after the steps in which you build your project (including the website content).

You can find an example of a project using this technique here.

For example, in one of our website deployment case studies, we developed Gradle tasks that cloned a GitHub repository. That would likely work if you ran it from the command line, because presumably you would already have an SSH key set up to give you access to GitHub. But it won’t work when dropped into a CI script, because the CI runner will not already have those credentials.

The secrets are added from the Secrets tab of a Github repository’s Settings page. Click the New repository secret button to add one.

For example, suppose we wanted to provide an SSH key that could be used from within an action.

1. Create a new SSH key pair. Make this one passphrase-free, as possession of the private key is going to be restricted to this single use.
2. Add the public half of the key to your (or some other team member’s) GitHub account.
3. Go to your repository’s Settings page on Github. Select the Secrets tab and click the New repository secret button.
4. Give this secret a name, for example, REPORTS_SSH_KEY. Paste the text of the private key into the “Value” box and click Add secret..

You can now use that name like a variable in your Github actions via the rather arcane syntax:

${{ secrets.secret-name }}

For example, to set up an SSH agent that can supply that key, you could add this to your CI script:

- name: Deploy reports using an SSH agent
      run: |
        eval $(ssh-agent -s -t 600)                           ➀
        ssh-add <(echo "${{ secrets.REPORTS_SSH_KEY }}")      ➁
        git config --global user.email "you@email.address"      ➂
        git config --global user.name "Project Actions"
        ./gradlew deployWebsite                               ➃

• ➀ This line launches an ssh key agent.

  The -t 600 option limits this agent to a maximum of 600 seconds before it stops offering the key.

• ➁ We know the ssh-add command as a way to add private ssh keys to the agent.

  In this case, the text of the (passphrase-free) private key is being supplied by Github as a secret project variable REPORTS_SSH_KEY.

• ➂ git will insist on these user properties being set before it will permit commits.

• ➃ The actual commands, whatever they might be, that make use fo the SSH agent must occur within the same run: block, because the environment variables, created by ssh-agent, used to access the SSH agent will not persist after the end of that set of instructions.

Any later steps in the CI script will now be able to access the Github account to which the public half of that key is registered.

You can find an example of a project using this technique here.

1. You install the runner application on your own machine and use your GitHub project settings to add that self-hosted runner to the project/repository.
2. On your hosting machine, either start the runner application manually or configure the machine to run the application automatically on startup.
3. Instead of saying that your CI scripts runs-on a GitHub machine, you specify one or more of the descriptive labels for your runner application.

• Continuous deployment publishes snapshots of deliverables as changes are checked in.

  • A variation of the rather common “daily build” practice seen on many projects.
  • Some Maven repositories (including our own Artifactory instance) provide separate “snapshot” repositories for this purpose.
  • We have not discussed publishing to repositories, although this is well supported within Gradle.

• Some organizations actually wire up build light indicators to provide a highly visible indicator of the status of the latest integration build.

  • Some then point a webcam at the light and broadcast their status.
  • Others opt for publishing a software analog of such lights on their project website.