Installing a GitLab CI Runner on AWS

Log in to AWS Educate.
Go “Classrooms & Credits” on the tool bar.
Select “Request or go to an AWS Educate Classroom”
Select “Go to my classrooms”

You should see this course listed under “Classrooms where I am a Student”.

1.2 AWS Education Starter Account

Still on the My Classrooms page, select “AWS Account” from the toolbar.
You should see something like

possibly with a link to set up your account.
Set your account up, if necessary.
Click the orange button to see your account status:
Click the AWS Console button to actually access AWS.

2 Creating a Linux Server

You should be looking at something like:

We’re going to set up a basic Linux server that we can use to do test and CI for the Java project whose build was described here.

2.1 Select an OS

We are limited to services that are handled by the AWS Educate program. This does not include LightBeam, the newer, simplified approach to launching Linux/Windows servers.

Fortunately, the older interface is not terribly complicated.

Select EC2

EC stands for “Elastic Computing”

Select “Launch Instance” from the orange menu titled “Launch Instance” in the box titled “Launch Instance”.
You’ll be presented with a list of OS versions and machine types that are available.

Select “Ubuntu Server 20.04 LTS”, 64-bit (x86).

2.2 Choose an Instance Type

Next, you will be asked to select the simulated machine “size”.

Our educational accounts are limited to “nano” and “micro” sizes.

Pick General purpose, t2.micro
Click Next: Configure Instance Details
- I selected “Request Spot instances”. Purportedly, this saves money at the cost of occasional service unavailibility. For a CI runner, I don’t think that would be a major concern.
- I left everything else on defaults, and clicked “Next: Add Storage”
I raised the drive size to 30Gb, the largest available without extra charge.
Click “Next: Add Tags”
Adding tags is optional. Can help ot identify your machines if you have a lot of them.

Click “Next: Configure Security Group”
This lets you set up firewall rules that restrict access to your machine.
- I used my home IP address and the address of my CS office PC.
- /32 on the end of the IP addresses allows the last 5 bits to vary.
Click “Review and Launch”, then “Launch”.

2.3 SSH Keys

Next you will be given the option of supplying an SSH key pair or createing a pair for access to the machine.

Follow the instructions to create a new pair and to download a copy of the private key (.pem format).

Note:

This is a passphrase-free key, so keeping it secure is critical.
If you lose it, you can’t get it back.

Follow through the instructions to actually launch your machine.

2.4 Checking on Your Instance

Return to the AWS Console and click on “Running instances” to see the details of your server.

Take note of the “Public IPv4 DNS” – that’s the name of your new server. You can use ordinary ssh with the private key you downloaded to access the server.
- User names are OS defaults, in this case, ubuntu.
- That’s more secure than it might look, because the ssh server is configured to only connect by SSH keys, not by username/password.
Alternatively, select the row with your server, click the Connect button above, then go to the “SSD client” tab for detailed instructions.

3 Installing and Setting Up a CI Runner

3.1 Install gitlab-ci-runner

Open an ssh connection to your new AWS server.

gitlab-ci-runner is distributed as a standard Ubuntu install package.

sudo apt-get update
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh | sudo bash
sudo apt-get install gitlab-runner
sudo gitlab-ci-multi-runner status

The last command should report that the service is “running”.

Note: this program can be invoked either as gitlab-ci-multi-runner or as gitlab-runner.

3.2 Create a shell Runner

Still in that SSH session:

sudo gitlab-runner register

You will be asked for the gitlab-ci coordinator URL (e.g. https://forge350.cs.odu.edu/ or https://git-community.cs.odu.edu/
You will be asked for the gitlab-ci token for this runner.

You get that by using your web browser to visit your GitLab project, look under Settings for CI/CD, click Expand in the “Runners” section and look for something like the text to the right. Copy the registration token, and paste it into your SSH session.
Enter a short description and tag (e.g., my-aws-shell)
When asked for the executor, answer shell.
- You can explore other possibilities later, but shell is the easiest to get started with.
When gitlab-runner is finished, give the command

sudo gitlab-runner list

You should see your new runner listed there.
Finally, the default account setup in Ubuntu clashes with the gitab runner:

sudo rm /home/gitlab-runner/.bash_logout
Return to your web browser and refresh the Runners page. You should see your new runner listed.

4 Install Other Required Software

If your runner will need additional software, now is the time to install it in your SSH. In this case, I will need java both to compile the code and simply to run gradle.

ubuntu@ip-xxx-xx-xx-xxx:~$ javac -version

Command 'javac' not found, but can be installed with:

sudo apt install default-jdk              # version 2:1.11-72, or
sudo apt install openjdk-11-jdk-headless  # version 11.0.8+10-0ubuntu1~20.04
sudo apt install openjdk-8-jdk-headless   # version 8u265-b01-0ubuntu2~20.04
sudo apt install openjdk-13-jdk-headless  # version 13.0.3+3-1ubuntu2
sudo apt install openjdk-14-jdk-headless  # version 14.0.1+7-1ubuntu1
sudo apt install ecj                      # version 3.16.0-1

ubuntu@ip-xxx-xx-xx-xxx:~$ sudo apt install openjdk-11-jdk-headless
    ⋮
ubuntu@ip-xxx-xx-xx-xxx:~$ javac -version
javac 11.0.8
ubuntu@ip-xxx-xx-xx-xxx:~$

5 Set up CI in the Project

All that remains is to set up the .gitlab-ci.yml file in the GitLab project:

.gitlab-ci.yml

stages:
  - build

build-job:
  tags:
    - aws-shell
  stage: build
  script:
    - eval $(ssh-agent -s)
    - ssh-add <(echo "$REPORTS_SSH_KEY")
    - export BUILD_LABEL="`date -Idate` $CI_BUILD_REF"
    - bash ./gradlew build deployReports --no-daemon -Dorg.gradle.project.buildNumber="$BUILD_LABEL"
  only:
    - master

5.1 Huzzah!

A few moments after committing the change with that .gitlab-ci.yml file, I saw the following in my project Pipelines:

Running with gitlab-runner 13.4.1 (e95f89a0)
   on aws-shell BeAectrA
Preparing the "shell" executor
00:00
 Using Shell executor...
Preparing environment
00:00
 Running on ip-172-31-22-147...
Getting source from Git repository
00:01
 Fetching changes with git depth set to 50...
 Initialized empty Git repository in /home/gitlab-runner/builds/BeAectrA/0/zeil/bcratchit/.git/
 Created fresh repository.
 Checking out 2b1c6059 as master...
 Skipping Git submodules setup
Executing "step_script" stage of the job script
 $ echo build number is $CI_BUILD_REF
 build number is 2b1c60598cdbf44a2b59e68ea8bb43acf77549e1
 $ eval $(ssh-agent -s)
 Agent pid 34451
 $ ssh-add <(echo "$REPORTS_SSH_KEY")
 Identity added: /dev/fd/63 (/dev/fd/63)
 $ export BUILD_LABEL="`date -Idate` $CI_BUILD_REF"
 $ cd bcratchit
 $ ./gradlew build deployReports
 Downloading https://services.gradle.org/distributions/gradle-6.3-bin.zip
 .................................................................................................
 Welcome to Gradle 6.3!
 Here are the highlights of this release:
  - Java 14 support
  - Improved error messages for unexpected failures
 For more details see https://docs.gradle.org/6.3/release-notes.html
 Starting a Gradle Daemon (subsequent builds will be faster)
 > Task :compileJava
 > Task :processResources NO-SOURCE
 > Task :classes
 > Task :jar
 > Task :startScripts
 > Task :distTar
 > Task :distZip
 > Task :assemble
     ⋮

(Truth be told, the build failed later, but that appears to be an issue with an incompatibility of JBake with Java 11.)