Project Reports & Websites
Steven J Zeil
Abstract
In this lesson we will continue looking at project documentation:
- Project reports
- Generating project websites
We will look at how to use our automated build tools to keep project reporting up-to-date.
1 Project Reports
1.1 Test Reports
We’ve already looked at JUnit, which can be used to generate test reports like this one or this one.
This is generated in Gradle via the java
plugin. It produces the web reports in build\reports\tests
.
1.2 Static Code Analyzers
Many tools that we will cover later for analyzing code can produce useful (or at least, impressive) documentation as a side effect.
1.3 Configuration Reports
Configuration managers generate reports about the dependencies among the software components.
For example, this report comes from Gradle by adding
plugins {
⋮
id 'project-report'
⋮
}
to the build.xml
file
2 Project Websites
-
In general, websites can be “manually” constructed, file by file, or any of a number of site-builder tools can be employed to create large numbers of pages that share a common look-and-feel.
-
An innovation of Maven was to consider construction of project websites as a part of the automated build. With a simple
mvn site
command, you could produce an entire site like this one.
-
In Gradle this can be managed by custom tasks or by various plugins.
2.1 Case study: Constructing a project website with Gradle
Let’s look at the process of building a simple website that provides
- A brief project description
- Links to (and copies of)
- the project configuration report
- the JUnit test report
- the Javadoc API documentation
We can consider adding other content in later lessons.
2.1.1 Build plan
After a build, we can construct a website in build/reports
by
-
Copying
build/doc/
intobuild/reports
-
This gets a copy of the Javadocs
-
Note that the JUnit and project configuration reports will already be in subdirectories of
build/reports
.
-
-
Copying HTML, CSS, and Javascript files from
src/main/html
into reports.-
Hand-crafted pages with the project description and links to the three reports.
-
2.1.2 The HTML File
<!DOCTYPE html>
<html>
<head>
<title>Project Reports</title>
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<link rel="stylesheet" type="text/css" href="projectReports.css"/>
<script src="projectReports.js"></script>
</head>
<body>
<h1>CodeCompCommon</h1>
<p>This project provides a common set of interfaces for all CS350
CodeComp project variants.</p>
<p>It also provides a SharedPhrases class used to store token lists
obtained via lexical analysis of the source code.</p>
<h2>Information</h2>
<p>Current version: 1.3</p>
<h2>Detailed Reports</h2>
<div class="reportLinks">
<ul>
<li>General:
<ul>
<li><a href="../docs/javadoc/">JavaDoc</a></li>
<li><a href="../project/dependencies/root.html">Project Dependencies</a></li>
</ul></li>
<li>Tests:
<ul>
<li><a href="../tests/test/">JUnit tests</a></li>
</ul>
</li>
</ul>
</div>
</body>
</html>
2.1.3 Gradle setup
After a build, we can construct a website in build/reports
by
- Copying
build/doc/
intobuild/website
- Copying HTML, CSS, and Javascript files from
src/main/html
into reports.test.ignoreFailures=true ➀ check.dependsOn htmlDependencyReport ➁ task copyWebPages (type: Copy) { ➄ from 'src/main/html' into 'build/reports' } task copyDocs (type: Copy) { ➃ from 'build/docs' into 'build/reports/docs' dependsOn 'javadoc' } task buildSite (dependsOn: ['javadoc', 'check', ➂ 'copyWebPages', 'copyDocs']) { description 'Generate reports website for this project' }
-
➀ Don’t want the build to stop just because some unit tests are failing
- In TDD, some tests are almost always failing
- One reason for posting the JUnit report is so that people can see what we fail and what we pass.
-
➁ Force the generation of the configuration report whenever we build the
check
target/task. -
➂ Here’s our new target,
buildSite
.It has no actions of its own, but the depedencies force everything else to happen.
-
➃ Here’s step 1 of our process: copying the
docs
into the website - ➄ Here’s step 2 of our process: copying the hand-crafted content from
src/html
into the website
2.2 Case study: Constructing a project website with Gradle and Jbake
For a more elaborate, multi-page site, we can use a static site generator to make it easy to enforce a common look-and-feel across pages.
I’ve chose Jbake mainly because it has an easy-to-use gradle plugin.
JBake injects content using a series of templates
-
Content files can contain HTML, or Markdown, or a number of other text forms.
-
Each has a small amount of metadata at the top. Among other things, this identifies the starting template used to render that content.
-
-
Templates are HTML-like snippets that indicate where to inject content and what other templates to include.
A typical “page” template may load separate templates for headers, footers, navigation bars, menus, etc.
2.2.1 Gradle & Jbake
Gradle has a jbake
plugin that adds a bake
task to generate a website.
- By default looks for source files in
src/jbake
- By default writes the site into
build/jbake
buildscript { ⋮ dependencies { ⋮ classpath "org.jbake:jbake-gradle-plugin:5.0.0" } } ⋮ apply plugin: 'org.jbake.site'
2.2.2 The website setup
src
|-- main
|-- |-- jbake
|-- |-- |-- assets
|-- |-- |-- |-- css
|-- |-- |-- |-- '-- base.css
|-- |-- |-- '-- js
|-- |-- |-- '-- jquery-1.11.1.min.js
|-- |-- |-- content
|-- |-- |-- |-- dependencies.html
|-- |-- |-- |-- home0.md
|-- |-- |-- |-- javadoc.html
|-- |-- |-- '-- junit.html
|-- |-- |-- jbake.properties
|-- |-- '-- templates
|-- |-- |-- footer.ftl
|-- |-- |-- header.ftl
|-- |-- |-- index.ftl
|-- |-- |-- menu.ftl
|-- |-- |-- page.ftl
|-- |-- '-- sitemap.ftl
The content
directory` holds the actual information we want to disseminate.
The templates
directory determines the “look and feel” of the website.
- The key template is
page.ftl
- Every content file will be rendered using this page template.
Templates:
<#include "header.ftl">
<div class="center">
<#include "menu.ftl">
<div class="rightPart">
<div class="page-header">
<h1><#escape x as x?xml>${content.title}</#escape></h1>
</div>
<p><em>${content.date?string("dd MMMM yyyy")}</em></p>
<p>${content.body}</p>
</div>
</div>
<hr />
<#include "footer.ftl">
- The content is loaded as
${content.body}
. - The page template also
#include
s the header, menu, and footer templates.
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<title><#if (content.title)??><#escape x as x?xml>${content.title}</#escape><#else>codecentric</#if></title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="description" content="">
<meta name="author" content="">
<meta name="keywords" content="">
<meta name="generator" content="JBake">
<link href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>css/base.css" rel="stylesheet">
<link href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>css/projectReports.css" rel="stylesheet">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.0/jquery.min.js" type="text/javascript"></script>
<script src="https://code.highcharts.com/highcharts.js"></script>
<script src="https://code.highcharts.com/modules/data.js"></script>
<script src="<#if (content.rootpath)??>${content.rootpath}<#else></#if>js/projectReports.js"></script>
</head>
<body>
<div id="mainBody">
<div class="leftPart">
<div class="menuBlock">
<span class="menuBlockHeader"><a href="<#if
(content.rootpath)??>${content.rootpath}<#else></#if>index.html">Home</a></span>
</div>
<div class="menuBlock">
<span class="menuBlockHeader">Project Info</span>
<ul>
<li><a href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>javadoc.html">API (Javadoc)</a></li>
<li><a href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>dependencies.html">Dependencies</a></li>
</ul>
</div>
<div class="menuBlock">
<span class="menuBlockHeader">Testing</span>
<ul>
<li><a href="<#if
(content.rootpath)??>${content.rootpath}<#else></#if>junit.html">Unit
Tests</a></li>
</ul>
</div>
<div class="menuBlock">
<span class="menuBlockHeader">Analysis Reports</span>
<ul>
<li><a href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>checkstyle.html">Checkstyle</a></li>
<li><a href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>spotbugs.html">SpotBugs</a></li>
</ul>
</div>
</div>
</div>
<div class="footer">
© 2020 Old Dominion University
</div>
</body>
</html>
Content:
title=code-grader Unit Tests
type=page
status=published
~~~~~~
</p>
<div class=reportGraphs>
<div id="junitGraph" class="graph">Test Results</div>
</div>
<iframe class="docFrame" src="tests/test/index.html"> </iframe>
<script type="text/javascript">
register2("junitGraph", "tests.csv", "JUnit Tests", "Test Cases");
</script>
<p>
title=Project Documentation: code grader
date=2024-02-18
type=page
status=published
~~~~~~
2.2.3 Build Steps
To generate a website in build/jbake
- Copy the
build/reports
andbuild/docs
tobuild/tmp/website
- Copy
src/jbake
tobuild/tmp/website
- Concatenate
build/tmp/website/content/home0.md
and the projectREADME.md
to formbuild/tmp/website/content/home.md
- Run
Jbake
onbuild/tmp/website/content/home.md
to generate the full website inbuild/jbake
In build.gradle
:
test.ignoreFailures=true
check.dependsOn htmlDependencyReport
task reports (dependsOn: ['javadoc', 'check']) {
description 'Generate all reports for this project'
}
task copyJDocs (type: Copy) {
from 'build/docs'
into 'build/tmp/website/assets'
dependsOn 'javadoc'
}
task copyReports (type: Copy) {
from 'build/reports'
into 'build/tmp/website/assets'
dependsOn 'reports'
}
task copyJbakeTemplates (type: Copy) {
from 'src/jbake'
into 'build/tmp/website'
}
task buildHomePage (dependsOn: copyJbakeTemplates) {
inputs.files ( 'build/tmp/website/content/home0.md', '../README.md')
outputs.file ( 'build/tmp/website/content/home.md' )
doLast {
outputs.files.singleFile.withOutputStream { out ->
for ( file in inputs.files ) file.withInputStream {
out << it << '\n' }
}
}
}
jbake {
srcDirName = "build/tmp/website"
}
task setupWebsite (dependsOn: ['buildHomePage', 'copyReports', 'copyJDocs']){
}
bake.dependsOn "setupWebsite"
- New target is
bake
which runsJBake
and depends onsetupWebsite
setupWebsite
carries out most of our process- Trickiest part is
buildHomePage
, which carries out the concatentation - Most of the rest resembles our previous website build
- Trickiest part is
3 Case Study: Enhanced reporting
Goal: add project website pages with reports from analysis tools and trend information (historical graphs) similar to those offered by Jenkins.
-
We will build on the example of our JBake-generated website.
-
An example of a page we would like to generate is this PMD report page.
3.1 Generating Graphs
Highcharts is a Javascript package that can generate plots from data captured in CSV (Comma-Separated-Values) format.
- For example, the plot on this page is generated from a file
pmd.csv
that looks like:pmd,Violations 2019-03-23T14:30,22.0 2019-03-23T14:33,22.0 2019-03-23T15:21,22.0 2019-03-23T15:45,22.0 2019-03-24T21:59,22.0 2019-03-31T19:48,22.0 2019-03-31T20:26,22.0 2019-04-03T20:42,11.0
Each line (after the headers) represents one data point in the chart.
Highcharts requires a bit of Javascript to inject a chart into an HTML div
element.
-
We load the Highcharts code in our website header
header.ftl.listing<!DOCTYPE html> <html lang="en"> <head> <meta charset="utf-8"/> <title><#if (content.title)??><#escape x as x?xml>${content.title}</#escape><#else>codecentric</#if></title> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta name="description" content=""> <meta name="author" content=""> <meta name="keywords" content=""> <meta name="generator" content="JBake"> <link href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>css/base.css" rel="stylesheet"> <link href="<#if (content.rootpath)??>${content.rootpath}<#else></#if>css/projectReports.css" rel="stylesheet"> <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.0/jquery.min.js" type="text/javascript"></script> <script src="https://code.highcharts.com/highcharts.js"></script> <script src="https://code.highcharts.com/modules/data.js"></script> <script src="<#if (content.rootpath)??>${content.rootpath}<#else></#if>js/projectReports.js"></script> </head> <body> <div id="mainBody">
-
This includes my own Javascript file to make it easier to share the code among several pages.
projectReports.js.listing/* * Register a div in the webpage as a HighCharts (http://www.highcharts.com/) chart * portraying a series of data in a CSV file */ /* register1 * For CSV files containing a single series of data. Column A contains the * Build numbers used as x values, column B the y values. * * @param graphName The id of the div to hold this chart. * @param csvURL URL to the .csv file * @param title Title for this chart. * @param yAxistitle Title for the Y axis. (Series title is taken from * row 1 of the CSV file). */ function register1(graphName, csvURL, title, yAxisTitle) { var divName = "#" + graphName; $(document).ready(function() { $.get(csvURL, function(csv) { $(divName).highcharts({ chart: { type: 'area' }, data: { csv: csv }, title: { text: title }, yAxis: { title: { text: yAxisTitle } }, xAxis: { title: { text: "Build #" } }, plotOptions: { series: { stacking: 'normal' } }, series: [{ color: "#0000cc" } ] }); }); }); } /* register2 * For CSV files containing two series of data. Column A contains the * Build numbers used as x values, columns B and C the y values. * * @param graphName The id of the div to hold this chart. * @param csvURL URL to the .csv file * @param title Title for this chart. * @param yAxistitle Title for the Y axis. (Series title is taken from * row 1 of the CSV file). */ function register2(graphName, csvURL, title, yAxisTitle) { var divName = "#" + graphName; $(document).ready(function() { $.get(csvURL, function(csv) { $(divName).highcharts({ chart: { type: 'area' }, data: { csv: csv }, title: { text: title }, yAxis: { title: { text: yAxisTitle } }, xAxis: { title: { text: "Build #" } }, plotOptions: { series: { stacking: 'normal' } }, series: [{ color: "#009933" }, { color: "#cc0000" } ] }); }); }); }
-
Which makes for a reasonably straightforward content page:
pmd.html.listingtitle=CodeCompCommon PMD Report type=page status=published ~~~~~~ </p> <div class=reportGraphs> <div id="theGraph" class="graph">PMD</div> ➀ </div> <iframe class="docFrame" src="pmd/main.html"> </iframe> ➁ <script type="text/javascript"> register1("theGraph", "pmd.csv", "PMD", "Warnings"); ➂ </script> <p>
- This is the
div
that will be replaced by the chart. - The body of the report.
This is very similar to the way we loaded our Javadoc and other reports earlier.
-
This calls my Javascript function, which in turn calls the Highchart functions, to schedule replacement of the above
div
by a chart derived from the data inpmd.csv
.
- This is the
3.2 Generating the Data
Where does the data for the plots come from?
-
Individual data points are extracted from the reports generated by PMD and other tools.
-
Each analysis tool requires some custom coding to get the data point.
-
-
Those data points are accumulated into a
.csv
file for the report by- Downloading the old CSV file (if there is one) fro mthe project website.
- Adding the new data point as a new line at the end of the CSV file.
-
When the website is uploaded, it will include the new, one-line-longer, CSV file,
-
These steps are carried out by my own Report Accumulator Gradle plugin.
3.3 Report Accumulator
The reportAccumulator
plugin is currently distributed from my own repository. So, in settings.gradle
:
pluginManagement {
repositories {
ivy { // Use my own CS dept repo
url 'https://www.cs.odu.edu/~zeil/ivyrepo' // My binary repository
}
gradlePluginPortal()
mavenCentral()
}
}
Then back to build.gradle
to load another plugin:
plugins {
⋮
id 'edu.odu.cs.report_accumulator' version '1.4' ➀
}
⋮
// Reporting
reportStats {
reportsURL = 'https://www.cs.odu.edu/~zeil/gitlab/codeCompCommon/' ➁
}
reportStats.dependsOn check ➂
task deployReports ➃
⋮
}
deployReports.dependsOn reportStats ➄
-
➀ The usual steps to include a plugin.
This plugin adds a new task,
reportStats
to your build. -
➁ The
reportStats
task needs to know where to look to find your website with any previously accumulated statistics.You would, of course, change this URL to something appropriate to your owen project.
-
➂ Make sure that all of the reports have already been run.
-
➃ You will presumably have some task that uploads your materials to your website.
You will want to make that task depend on
reportStats
. ➄
See the documentation on the reportAccumulator plugin for more details.
3.4 Forges
A software forge is a collection of web services for the support of collaborative software devlopment:
-
Project web sites
-
Networked access to version control
- Release (download) support
-
Communications (e.g., messaging, wikis, announcements)
-
Bug reporting and tracking
-
Project personnel management
Forge Examples
Among the best known forges are
-
the original, SourceForge, (1999)
-
Google Code, (2006)
-
GitHub, (2008)