Do Comments Matter?

McConnell has a good & balanced discussion on this.

Source code commenting is often a crutch to hide
- poor naming
- a failure to extract code blocks into recognizable functions
- poor design
- lack of quality tools (version control, issue tracking, source formatters)
Still useful for
- Explaining why a thing is being done
- Documenting a pseudo-code based design
- Cross-referencing related items

Modern focus has shifted considerably away from commenting bodies towards API documentation.

Which is better?

double m; // mean average
double s; // standard deviation

double meanAverage
double standardDeviation

Which is better?

// Sum up the data
double sum = 0.0;
double sumSquares = 0.0;
// Add up the sums
for (double d: scores)
{
   sum += d;
   sumSquares += d*d;
}

// Compute the average and standard
//  deviation
double meanAverage = sum / numScores;
double standardDeviation =
   sqrt ((sumSquares - numScores*sum*sum)
            /(numScores - 1.0));

// Subtract the average from each data
// item and divide by the standard
// deviation.
for (int i = 0; i < numScores; ++i)
{
   scores[i] = (scores[i] - meanAverage)
       / standardDeviation;
}

// Compute summary statistics
double sum = 0.0;
double sumSquares = 0.0;

for (double d: scores)
{
   sum += d;
   sumSquares += d*d;
}

double meanAverage = sum / numScores;
double standardDeviation =
   sqrt ((sumSquares - numScores*sum*sum)
            / (numScores - 1.0));

// Normalize the scores
for (int i = 0; i < numScores; ++i)
{
   scores[i] = (scores[i] - meanAverage)
       / standardDeviation;
}

Which is better?

// Compute summary statistics
double sum = 0.0;
double sumSquares = 0.0;

for (double d: scores)
{
   sum += d;
   sumSquares += d*d;
}

double meanAverage = sum / numScores;
double standardDeviation =
   sqrt ((sumSquares - numScores*sum*sum)
            /(numScores - 1.0));

// Normalize the scores
for (int i = 0; i < numScores; ++i)
   scores[i] = (scores[i] - meanAverage)
       / standardDeviation;

void computeSummaryStatistics (
   const double* scores,      // inputs
   int numScores,
   double& meanAverage,       // outputs
   double& standardDeviation)
{
  double sum = 0.0;
  double sumSquares = 0.0;
  for (double d: scores)
  {
     sum += d;
     sumSquares += d*d;
  }

  meanAverage = sum / numScores;
  standardDeviation =
     sqrt ((sumSquares - numScores*sum*sum)
              /(numScores - 1.0));
}


void normalizeData (double* data,
                    int numData,
                    double center,
                    double spread)
{
  for (int i = 0; i < numData; ++i)
    data[i] = (data[i] - center) / spread;
}

    ⋮

double meanAverage;
double standardDeviation;
computeSummaryStatistics (scores, numScores,
    meanAverage, standardDeviation);
normalizeData (scores, numScores,
    meanAverage, standardDeviation);

Which is better?

void computeSummaryStatistics (
   const double* scores,      // inputs
   int numScores,
   double& meanAverage,       // outputs
   double& standardDeviation)
{
  double sum = 0.0;
  double sumSquares = 0.0;

  for (double d: scores)
  {
     sum += d;
     sumSquares += d*d;
  }

  meanAverage = sum / numScores;
  standardDeviation =
     sqrt ((sumSquares - numScores*sum*sum)
              /(numScores - 1.0));
}


void normalizeData (double* data,
                    int numData,
                    double center,
                    double spread)
{
  for (int i = 0; i < numData; ++i)
     data[i] = (data[i] - center) / spread;
}
    ⋮

double meanAverage;
double standardDeviation;
computeSummaryStatistics (scores, numScores,
    meanAverage, standardDeviation);
normalizeData (scores, numScores,
    meanAverage, standardDeviation);

void computeSummaryStatistics (
   const double* scores,      // inputs
   int numScores,
   double& meanAverage,       // outputs
   double& standardDeviation)
{
  double sum = accumulate(
     scores, scores+numScores);
  double sumSquares = accumulate(
     scores, scores+numScores,
     [](double x, double y)
       {return x + y*y;});

  meanAverage = sum / numScores;
  standardDeviation =
     sqrt ((sumSquares - numScores*sum*sum)
              /(numScores - 1.0));
}


    ⋮

// Normalize the scores
double meanAverage;
double standardDeviation;
computeSummaryStatistics (scores, numScores,
    meanAverage, standardDeviation);
transform (
    scores, scores+numScores,
    scores,
    [] (double d) {
      return (d - meanAverage)
               / standardDeviation});

1.2 Charting

How many forms of software documentation charting do you know?

Control Flow
- Flowcharts
- Nassi-Schneidermann Charts
- State diagrams
- UML interaction diagrams
Module relationships
- Structure (call) charts
- Data-Flow Diagrams
- SADT (Structured Analysis and Design Technique)
- E-R
- UML class relationship diagrams

From Code to Charts

For as long as people have been writing source code, they’ve been looking for ways to ease the effort of documenting that code.
- Often after-the-fact
Earliest examples were automatic flowchart generators

Generating flowcharts from source code.
- Raw results were poor quality
  - But still could be claimed to satisfy client requirements
- As flowcharts declined in popularity, so did the demand for these tools.
- Still offered in reverse engineering tools ( e.g. )
  - Flowchart synced to code viewer
  - Human retitles blocks as “understanding” of the code progresses

From Charts to Code

A hallmark of so-called CASE (Computer-Aided Software Engineering) systems

Modern versions generate class declarations from UML class diagrams

2. API Documentation

API documentation tools are now more common

Reflect modern emphasis on re-usable interfaces
Combine info from
- a (limited) language parser
  - Extracts info about module/function structure and function parameters
- and specially formatted blocks of comments embedded in the source code
Encourages updating comments as code is modified

Comments become a legitimately useful tool for application writers.
- Application writers have less need to access actual code.
- Generate linked documents to facilitate browsing of referenced type names and other entities
- Some IDEs understand this markup as well and use it enhance “live” help while editing code.

2.1 javadoc

Perhaps the best known tool in this category

part of the standard Java distribution
achieved prominence when Sun used it to document the Java “standard library”.
- E.g., 1.6, 1.7

Javadoc Comments

Javadoc markup is enclosed in comments delineated by /** ... */
- And therefore processed as normal comments by the Java compiler.
A comment block precedes the entity that it describes
- e.g., This page is generated from

SegmentationTransformer.java

/**
 * 
 */
package edu.odu.cs.extract.control;

import org.jdom.Document;

import edu.odu.cs.extract.dataflow.Dataflow;
import edu.odu.cs.extract.dataflow.QuickTransformer;
import edu.odu.cs.extract.dataflow.TransformationResult;
import edu.odu.cs.extract.inputprocessing.segmentation.Segmentation;
import edu.odu.cs.extract.utils.Properties;

/**
 * Transforms a PDF file dataflow into Raw IDM by attempting a direct translation
 * of text PDF, but passing pages thought to be scanned on for OCR and thenby trimming to a selected number of pages
 * OCR-to-rawIDM conversion.
 * 
 * @author zeil
 *
 */
public class SegmentationTransformer extends QuickTransformer {

    
    /**
     * 
     */
    public SegmentationTransformer() {
        super();
    }

    /* (non-Javadoc)
     * @see edu.odu.cs.extract.dataflow.ThreadedTransformer#doTransform(edu.odu.cs.extract.dataflow.Dataflow[])
     */
    @Override
    public TransformationResult doTransform(Dataflow[] in) throws Exception {
        String status = "success";
        String message = "OK";

        IDMDataflow inputDF = (IDMDataflow) in[0];
        Document unsegmentedIDM = inputDF.getDocument();
        String mergeFailed = unsegmentedIDM.getRootElement().getAttributeValue("OCRmerge");
        
        if (mergeFailed != null && "failed".equals(mergeFailed)) {
            status = "warning";
            message = "unable to merge pages from OCR";
        }
        
        // Segment document
        Document segmentedIDM = new Segmentation(unsegmentedIDM).reSegment();

        IDMDataflow outputDF = new IDMDataflow (in[0].getTrace(), segmentedIDM);
        
        return new TransformationResult(outputDF,status, message, null);
    }

    @Override
    public String getOutputExtension() {
        Properties p = Properties.getProperties();
        return p.getProperty(Properties.Names.SEGMENTATION_OUT_EXT);
    }

        

}

In addition to “free-form” text, can contain special markup

Common Javadoc Markup

@author authorName
@version versionNumber
@param name description
@return description
@throws exceptionClassName description
@see crossReference

Running javadoc

Command line
```
javadoc -d destinationDir -sourcepath sourceCodeDir \
    -link http://docs.oracle.com/javase/7/docs/api/
```
- Can add multiple source paths, links to external libraries
- Can also specify which packages from source code to document
Eclipse: Project ⇒ Generate Javadoc ...
ant

<javadoc packagenames="edu.odu.cs.*" 
         destdir="target/javadoc" 
         classpathref="javadoc.classpath" Author="yes" 
         Version="yes" Use="yes" defaultexcludes="yes">
   <fileset dir="." defaultexcludes="yes">
      <include name="extractor/src/main/java/**" />
      <include name="generatedSource/gen-src/**" />
      <exclude name="**/*.html" />
   </fileset>
   <doctitle><![CDATA[<h1>ODU CS Extract
                    Project</h1>]]></doctitle>
</javadoc>

2.2 doxygen

the most popular API generator for C/C++
- Also works with Objective-C, C#, Java, IDL, Python, PHP, VHDL, and FORTRAN
Markup is essentially identical to javadoc
- Internal formatting available via Markdown
Output can be HTML, LaTeX, or RTF
Can also generate
- various non-quite-UML diagrams
- and hyperlinked source code

Running doxygen

Command line
```
doxygen configFile
```
The config file can contain any of a bewildering set of options in typical property-file style:

PROJECT_NAME = C++ Spreadsheet INPUT = src/model OUTPUT_DIRECTORY = target/doc EXTRACT_ALL = YES CLASS_DIAGRAMS = YES GENERATE_HTML = YES GENERATE_LATEX = YES USE_PDFLATEX = YES

Eclipse: Eclox plugin

Ant (3rd-party contributed task)

2.3 Other API Documentation Generators

Because a documentation generator needs to module and function structure and function parameters, a distinct parser is needed for each programming language.

This leads to a variety of language-specific tools, e.g.,

jsDoc for Javascript

YARD for Ruby

sandcastle for .Net

3. Project Reports

3.1 Test Reports

We’ve already looked JUnit, which can be used to generate test reports like this one.

This is generated in ant via the junitreport task:

junitreport.xml.listing
<project name="code2html" basedir="." default="build"> <record name="ant.log" action="start" append="false" /> <taskdef classpath="JFlex.jar" classname="JFlex.anttask.JFlexTask" name="jflex" /> <echo>loading build-${os.name}.paths</echo> <include file="build-${os.name}.paths"/> <target name="generateSource"> <mkdir dir="src/main/java"/> <jflex file="src/main/jflex/code2html.flex" destdir="src/main/java"/> <jflex file="src/main/jflex/code2tex.flex" destdir="src/main/java"/> <jflex file="src/main/jflex/list2html.flex" destdir="src/main/java"/> <jflex file="src/main/jflex/list2tex.flex" destdir="src/main/java"/> </target> <target name="compile" depends="generateSource"> <mkdir dir="target/classes"/> <javac srcdir="src/main/java" destdir="target/classes" source="1.6" includeantruntime="false"/> </target> <target name="compile-tests" depends="compile"> <mkdir dir="target/test-classes"/> <javac srcdir="src/test/java" destdir="target/test-classes" source="1.6" includeantruntime="false"> <classpath refid="testCompilationPath"/> </javac> </target> <target name="test" depends="compile-tests"> <property name="mypath" refid="testExecutionPath"/> <echo>testExecutioPath is ${mypath}</echo> <echoproperties/> <mkdir dir="target/test-results/details"/> <junit printsummary="yes" haltonfailure="yes" fork="no" > <classpath refid="testExecutionPath"/> <formatter type="xml"/> <batchtest todir="target/test-results/details"> <fileset dir="target/test-classes"> <include name="**/*Test*.class"/> </fileset> </batchtest> </junit> <junitreport todir="target/test-results"> <fileset dir="target/test-results/details"> <include name="TEST-*.xml"/> </fileset> <report format="frames" todir="target/test-results/html"/> </junitreport> </target> <target name="build" depends="test"> <jar destfile="codeAnnotation.jar" basedir="target/classes"> <manifest> <attribute name="Main-Class" value="edu.odu.cs.code2html.Code2HTML"/> </manifest> </jar> </target> <target name="clean"> <delete dir="target"/> </target> </project>

Other common test reports

Javadoc of unit test code

Coverage reports

3.2 Static Code Analyzers

Many tools that we will cover later for analyzing code can produce useful (or at least, impressive) documentation as a side effect.

Example

3.3 Configuration Reports

Configuration managers (to be covered later) generate reports about the dependencies among the software components.

Examples:

Maven

Ivy

4. Project Websites

Traditionally hand-constructed

Or “grown” (Wikis)

Some build managers will generate websites linking together reports

Example

4.1 Publishing to a website

You can also add instructions to your build manager to post files to a website.

Sync to a website on a local file system:

<target name="deploy" depends="build" description="Sync with the website directory" > <sync todir="${deploymentDestination}/reports" includeEmptyDirs="true" granularity="2000"> <fileset dir="target/reports"> <include name="**/*.html"/> <include name="**/*.png"/> <exclude name="**/*.xml"/> </fileset> <preserveintarget> <include name="**/.ht*"/> </preserveintarget> </sync> </target>

Posting to a website

A website on a remote machine:

<target name="publish-reports" depends="reports" description="send project reports to web server"> <tar destfile="target/project-reports.tz" compression="gzip"> ➊ <tarfileset dir="target"> <include name="project-reports/**/*"/> </tarfileset> </tar> <input message="login name for ${webserver}:" addproperty="scp.login"/> ➋ <input message="password for ${webserver}:" addproperty="scp.password"/> <scp file="target/project-reports.tz" sftp="true" ➌ remoteToDir="${scp.login}:${scp.password}@${webserver}:${website.path}"/> <sshexec host="${webserver}" username="${scp.login}" password="${scp.password}" command="cd ${website.path}; tar xzf project-reports.tz"/> ➍ </target>

➊ Packs the reports into a single archive file

➋ Prompts the developer for login info on the remote machine

Could be eliminated if you use an ssh key agent

➌ Copies the archive to the remote machine with scp

➍ Uses ssh to issue an command on the remote machine, unpacking the transferred archive.

4.2 Forges

A software forge is a collection of web services for the support of collaborative software devlopment:

Project web sites

Networked access to version control

Release (download) support

Communications (e.g., messaging, wikis, announcements)

Bug reporting and tracking

Project personnel management

Forge Examples

Among the best known forges are

the original, SourceForge, (1999)

Google Code, (2006)

GitHub, (2008)

The CS 350 course has its own forge

GitLab for project management, version control, issue tracking, wiki

Archiva for binary libraries

Jenkins for continuous integration

Documentation and Documentation Generators

Steven J Zeil

1. Source Code Documentation

1.1 Comments

Do Comments Matter?

Which is better?

Which is better?

Which is better?

Which is better?

1.2 Charting

From Code to Charts

From Charts to Code

2. API Documentation

2.1 javadoc

Javadoc Comments

2.2 doxygen

2.3 Other API Documentation Generators

3. Project Reports

3.1 Test Reports

3.2 Static Code Analyzers

3.3 Configuration Reports

4. Project Websites

4.1 Publishing to a website

4.2 Forges