1 Representing Programs

Most static analysis is based upon one of these graphs

• Abstract syntax trees

• Control Flow Graphs

1.1 Abstract Syntax Trees (ASTs)

• Output of a language parser

  • Simpler than parse trees

• Generally viewed as a generalization of operator-applied-to-operands

• ASTs can be applied to larger constructions than just expressions

• In fact, generally reduce entire program or compilation unit to one AST

1.1.1 Abstract Syntax Graphs

• In most programming languages, any given variable name (e.g., i, x) could actually refer to many different objects depending upon the scope rules of the language.
• The semantic analysis portion of a compiler pairs uses of variable names with the corresponding declarations
  • What we have left is no longer a tree, but a graph.

1.2 Control Flow Graphs

Represent each executable statement in the code as a node,

• with edges connecting nodes that can be executed one after another.
  • Nodes for conditional statements have two or more outgoing edges.

1.2.1 Sample CFG

01: procedure SQRT (Q, A, B: in float;
02:                 X: out float);
03: // Compute X = square root of Q,
04: //    given that A <= X <= B
05:    X1, F1, F2, H: float;
06: begin
07:    X1 := A;
08:    X2 := B;
09:    F1 := Q - X1**2
10:    H := X2 - X1;
11:    while (ABS(H) >= 0.001) loop
12:       F2 := Q - X2**2;
13:       H := - F2 * ((X2-X1)/(F2-F1));
14:       X1 := X2;
15:       X2 := X2 + H;
16:       F1 := F2
17:    end loop;
18:    X := (X1 + X2) / 2.;
19: end SQRT;

procedure SQRT (Q, A, B: in float; //  node 0
                X: out float);
// Compute X = square root of Q,
//    given that A <= X <= B
   X1, F1, F2, H: float;
begin
   X1 := A;
   X2 := B;                        // node 1
   F1 := Q - X1**2
   H := X2 - X1;
   while (ABS(H) >= 0.001) loop    // node 2
      F2 := Q - X2**2;
      H := - F2 * ((X2-X1)/(F2-F1));
      X1 := X2;                    // node 3
      X2 := X2 + H;
      F1 := F2
   end loop;
   X := (X1 + X2) / 2.;            // node 4
end SQRT;                          // node 5

1.2.2 Data Flow Analysis

• All data-flow information is obtained by propagating data flow markers through the program.

• The usual markers are

  • di(x) : a definition of variable x (any location where x is assigned a value) at node i
  • r<sub>i</sub>(x): a reference to x (any location where the value of x is used) at node i
  • u<sub>i</sub>(x): an undefinition of x (any location where x becomes undefined/illegal) at node i

procedure SQRT (Q, A, B: in float; //  node 0
                X: out float);
// Compute X = square root of Q,
//    given that A <= X <= B
   X1, F1, F2, H: float;
begin
   X1 := A;
   X2 := B;                        // node 1
   F1 := Q - X1**2
   H := X2 - X1;
   while (ABS(H) >= 0.001) loop    // node 2
      F2 := Q - X2**2;
      H := - F2 * ((X2-X1)/(F2-F1));
      X1 := X2;                    // node 3
      X2 := X2 + H;
      F1 := F2
   end loop;
   X := (X1 + X2) / 2.;            // node 4
end SQRT;                          // node 5

1.2.3 Reaching Definitions

A definition di(x) reaches a node nj iff there exists a path from ni to nj on which x is neither defined nor undefined.

What definitions reach the reference to X1 in node 4?

What definitions reach the reference to H in node 2?

1.2.4 Data Flow Anomalies

The reaching definitions problem can be used to detect anomolous patterns that may reflect errors.

• ur anomalies: if an undefinition of a variable reaches a reference of the same variable

• dd anomalies: if a definition of a variable reaches a definition of the same variable

• du anomalies: if a definition of a variable reaches an undefinition of the same variable

2 Style and Anomaly Checking

A common form of static analysis:

Combines static analysis with style recommendations, e.g.,

• data flow anomalies

• potential arithmetic overflow

  • e.g., storing an int calculation in a char

• conditional statements with constant values

• potential = versus == confusion

• lint was a response, in part, to the weak capabilities of early C compilers

• Much of what lint does is now handled by optimizing compilers

  • However compilers seldom do cross-module or even cross-function analysis

2.2 Static Analysis by Compilers

• Over time, compilers offer more and more static analysis features.

  • E.g., GNU g++

• One caution is that these are often not turned on by default, but need to be added as command line flags.

  • IDEs often do not use these flags by default.

g++ offers several “collections” flags that turn on multiple warnings (which could have been turned on individually).

2.3.1 Example: Adding Checkstyle to gradle

2.4.1 SpotBugs in Gradle

In build.gradle:

plugins {
   id 'java'
   ⋮
   id "com.github.spotbugs" version "4.6.0"
}

spotbugsMain {
    ignoreFailures = true
    effort = 'max'
    reportLevel = 'medium'
    reports {
       xml.enabled = false
       html.enabled = true
    }
}

spotbugsTest.enabled = false

2.5.1 PMD Reports

2.5.2 Example: Adding PMD to gradle

• In build.gradle:

  plugins {
     id 'java'
     ⋮
     id 'pmd'
  }
  
  pmd {
  ignoreFailures = true
      consoleOutput = false
  }
  pmdTest.enabled = false
  

  • Report is generated on target check.

2.5.3 Example: Adding PMD to Eclipse

• Open the Eclipse Marketplace (Help menu) and search for “pmd”
• Select the “pmd-eclipse-plugin” and install
• Open your Java project Properties and look for PMD.
• Select “Enable PMD”, then “Apply and Close”.

  Look at all the colorful flags!

• Open the PMD perspective (Window -> Perspective -> Open Perspective) to examine and manage messages.

  • You can ask for Details, remove a flag, mark a statement as approved under review, or suppress a rule entirely.

2.5.4 Customizing PMD

3 Reverse-Engineering Tools

Reverse engineering makes heavy use of static analysis, and is even more closely tied to compiler technology than the tools we have looked at so far.

3.1 Reverse Compilers

a.k.a. “uncompilers”

• Generate source code from object code

• Originally clunky & more of a curiosity than usable tools

  • Improvements based on
    • “deep” knowledge of compilers (aided by increasingly limited field of available compilers)
    • Information-rich object codes (e.g., Java bytecode formats)

• Legitimate uses include

  • reverse-engineering
  • generating input for source-based analysis tools

• But also great tools for plagiarism

• Java is a particularly friendly field for decompilers
  • Rich object code format
  • Nearly monopolistic compiler suite

3.1.1 Example of Java Decompilation

For example, I might write the following code:

void drawGraphics(Graphics g, Point[] pts)
{
  double xMin = pts[0].x;
  double xMax = pts[0].x;
  double yMin = pts[0].y;
  double yMax = pts[0].y;

• If I compile this with the -g debugging option on (which saves variable names and other internal information so that a debugger can access them), and run any of several well-known decompilers, I would get back the same code, with only formatting changes.

• If I compile this code without debugging info, one well-known decompiler would give me this:

  void drawGraphics(Graphics g, Point[] pts)
  {
    double d0 = pts[0].x;
    double d1 = pts[0].x;
    double d2 = pts[0].y;
    double d3 = pts[0].y;
  

  Compiled Java .class files always preserve the API info.

• Options for “protecting” programs compiled in Java:
  • gjc: compile into native code with a far less popular compiler
  • obfuscators…

3.2 Java Obfuscators

Work by a combination of

• Renaming variables, functions, and classes to meaningless, innocuous, and very similar name sets

  • Challenge is to preserve those names of entry points needed to execute a program or applet or make calls upon a library’s public API

• Stripping away debugging information (e.g., source code file names and line numbers associated with blocks of code)

• Applying optimization techniques to reduce code size while also confusing the object-to-source mapping

• Replacing some expressions by calls to “dummy” functions that actually simply compute the replaced expression.

3.3 Obfuscation Example

Example, given the compiled code from

void drawGraphics(Graphics g, Point[] pts)
{
  double xMin = pts[0].x;
  double xMax = pts[0].x;
  double yMin = pts[0].y;
  double yMax = pts[0].y;

4 Dynamic Analysis Tools

Not all useful analysis can be done statically

• Profiling

• Memory leaks, corruption, etc.

• Data structure abuse

• Traditionally, the C++ standard library does not check for common abuses such as over-filling and array or accessing non-existent elements

  • Various authors have filled in with “checking” implementations of the library for use during testing and debugging

• In a sense, the assert command of C++ and Java is the language’s own extension mechanism for such checks.

4.1 Pointer/Memory Errors

Memory Abuse

• Pointer errors in C++ are both common and frustrating

  • Traditionally unchecked by standard run-time systems

• Monitors can be added to help catch these

  • In C++, link in a replacement for malloc & free

• Use fenceposts around allocated blocks of memory

  • check for unchanged fenceposts to detect over-writes
  • Check for fenceposts before a delete to detect attempts to delete addresses other than the start of an allocated block

• Add tracking info to allocated blocks indicating location of the allocation call

  • Scan heap at end of program for unrecovered blocks of memory
  • Report on locations from which those were allocated

• Add a “freed” bit to allocated blocks that is cleared when first allocated and set when the block is freed

  • Detect when a block is freed twice