1 Representing Programs

Most static analysis is based upon one of these graphs

• Abstract syntax trees

• Control Flow Graphs

1.1 Abstract Syntax Trees (ASTs)

• Output of a language parser

  • Simpler than parse trees

• Generally viewed as a generalization of operator-applied-to-operands

• ASTs can be applied to larger constructions than just expressions

1.1.1 Abstract Syntax Graphs

• In most programming languages, any given variable name (e.g., i, x) could actually refer to many different objects depending upon the scope rules of the language.
• The semantic analysis portion of a compiler pairs uses of variable names with the corresponding declarations
  • What we have left is no longer a tree, but a graph.

1.2 Control Flow Graphs

Represent each executable statement in the code as a node,

• with edges connecting nodes that can be executed one after another.
  • Nodes for conditional statements have two or more outgoing edges.

1.2.1 Sample CFG

01: procedure SQRT (Q, A, B: in float;
02:                 X: out float);
03: // Compute X = square root of Q, 
04: //    given that A <= X <= B
05:    X1, F1, F2, H: float;
06: begin
07:    X1 := A;
08:    X2 := B;
09:    F1 := Q - X1**2
10:    H := X2 - X1;
11:    while (ABS(H) >= 0.001) loop
12:       F2 := Q - X2**2;
13:       H := - F2 * ((X2-X1)/(F2-F1));
14:       X1 := X2;
15:       X2 := X2 + H;
16:       F1 := F2
17:    end loop;
18:    X := (X1 + X2) / 2.;
19: end SQRT;

procedure SQRT (Q, A, B: in float; //  node 0 
                X: out float);
// Compute X = square root of Q, 
//    given that A <= X <= B
   X1, F1, F2, H: float;
begin
   X1 := A;
   X2 := B;                        // node 1 
   F1 := Q - X1**2
   H := X2 - X1;
   while (ABS(H) >= 0.001) loop    // node 2 
      F2 := Q - X2**2;
      H := - F2 * ((X2-X1)/(F2-F1));
      X1 := X2;                    // node 3
      X2 := X2 + H;
      F1 := F2
   end loop;
   X := (X1 + X2) / 2.;            // node 4
end SQRT;                          // node 5

1.2.2 Data Flow Analysis

• All data-flow information is obtained by propagating data flow markers through the program.

• The usual markers are

  • di(x) : a definition of variable x (any location where x is assigned a value) at node i
  • r<sub>i</sub>(x): a reference to x (any location where the value of x is used) at node i
  • u<sub>i</sub>(x): an undefinition of x (any location where x becomes undefined/illegal) at node i

procedure SQRT (Q, A, B: in float; //  node 0 
                X: out float);
// Compute X = square root of Q, 
//    given that A <= X <= B
   X1, F1, F2, H: float;
begin
   X1 := A;
   X2 := B;                        // node 1 
   F1 := Q - X1**2
   H := X2 - X1;
   while (ABS(H) >= 0.001) loop    // node 2 
      F2 := Q - X2**2;
      H := - F2 * ((X2-X1)/(F2-F1));
      X1 := X2;                    // node 3
      X2 := X2 + H;
      F1 := F2
   end loop;
   X := (X1 + X2) / 2.;            // node 4
end SQRT;                          // node 5

1.2.3 Reaching Definitions

A definition di(x) reaches a node nj iff there exists a path from ni to nj on which x is neither defined nor undefined.

What definitions reach the reference to X1 in node 4?

What definitions reach the reference to H in node 2?

1.2.4 Data Flow Anomalies

The reaching definitions problem can be used to detect anomolous patterns that may reflect errors.

• ur anomalies: if an undefinition of a variable reaches a reference of the same variable

• dd anomalies: if a definition of a variable reaches a definition of the same variable

• du anomalies: if a definition of a variable reaches an undefinition of the same variable

2 Style and Anomaly Checking

A common form of static analysis:

2.2 Static Analysis by Compilers

• Over time, compilers offer more and more static analysis features.

  • E.g., GNU g++

• One caution is that these are often not turned on by default, but need to be added as command line flags.

  • IDEs often do not use these flags by default.

2.3.1 Example: Adding Checkstyle to gradle

2.4.1 What Bugs does FindBugs Find?

Unlike Checkstyle, FindBugs goes well beyond cosmetics:

2.4.2 Example: Adding Findbugs to gradle

2.5.1 SpotBugs in Gradle

In build.gradle:

buildscript {
    repositories {
       jcenter()
       maven {
          url "https://plugins.gradle.org/m2/"  // for spotbugs
       }
    }
    dependencies {
        ⋮
       classpath "gradle.plugin.com.github.spotbugs:gradlePlugin:1.6.0"   
    }
}

apply plugin: 'com.github.spotbugs'

spotbugsMain {
    ignoreFailures = true
    effort = 'max'
    reportLevel = 'medium'
    reports {
       xml.enabled = false
       html.enabled = true
    }
}

spotbugsTest.enabled = false

2.6.1 PMD Reports

2.6.2 Example: Adding PMD to gradle

• In build.gradle:

  apply plugin: 'pmd'
  ⋮
  pmd {
      ignoreFailures = true
      consoleOutput = false
  }
  pmdTest.enabled = false
  

  • Report is generated on target check.

3 Reverse-Engineering Tools

Reverse engineering makes heavy use of static analysis, and is even more closely tied to compiler technology than the tools we have looked at so far.

3.1 Reverse Compilers

a.k.a. “uncompilers”

• Generate source code from object code

• Originally clunky & more of a curiosity than usable tools

  • Improvements based on
    • “deep” knowledge of compilers (aided by increasingly limited field of available compilers)
    • Information-rich object codes (e.g., Java bytecode formats)

• Legitimate uses include

  • reverse-engineering
  • generating input for source-based analysis tools

• But also great tools for plagiarism

3.1.1 Example of Java Decompilation

For example, I might write the following code:

void drawGraphics(Graphics g, Point[] pts)
{
  double xMin = pts[0].x;
  double xMax = pts[0].x;
  double yMin = pts[0].y;
  double yMax = pts[0].y;

• If I compile this with the -g debugging option on (which saves variable names and other internal information so that a debugger can access them), and run any of several well-known decompilers, I would get back the same code, with only formatting changes.

• If I compile this code without debugging info, one well-known decompiler would give me this:

  void drawGraphics(Graphics g, Point[] pts)
  {
    double d0 = pts[0].x;
    double d1 = pts[0].x;
    double d2 = pts[0].y;
    double d3 = pts[0].y;
  

  Compiled Java .class files always preserve the API info.

3.2 Java Obfuscators

Work by a combination of

• Renaming variables, functions, and classes to meaningless, innocuous, and very similar name sets

  • Challenge is to preserve those names of entry points needed to execute a program or applet or make calls upon a library’s public API

• Stripping away debugging information (e.g., source code file names and line numbers associated with blocks of code)

• Applying optimization techniques to reduce code size while also confusing the object-to-source mapping

• Replacing some expressions by calls to “dummy” functions that actually simply compute the replaced expression.

3.3 Obfuscation Example

Example, given the compiled code from

void drawGraphics(Graphics g, Point[] pts)
{
  double xMin = pts[0].x;
  double xMax = pts[0].x;
  double yMin = pts[0].y;
  double yMax = pts[0].y;

4 Dynamic Analysis Tools

Not all useful analysis can be done statically

• Profiling

• Memory leaks, corruption, etc.

• Data structure abuse

4.1 Pointer/Memory Errors

Memory Abuse

• Pointer errors in C++ are both common and frustrating

  • Traditionally unchecked by standard run-time systems

• Monitors can be added to help catch these

  • In C++, link in a replacement for malloc & free