Debugging

Steven Zeil

Last modified: Jul 15, 2014

Contents:
1. You can’t debug what you don’t understand
1.1 Read the Documentation
1.2 Know What’s Reasonable
1.3 Know the Roadmap
2. Make it fail …
2.1 Make it fail Reliably
2.2 Make it fail Easily
2.3 Make it fail Visibly
3. Example: the Auction Program
3.1 Simplifying the Failing Tests
3.2 Instrument the Code
3.3 Working Backwards
4. Lessons

Testing versus Debugging


Rules of Debugging

  1. You can’t debug what you don’t understand

  2. Make it fail…

    • Make it fail reliably

    • Make it fail easily

    • Make it fail visibly

  3. Track down the culprit

    • Don’t Guess, Hypothesize

    • Divide and Conquer

    • If you didn’t fix it, it ain’t fixed!

In this lesson we’ll focus on the first two

1. You can’t debug what you don’t understand

How can you hope to fix (or even recognize) what is wrong if you don’t know what it will look like when it’s right?

1.1 Read the Documentation

1.2 Know What’s Reasonable

Engineers often talk about “sanity checks” and “back-of-the-envelope” estimates. These refer to quick considerations of what is actually a reasonable result in a given situation.

So if you see something that isn’t reasonable, your first responses should be to mistrust the calculations (or, in our case, the code) that produced that “obviously” unreasonable behavior.

Where do we get the knowledge of what is and isn’t reasonable?

1.3 Know the Roadmap

Large programs are made of many components. When you see an output that is incorrect, your debugging will be much more efficient if you have an idea which of those components contributed to that particular output.

Example: if you were debugging this program after seeing an identical mistake in both reports, would you look first at the report generators or would you concentrate on the “shared” input processing and main calculation?

2. Make it fail …

Typically you start debugging because your program failed

What’s the next thing you need to do?


Make It Fail Again

Once You’ve Made it Fail …


Why “Make It Fail Again”?

2.1 Make it fail Reliably

Intermittent Failures


Can Hardware Failures Be Truly Intermittent?


Can Software Failures Be Truly Intermittent?


Tracking Down Intermittent Failures


What Causes Intermittent Software Failures?

Typically,


Uninitialized variables


Out of bounds data access


Pointer errors - dangling pointers

Dangling pointers

We saw earlier that if bc is destroyed, …

we are left with dangling pointers.


Pointer errors - Deleting a pointer twice

Question: What happens if all the collections were destroyed and they each deleted the pointer?

Answer

2.2 Make it fail Easily

Debugging often requires you to run the failing test cases over and over again.

Simplify Your Failing Tests


Simplify, Simplify, Simplify

If you made the program fail on the 100th input or 100th time around a loop, can you find a simpler test that fails after the 10th time?

Shortening that loop gives you less to debug.


KISS

Later we will make a distinction between unit testing and system testing,


If It Takes Too Long to Fail…

Example: When Do Pointer Errors Fail?


Example: When Do Pointer Errors Fail? (cont.)


Tracking Pointer Errors - Redefining Failure

2.3 Make it fail Visibly


Program States

A program state is the combination of


Infected States

A program state is infected if one of its components is affected by a bug :


Propagating an Infection

As we move forward through the code, an infected state can


From Infection to Failure

A failure occurs when an infected state affects (propagates to) the program output.

Instrument the System


Design Instrumentation In

Good rule of thumb for C++ ADT designers:

Always provide an output function for your ADT, showing the value of each data member.


Build Instrumentation In Later

debugOutputDemo.cpp


Never Throw a Debugging Tool Away

When you have finished debugging a problem, deactivate your debug code, don’t delete it!

debugOutputDemo2.cpp


The Heisenberg Uncertainty Principle

Debugging instrumentation may affect the behavior of the bug

3. Example: the Auction Program

As last seen: Auction ADTs


Failure Report

During an early run, the seller of a Chinese Vase complained that a bid for $27 was accepted as a winner even though a reserve price of $50 had been set.

3.1 Simplifying the Failing Tests


Rounding Up the Suspects

We have our test case!

3.2 Instrument the Code

The most obvious place to check for this problem is in the resolveAuction function

3.3 Working Backwards

Problem appears to be in the selection of winningBidderSoFar and highestBidSoFar.

resolveAuction1.cpp

 These are initialized outside (before) the loop and reset
  inside, so we might want to check that the values are reasonable when
  they are reset.

When a New Best Bid is Found

Adding debugging output:

resolveAuction2.cpp

gives us the output:

Best bid so far is 26 from jjones
Hummel Figurine won by jjones for 26
Best bid so far is 27 from ssmith
Chinese Vase won by ssmith for 27

which does not really tell us much.


Adding Detail

resolveAuction3.cpp

gives us the output:

Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
Hummel Figurine won by jjones for 26
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Chinese Vase won by ssmith for 27

all of which looks OK.


Adding Detail the Easy Way

resolveAuction4.cpp

For this to work, though, we need to add the print functions to bids and bidders (that we should probably have had there in the first place.)


Auction with Output Functions

  • The Bid ADT:

auction/bids.h

and

auction/bids.cpp

  • The Bidder ADT:

auction/bidders.h

and

auction/bidders.cpp

  • The Item ADT:

auction/items.h

and

auction/items.cpp

  • The Time ADT:

auction/time.h

and

auction/time.cpp

  • The BidCollection ADT:

auction/bidcollection.h

and

auction/bidcollection.cpp

  • The BidderCollection ADT:

auction/biddercollection.h

and

auction/biddercollection.cpp

  • The ItemCollection ADT:

auction/itemcollection.h

and

auction/itemcollection.cpp


Looking at All Bids

Since the determination of the best bid is not obviously wrong, we might wonder if all the non-best bids are being handled correctly. In particular, we recall that there were more than two bids for the two items.

resolveAuction5.cpp

gives output

resolving item [Hummel Figurine 25 15:30:11]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Bidder is [jjones 75]
Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Hummel Figurine won by jjones for 26
resolving item [Chinese Vase 50 20:06:27]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Bidder is [jjones 49]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Bidder is [ssmith 55]
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Chinese Vase won by ssmith for 27

It’s still not clear why the low bid by smith is being accepted, but we can see that all bids are in fact being processed in some fashion.

Since we are not getting enough info about the reserve, let’s also monitor when the reservePriceMet flag changes.


When Does reservePriceMet change?

resolveAuction6.cpp

which has output

resolving item [Hummel Figurine 25 15:30:11]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Bidder is [jjones 75]
Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
reserve price met by bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
reserve price met by bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
reserve price met by bid [ssmith 27 Chinese Vase 04:03:05]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Hummel Figurine won by jjones for 26
resolving item [Chinese Vase 50 20:06:27]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Bidder is [jjones 49]
reserve price met by bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Bidder is [ssmith 55]
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Chinese Vase won by ssmith for 27

Diagnosis

Looking at that output:

debugOutput.listing

Now we can see, not just one, but two problems.

So the condition

  if (bid.getAmount() > item.getReservedPrice()) 
    {
      reservePriceMet = true;

is far too lax.


Diagnosis Precedes the Cure

At this point, we have diagnosed the problem.

auction/auctionMain.cpp

4. Lessons