Debugging

Steven Zeil

Last modified: Dec 26, 2016
Contents:

1 Testing versus Debugging

2 Rules of Debugging

  1. You can’t debug what you don’t understand

  2. Make it fail…

    • Make it fail reliably

    • Make it fail easily

    • Make it fail visibly
  3. Track down the culprit

    • Don’t Guess, Hypothesize

    • Divide and Conquer

    • If you didn’t fix it, it ain’t fixed!

In this lesson we’ll focus on the first two

3 You can’t debug what you don’t understand

How can you hope to fix (or even recognize) what is wrong if you don’t know what it will look like when it’s right?

3.1 Read the Documentation

3.2 Know What’s Reasonable

Engineers often talk about “sanity checks” and “back-of-the-envelope” estimates. These refer to quick considerations of what is actually a reasonable result in a given situation.

So if you see something that isn’t reasonable, your first responses should be to mistrust the calculations (or, in our case, the code) that produced that “obviously” unreasonable behavior.

Where do we get the knowledge of what is and isn’t reasonable?

3.3 Know the Roadmap

Large programs are made of many components. When you see an output that is incorrect, your debugging will be much more efficient if you have an idea which of those components contributed to that particular output.

 

Example: if you were debugging this program after seeing an identical mistake in both reports, would you look first at the report generators or would you concentrate on the “shared” input processing and main calculation?

4 Make it fail …

Typically you start debugging because your program failed

What’s the next thing you need to do?

4.1 Make It Fail Again

Once You’ve Made it Fail …

4.2 Why “Make It Fail Again”?

4.3 Make it Fail Reliably

4.3.1 Intermittent Failures


Can Hardware Failures Be Truly Intermittent?


Can Software Failures Be Truly Intermittent?


Tracking Down Intermittent Failures


What Causes Intermittent Software Failures?

Typically,


Uninitialized variables


Out of bounds data access


Pointer errors - dangling pointers

 

Dangling pointers

We saw earlier that if bc is destroyed, …

 

we are left with dangling pointers.


Pointer errors - Deleting a pointer twice

 

Question: What happens if all the collections were destroyed and they each deleted the pointer?

**Answer**

4.4 Make it fail Easily

Debugging often requires you to run the failing test cases over and over again.

4.4.1 Simplify Your Failing Tests


Simplify, Simplify, Simplify

 

If you made the program fail on the 100th input or 100th time around a loop, can you find a simpler test that fails after the 10th time?

Shortening that loop gives you less to debug.


KISS

 

Later we will make a distinction between unit testing and system testing,


If It Takes Too Long to Fail…

4.4.2 Example: When Do Pointer Errors Fail?

 


Example: When Do Pointer Errors Fail? (cont.)

 


Tracking Pointer Errors - Redefining Failure

4.5 Make it fail Visibly


Program States

A program state is the combination of


Infected States

A program state is infected if one of its components is affected by a bug :


Propagating an Infection

As we move forward through the code, an infected state can


From Infection to Failure

A failure occurs when an infected state affects (propagates to) the program output.

4.5.1 Instrument the System


Design Instrumentation In

Good rule of thumb for C++ ADT designers:

Always provide an output function for your ADT, showing the value of each data member.


Build Instrumentation In Later


Never Throw a Debugging Tool Away

When you have finished debugging a problem, deactivate your debug code, don’t delete it!

debugOutputDemo2.cpp
#include <iostream>

using namespace std;


#ifdef DEBUG
#define DEBUGOUT(x) cerr << #x << "@" << __FILE__ << ":" << __LINE__ \
   << ": " << x << endl;
#else
#define DEBUGOUT(x) // ignored if not debugging
#endif

int main() 
{
  int k = 0;
  string y = "hello";
  cout << "Real output " << k << endl;
#ifdef DEBUG
  cerr << "Debugging output " << k+1 << endl;
#endif
  cout << "Real output " << y << endl;
#ifndef NDEBUG
  cerr << "I suspect " << y << " is incorrect at "
       << __FILE__ << ":" << __LINE__ << endl;
#endif
  cout << "Real output " << k << endl;
  DEBUGOUT (k);
  DEBUGOUT (y);
  DEBUGOUT (k + k);  
  DEBUGOUT (y.size());
  return 0;
}

The Heisenberg Uncertainty Principle

Debugging instrumentation may affect the behavior of the bug

5 Example: the Auction Program

As last seen: Auction ADTs


Failure Report

During an early run, the seller of a Chinese Vase complained that a bid for \$27 was accepted as a winner even though a reserve price of \$50 had been set.

5.1 Simplifying the Failing Tests


Rounding Up the Suspects

We have our test case!

5.2 Instrument the Code

The most obvious place to check for this problem is in the resolveAuction function

5.3 Working Backwards

Problem appears to be in the selection of winningBidderSoFar and highestBidSoFar.


When a New Best Bid is Found

Adding debugging output:

resolveAuction2.cpp
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids)
{
  double highestBidSoFar = 0.0;
  string winningBidderSoFar;
  bool reservePriceMet = false;
  for (int bidNum = 0; bidNum < bids.getSize(); ++bidNum)
    {
      Bid bid = bids.get(bidNum);
      if (bid.getTimePlacedAt().noLaterThan(item.getAuctionEndTime()))
        {
          if (bid.getItem() == item.getName()
              && bid.getAmount() > highestBidSoFar
              )
            {
              int bidderNum = bidders.findBidder(bid.getBidder());
              Bidder bidder = bidders.get(bidderNum);
              // Can this bidder afford it?
              if (bid.getAmount() <= bidder.getBalance())
                {
                  highestBidSoFar = bid.getAmount();
                  winningBidderSoFar = bid.getBidder();
                  cerr << "Best bid so far is "
		       << highestBidSoFar << " from "
                         << winningBidderSoFar << endl;
                }
            }
            if (bid.getAmount() > item.getReservedPrice())
                reservePriceMet = true;
        }
    }

    // If highestBidSoFar is non-zero, we have a winner
    if (reservePriceMet && highestBidSoFar > 0.0)
      {
        int bidderNum = bidders.findBidder(winningBidderSoFar);
        cout << item.getName()
             << " won by " << winningBidderSoFar
             << " for " << highestBidSoFar << endl;
        Bidder& bidder = bidders.get(bidderNum);
        bidder.setBalance (bidder.getBalance() - highestBidSoFar);
      }
    else
      {
        cout << item.getName()
             << " reserve not met"
             << endl;
      }
}

gives us the output:

Best bid so far is 26 from jjones
Hummel Figurine won by jjones for 26
Best bid so far is 27 from ssmith
Chinese Vase won by ssmith for 27

which does not really tell us much.


Adding Detail

resolveAuction3.cpp
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids)
{
  double highestBidSoFar = 0.0;
  string winningBidderSoFar;
  bool reservePriceMet = false;
  for (int bidNum = 0; bidNum < bids.getSize(); ++bidNum)
    {
      Bid bid = bids.get(bidNum);
      if (bid.getTimePlacedAt().noLaterThan(item.getAuctionEndTime()))
        {
          if (bid.getItem() == item.getName()
              && bid.getAmount() > highestBidSoFar
              )
            {
              int bidderNum = bidders.findBidder(bid.getBidder());
              Bidder bidder = bidders.get(bidderNum);
              // Can this bidder afford it?
              if (bid.getAmount() <= bidder.getBalance())
                {
                  highestBidSoFar = bid.getAmount();
                  winningBidderSoFar = bid.getBidder();
                  cerr << "Best bid so far is ["
		       << bid.getBidder() << " "
		       << bid.getAmount() << " "
		       << bid.getItem() << " ";
		  bid.getTimePlacedAt().print(cerr);
		  cerr << "] from ["
		       << bidder.getName() << " "
		       << bidder.getBalance() << "]"
		       << endl;
                }
            }
            if (bid.getAmount() > item.getReservedPrice())
                reservePriceMet = true;
        }
    }

    // If highestBidSoFar is non-zero, we have a winner
    if (reservePriceMet && highestBidSoFar > 0.0)
      {
        int bidderNum = bidders.findBidder(winningBidderSoFar);
        cout << item.getName()
             << " won by " << winningBidderSoFar
             << " for " << highestBidSoFar << endl;
        Bidder& bidder = bidders.get(bidderNum);
        bidder.setBalance (bidder.getBalance() - highestBidSoFar);
      }
    else
      {
        cout << item.getName()
             << " reserve not met"
             << endl;
      }
}

gives us the output:

Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
Hummel Figurine won by jjones for 26
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Chinese Vase won by ssmith for 27

all of which looks OK.


Adding Detail the Easy Way

resolveAuction4.cpp
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids)
{
  double highestBidSoFar = 0.0;
  string winningBidderSoFar;
  bool reservePriceMet = false;
  for (int bidNum = 0; bidNum < bids.getSize(); ++bidNum)
    {
      Bid bid = bids.get(bidNum);
      if (bid.getTimePlacedAt().noLaterThan(item.getAuctionEndTime()))
        {
          if (bid.getItem() == item.getName()
              && bid.getAmount() > highestBidSoFar
              )
            {
              int bidderNum = bidders.findBidder(bid.getBidder());
              Bidder bidder = bidders.get(bidderNum);
              // Can this bidder afford it?
              if (bid.getAmount() <= bidder.getBalance())
                {
                  highestBidSoFar = bid.getAmount();
                  winningBidderSoFar = bid.getBidder();
                  cerr << "Best bid so far is ";
		  bid.print (cerr);
		  cerr << " from ";
		  bidder.print (cerr);
		  cerr << endl;
                }
            }
            if (bid.getAmount() > item.getReservedPrice())
                reservePriceMet = true;
        }
    }

    // If highestBidSoFar is non-zero, we have a winner
    if (reservePriceMet && highestBidSoFar > 0.0)
      {
        int bidderNum = bidders.findBidder(winningBidderSoFar);
        cout << item.getName()
             << " won by " << winningBidderSoFar
             << " for " << highestBidSoFar << endl;
        Bidder& bidder = bidders.get(bidderNum);
        bidder.setBalance (bidder.getBalance() - highestBidSoFar);
      }
    else
      {
        cout << item.getName()
             << " reserve not met"
             << endl;
      }
}

For this to work, though, we need to add the print functions to bids and bidders (that we should probably have had there in the first place.)


Auction with Output Functions


Looking at All Bids

Since the determination of the best bid is not obviously wrong, we might wonder if all the non-best bids are being handled correctly. In particular, we recall that there were more than two bids for the two items.

resolveAuction5.cpp
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids)
{
  cerr << "resolving item ";
  item.print(cerr);
  cerr << endl;
  double highestBidSoFar = 0.0;
  string winningBidderSoFar;
  bool reservePriceMet = false;
  for (int bidNum = 0; bidNum < bids.getSize(); ++bidNum)
    {
      Bid bid = bids.get(bidNum);
      cerr << "Looking at bid ";
      bid.print (cerr);
      cerr << endl;
      if (bid.getTimePlacedAt().noLaterThan(item.getAuctionEndTime()))
        {
          if (bid.getItem() == item.getName()
              && bid.getAmount() > highestBidSoFar
              )
            {
              int bidderNum = bidders.findBidder(bid.getBidder());
              Bidder bidder = bidders.get(bidderNum);
	      cerr << "Bidder is ";
	      bidder.print (cerr);
	      cerr << endl;
              // Can this bidder afford it?
              if (bid.getAmount() <= bidder.getBalance())
                {
                  highestBidSoFar = bid.getAmount();
                  winningBidderSoFar = bid.getBidder();
		  cerr << "Best bid so far is ";
		  bid.print (cerr);
		  cerr << " from ";
		  bidder.print (cerr);
		  cerr << endl;
                }
            }
            if (bid.getAmount() > item.getReservedPrice())
                reservePriceMet = true;
        }
    }

    // If highestBidSoFar is non-zero, we have a winner
    if (reservePriceMet && highestBidSoFar > 0.0)
      {
        int bidderNum = bidders.findBidder(winningBidderSoFar);
        cout << item.getName()
             << " won by " << winningBidderSoFar
             << " for " << highestBidSoFar << endl;
        Bidder& bidder = bidders.get(bidderNum);
        bidder.setBalance (bidder.getBalance() - highestBidSoFar);
      }
    else
      {
        cout << item.getName()
             << " reserve not met"
             << endl;
      }
}

gives output

resolving item [Hummel Figurine 25 15:30:11]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Bidder is [jjones 75]
Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Hummel Figurine won by jjones for 26
resolving item [Chinese Vase 50 20:06:27]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Bidder is [jjones 49]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Bidder is [ssmith 55]
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Chinese Vase won by ssmith for 27

It’s still not clear why the low bid by smith is being accepted, but we can see that all bids are in fact being processed in some fashion.

Since we are not getting enough info about the reserve, let’s also monitor when the reservePriceMet flag changes.


When Does reservePriceMet change?

resolveAuction6.cpp
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids)
{
  cerr << "resolving item ";
  item.print(cerr);
  cerr << endl;
  double highestBidSoFar = 0.0;
  string winningBidderSoFar;
  bool reservePriceMet = false;
  for (int bidNum = 0; bidNum < bids.getSize(); ++bidNum)
    {
      Bid bid = bids.get(bidNum);
      cerr << "Looking at bid ";   
      bid.print (cerr);
      cerr << endl;
      if (bid.getTimePlacedAt().noLaterThan(item.getAuctionEndTime()))
        {
          if (bid.getItem() == item.getName()
              && bid.getAmount() > highestBidSoFar
              )
            {
              int bidderNum = bidders.findBidder(bid.getBidder());
              Bidder bidder = bidders.get(bidderNum);
	      cerr << "Bidder is ";
	      bidder.print (cerr);
	      cerr << endl;
              // Can this bidder afford it?
              if (bid.getAmount() <= bidder.getBalance())
                {
                  highestBidSoFar = bid.getAmount();
                  winningBidderSoFar = bid.getBidder();
		  cerr << "Best bid so far is ";
		  bid.print (cerr);
		  cerr << " from ";
		  bidder.print (cerr);
		  cerr << endl;
                }
            }
            if (bid.getAmount() > item.getReservedPrice()) 
	      {
                reservePriceMet = true;
                cerr << "reserve price met by bid ";
		bid.print (cerr);
		cerr << endl;
	      }
        }
    }

    // If highestBidSoFar is non-zero, we have a winner
    if (reservePriceMet && highestBidSoFar > 0.0)
      {
        int bidderNum = bidders.findBidder(winningBidderSoFar);
        cout << item.getName()
             << " won by " << winningBidderSoFar
             << " for " << highestBidSoFar << endl;
        Bidder& bidder = bidders.get(bidderNum);
        bidder.setBalance (bidder.getBalance() - highestBidSoFar);
      }
    else
      {
        cout << item.getName()
             << " reserve not met"
             << endl;
      }
}

which has output

resolving item [Hummel Figurine 25 15:30:11]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Bidder is [jjones 75]
Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
reserve price met by bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
reserve price met by bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
reserve price met by bid [ssmith 27 Chinese Vase 04:03:05]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Hummel Figurine won by jjones for 26
resolving item [Chinese Vase 50 20:06:27]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Bidder is [jjones 49]
reserve price met by bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Bidder is [ssmith 55]
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Chinese Vase won by ssmith for 27

Diagnosis

Looking at that output:

debugOutput.listing
resolving item [Hummel Figurine 25 15:30:11]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Bidder is [jjones 75]
Best bid so far is [jjones 26 Hummel Figurine 00:18:03] from [jjones 75]
reserve price met by bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
reserve price met by bid [jjones 60 Chinese Vase 04:03:01]
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
reserve price met by bid [ssmith 27 Chinese Vase 04:03:05] ➀
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Hummel Figurine won by jjones for 26
resolving item [Chinese Vase 50 20:06:27]
Looking at bid [jjones 26 Hummel Figurine 00:18:03]
Looking at bid [jjones 60 Chinese Vase 04:03:01]
Bidder is [jjones 49]                                      ➂
reserve price met by bid [jjones 60 Chinese Vase 04:03:01] ➁
Looking at bid [ssmith 27 Chinese Vase 04:03:05]
Bidder is [ssmith 55]
Best bid so far is [ssmith 27 Chinese Vase 04:03:05] from [ssmith 55]
Looking at bid [jjones 25 Chinese Vase 05:00:00]
Chinese Vase won by ssmith for 27

Now we can see, not just one, but two problems.

So the condition

  if (bid.getAmount() > item.getReservedPrice()) 
    {
      reservePriceMet = true;

is far too lax.


Diagnosis Precedes the Cure

At this point, we have diagnosed the problem.

../debugging1-auction/auctionMain.cpp
#include <iostream>
#include <string>

using namespace std;

#include "items.h"
#include "itemcollection.h"
#include "bidders.h"
#include "biddercollection.h"
#include "bids.h"
#include "bidcollection.h"
#include "time.h"

const int MaxBidders = 100;
const int MaxBids = 5000;
const int MaxItems = 100;


/**
 * Determine the winner of the auction for item #i.
 * Announce the winner and remove money from winner's account.
 */
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids);


int main (int argc, char** argv)
{
  if (argc != 4)
    {
      cerr << "Usage: " << argv[0] << " itemsFile biddersFile bidsFile" << endl;
      return -1;
    }

  ItemCollection items (MaxItems);
  items.readItems (argv[1]);

  BidderCollection bidders (MaxBidders);
  bidders.readBidders (argv[2]);

  BidCollection bids (MaxBids);
  bids.readBids (argv[3]);

  for (int i = 0; i < items.getSize(); ++i)
    {
      resolveAuction(items.get(i), bidders, bids);
    }
  return 0;
}


/**
 * Determine the winner of the auction for an item.
 * Announce the winner and remove money from winner's account.
 */
void resolveAuction (const Item& item, 
                     BidderCollection& bidders, 
                     BidCollection& bids)
{
#ifdef DEBUG
  cerr << "resolving item ";
  item.print(cerr);
  cerr << endl;
#endif
  double highestBidSoFar = 0.0;
  string winningBidderSoFar;
  bool reservePriceMet = false;
  for (int bidNum = 0; bidNum < bids.getSize(); ++bidNum)
    {
      Bid bid = bids.get(bidNum);
#ifdef DEBUG
      cerr << "Looking at bid ";   
      bid.print (cerr);
      cerr << endl;
#endif
      if (bid.getTimePlacedAt().noLaterThan(item.getAuctionEndTime()))
        {
          if (bid.getItem() == item.getName()
              && bid.getAmount() > highestBidSoFar
              )
            {
              int bidderNum = bidders.findBidder(bid.getBidder());
              Bidder bidder = bidders.get(bidderNum);
#ifdef DEBUG
	      cerr << "Bidder is ";
	      bidder.print (cerr);
	      cerr << endl;
#endif
              // Can this bidder afford it?
              if (bid.getAmount() <= bidder.getBalance())
                {
                  highestBidSoFar = bid.getAmount();
                  winningBidderSoFar = bid.getBidder();
#ifdef DEBUG
		  cerr << "Best bid so far is ";
		  bid.print (cerr);
		  cerr << " from ";
		  bidder.print (cerr);
		  cerr << endl;
#endif
		  if (bid.getAmount() > item.getReservedPrice()) 
		    {
		      reservePriceMet = true;
#ifdef DEBUG
		      cerr << "reserve price met by bid ";
		      bid.print (cerr);
		      cerr << endl;
#endif
		    }
                }
            }
        }
    }

    // If highestBidSoFar is non-zero, we have a winner
    if (reservePriceMet && highestBidSoFar > 0.0)
      {
        int bidderNum = bidders.findBidder(winningBidderSoFar);
        cout << item.getName()
             << " won by " << winningBidderSoFar
             << " for " << highestBidSoFar << endl;
        Bidder& bidder = bidders.get(bidderNum);
        bidder.setBalance (bidder.getBalance() - highestBidSoFar);
      }
    else
      {
        cout << item.getName()
             << " reserve not met"
             << endl;
      }
}



6 Lessons