Reminder: Instructions for setting up the lab are available here.

Debugging lab

This lab is designed to give you experience with various debugging strategies. The lab files contain a wide variety of mistakes: you get to fix them.

With OCaml, we have three basic debugging tools at our disposal:

Manual Analysis

The first tool is something you probably already use regularly; it is your own power of reasoning! Before you can begin to understand what is wrong with your code, you must be sure that you understand how your code executes in the first place. OCaml works by altering and, eventually, returning a specific value. It is often simple to break down a piece of code and analyze the state of this value after each line. In lecture and class notes we typically explain OCaml functions by breaking them down in this manner.

Example one and problems 1 and 2 in debug_lab.ml deal with this type of analysis.

Testing

Being able to confidently analyze your own code is important but often not sufficient. Particularly for longer programs featuring helper functions, loops, and changes to internal state, line by line analysis can be as impractical as it is daunting.

When it comes to debugging lengthier blocks of code, testing is your greatest friend! Yeah yeah, we know how you all feel about testing. But we don't make you do it just because we find great pleasure in ensuring nobody ever quite gets a 100 on their homework assignments. And neither do we make you do it simply for you to discover all potential bugs in your code. In addition to aiding in the discovery of bugs, good testing can also help you address them!

The first step is understanding what tests really are. Once written, your code is simply a series of instructions for your processor to perform on some given starting parameters. Tests are meant to vary these parameters across the full range of values possible and ensure that your set of instructions still produce the end result you wish to see. Of course, time and patience are finite, whereas ranges of anticipated values can be large, even infinite. Thus you must choose the values you test wisely, being sure that they are exhaustive without being exorbitant in number. This is possible because it is highly likely that certain sets of starting parameters will exercise your code in the exact same way.

One way code can execute in different ways is through branching. In OCaml, branching is typically a result of if/then/else blocks or pattern matching. Some starting parameters will cause our code to choose one branch over another, leaving the “other” branch untested. As a result, one basic metric for exhaustive testing is line coverage. Do your tests force your code to execute every line? If not, entire parts of your code have not even been run!

Problem 3 provides a function and asks you to write tests the cover all of its branches.

Most often, however, line coverage alone is not sufficient. In OCaml, anything from a “strange” value, such as the integer 0, to input size can affect the way in which our code executes.

Problem 4 requires you to exhaustively test a given function to determine its error.

But what do we do once we have discovered a failing test - a bug? Well, if your tests are specific enough and you have written them with a particular behavior of your code in mind, much of your work may already be done! Good testing should reveal to you the logical nature of an error. Provided your code is readable, making the step from logical to practical understanding should be a simple matter of looking up the lines that take care of the logic.

Problems 5 and 6 ask you to debug some functions by first determining the logical nature of their errors (through testing), and then discovering the erroneous code.

Using Print Statements

The third tool we use to debug OCaml functions is using print statements. Using the OCaml print commands,

For now, we will only concern ourselves with printing the output of a function's test. This allows us to view what a certain test actually resulted in. Otherwise, OCaml by default will tell you whether or not a test failed, but not what value it resulted in. Later in the course, when we've properly covered OCaml commands, you will be able to use print-statements from within the function body itself for more detailed debugging feedback.

Problems 7 and 8 each provide a function and some tests. Print the ouput of all tests to help you debug the functions.

So...

In conclusion, we have explored three tools at your disposal for debugging OCaml code:

We encourage you to avoid simply picking one of these at the cost of the others. Use your discretion and determine when it is most appropriate to use any one of them individually or multiple in combination. If you have a strong theory about the nature of a bug from the get-go, then by all means pursue it! Go straight to manual analysis around the problem area in the code. However, such a theory is not always apparent. In this case, perhaps you should ask yourself “do I fully understand my code to begin with?” and go back to basics with line-by-line analysis from the top of the function. Or else, maybe it's time to narrow down the problem(s) with some rigorous testing!