Testing in Java

Why do we care about testing?

Welcome to Java! Even though we’re switching languages, testing remains important. If you’re new to our testing guides, we covered a lot of testing fundamentals in our OCaml Testing Guide.

You may be thinking that reading this guide (and testing in general) is a lot to do, but it’s well worth the time to learn how to write thorough, effective tests. Testing helps you write code more quickly and more easily, and reading this guide and writing tests for your code will save you time and many headaches in the long run.

• Testing in Java
  • Why do we care about testing?
• How to write and run tests in Java <!-- raw HTML omitted -->
  • Setting up your test files <!-- raw HTML omitted -->
  • Running your tests <!-- raw HTML omitted -->
  • Writing test syntax <!-- raw HTML omitted -->
    • How to actually write a test <!-- raw HTML omitted -->
    • How to write failing tests <!-- raw HTML omitted -->
    • How to ensure your tests have good style <!-- raw HTML omitted -->
• How to approach testing in Java <!-- raw HTML omitted -->
  • Testing from simple to complex methods <!-- raw HTML omitted -->
    • Figuring out what to test for and how to test it <!-- raw HTML omitted -->
    • Testing simple methods <!-- raw HTML omitted -->
    • Testing complex methods <!-- raw HTML omitted -->
  • Testing Objects <!-- raw HTML omitted -->
    • Encapsulation <!-- raw HTML omitted -->
    • Structural equality <!-- raw HTML omitted -->
    • compareTo <!-- raw HTML omitted -->
  • Model-View-Controller Testing (for user interfaces) <!-- raw HTML omitted -->
    • What is Model-View-Controller <!-- raw HTML omitted -->
    • What does Model-View-Controller mean for user interface testing <!-- raw HTML omitted -->
    • Testing with tests vs the GUI <!-- raw HTML omitted -->
• Appendix <!-- raw HTML omitted -->

How to write and run tests in Java

Setting up your test files

Follow the instructions in our IntelliJ Setup guide. Make sure you follow step six of “Step 4: Creating a Java Project.”

We will give you files in which you’ll write your tests for every Java assignment except HW09. To write tests in HW09, create a new class, give it a descriptive name (like MyTests.java), and write the following lines at the top of the file:

import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

This will import all of the tools you need to write JUnit tests into the file. Read below for the syntax of individual test cases.

Running your tests

To run your tests, open the testing file you want to run and press the green play button in the top left of your screen.

Sometimes this may run your entire program instead of the test file. To run only the test file, right click on the file in the Package Explorer (left of your screen, by default), click Run As, and click JUnit Test.

You will be able to see which tests passed, failed, or resulted in an error.

Double-click on a failing test case in the above window to display its failure trace, which will help you debug!

Writing test syntax

How to actually write a test

As much as you might want to forget about OCaml, we don’t want all of your good test-writing skills to go to waste! Recall the syntax we used to write this test for rainfall in HW01:

let test () : bool =
    rainfall [1; 2; 3; -999] = 2
;; run_test "rainfall: -999 ends the list" test 

We write run_test to indicate that we’re running a test. We provide run_test with two arguments: a string that describes the test and the test that should be run. In the test, we compare the actual result, rainfall [1; 2; 3; -999], to the expected result, 2. We’ll find the same structure in tests in Java.

Let’s look at the syntax of a test we write for you in HW06:

@Test
public void testConstructInBounds() {
    Pixel p = new Pixel(40, 50, 60);
    assertEquals(40, p.getRed());
    assertEquals(50, p.getGreen());
    assertEquals(60, p.getBlue());
}

First, annotations before every method in the class are used to indicate the purpose of the method and are designated with the @ symbol. Here, we want to use the above method to test one of the Pixel constructors, so we use @Test to indicate we want JUnit to run it as a test.

We’re also not just limited to writing testing methods! There are other behaviors the method could have. For example, @BeforeEach indicates the method should be performed before running each test in the class. Other annotations include @BeforeAll, @AfterEach, @AfterAll, @RepeatedTest, and more. To see a complete list of annotations, check out the JUnit 5 Javadocs.

Let’s look at an example of when we’d want to use these annotations. It’s ~spooky~ season at the time of writing, so let’s consider a class called CandyBowl. The full code with comments is available in the appendix.

CandyBowl has two private instance variables: the type of candy in the bowl and the number of pieces left. The type of candy is taken in as an argument when initializing a new bowl, but the number of pieces is always initialized as 10. CandyBowl also has four methods to take candy out, refill with more candy, get the name of the candy, or get the amount of candy left.

public class CandyBowl {
    private String candy;
    private int count;
            
    public CandyBowl(String candyName) {
        candy = candyName;
        count = 10;
    }

    public void takeCandy(int taken) {
        count = Math.max(count - taken, 0);
    }

    public void refillCandy(int refill) {
        count = count + refill;
    }

    public int checkCandyCount() {
        return count;
    }

    public String getCandyName() {
        return candy; 
    }
}

Most of your tests for this class will probably deal with taking and refilling the candy and checking the final value with checkCandyCount(). Let’s say we have a test like this:

@Test
public void testTake5Refill7() {
    CandyBowl b = new CandyBowl("Milky Way");
    b.takeCandy(5);
    b.refillCandy(7);
    assertEquals(b.checkCandyCount(), 12);
} 

Imagine we wanted to run another test after this. It was already a lot of math to get the final count to be (10 - 5 + 7) = 12. Now imagine having to add and subtract all the tests before to get the starting count for a new test. Not fun right?

Now imagine we write a method, createNew() that makes a brand new candy bowl where we know the count will be 10. If we run this before each test, we can use a new candy bowl as a starting point.

In our testing class, we’ll need to make a global CandyBowl first. The testing class will therefore look like this.

public class MyCandyTests {
    private CandyBowl b;
    ...
}

We can then write a method to make a new one and tag it with @BeforeEach.

@BeforeEach
public void createNewCandyBowl() {
    b = new CandyBowl("Milky Way");
}

Our test can then look like this:

@Test
public void testTake5Refill7() {
    b.takeCandy(5);
    b.refillCandy(7);
    assertEquals(b.checkCandyCount(), 12);
}

When JUnit reports the outcome of all the tests you write, it reports the name of the method, so you should write descriptive and clear method names just like you did in OCaml. In the above example, it would tell you whether testTake5Refill7() passes, failed, or resulted in an error.

In OCaml, when we tested the behavior of rainfall, we used = to compare the output to the expected value. In Java, we use the Assertions class, which provides a variety of helpful methods for testing. The most relevant method in Assertions is assertEquals, which we used in this test. assertEquals checks whether its two arguments are structurally equal.

There are many different types of assert methods, including assertTrue, assertFalse, and assertNull. Hopefully you can guess what they do from their names :).

To see a complete list of assert methods you can use in testing, check out the Assertions Javadocs.

How to write failing tests

In OCaml, we run run_failing_test instead of run_test when we want to test code that should fail. In Java, you can do something very similar.

Imagine a function squareRoot defined like so:

public static double squareRoot(int x) {
    ...
}

Suppose we want squareRoot to throw an IllegalArgumentException when x is less than zero. We’ll use the assertThrows method to test that, using the following syntax:

@Test
public void testSquareRootThrowsIllArgExcNegativeInput() {
  assertThrows(IllegalArgumentException.class, () -> {
     squareRoot(-1);
  });
}

We specify the exception that we expect to be thrown, and then we write the code that should throw that exception between the brackets.

How to ensure your tests have good style

Use appropriate assert statements. In OCaml, you avoided statements like if x = true when x was a boolean value. Similarly, we want to avoid writing assertEquals when one of the two arguments is true, false, or null. Instead of writing assertEquals(x, true), write assertTrue(x).

How to approach testing in Java

Testing from simple to complex methods

Figuring out what to test for and how to test it

Just like in OCaml, we highly recommend writing tests as you go before implementing each method. Writing tests before you write the code makes sure you completely understand the method: the parameters, the return value, the behavior for edge cases or invalid inputs, etc.

The first thing you should do before writing a test is read the method description. Before each method we ask you to implement, we have a description of the method detailing the parameters, what it’s supposed to return, what it’s supposed to do, and information about exceptions or invalid inputs. All of this information can be directly translated into tests.

Let’s look at the method description for the first Pixel constructor we ask you to implement:

/**
 * Create a new pixel with the provided color intensities.
 * 
 * If the supplied arguments are not between 0 and 255, they are clipped:
 * 
 * Negative components are set to 0.
 * Components greater than 255 are set to 255.
 * 
 *
 * @param r the red component of the pixel
 * @param g the green component of the pixel
 * @param b the blue component of the pixel
 */

Just from this description, you can see you’ll want to test creating a Pixel with all valid inputs, with at least one input less than 0, and at least one input greater than 255. This can be generalized to be testing:

• Valid, non-special case inputs
• Valid special case inputs
• Invalid inputs

If your method has multiple different behaviors it could exhibit, make sure you test each of them.

Now that you know what to test, you need to learn how to test it. Our provided test, testConstructInBounds, uses getRed(), getBlue(), and getGreen() to check that the components of the Pixel were initialized correctly.

getRed(), getBlue(), and getGreen() are examples of getters. Most classes you work with will have methods referred to as getters and setters. A getter allows you to get (a copy of) some information (usually stored in a field) from an object. getRed() is a getter for the red color component of Pixel. A setter allows you to set the value of some field in an object. Setters may not always be available - some fields are meant to be private, or encapsulated, within the class. Setters might also place limits on what you can change in order to maintain internal invariants. Pixel.java does not provide setters, because a Pixel should be immutable once it is instantiated.

Getters will be one of the most commonly used tools in your Java testing toolbox.

Testing simple methods

In OCaml, we recommended “bottom up” testing, where you write and debug simpler tests before more complex tests. In OCaml, this looked like testing base and edge cases first or testing a function earlier in the homework that we’d ask you to utilize as a helper function later in the homework.

In Java, you will implement many large classes with complex behaviors, and bottom up testing will become only more important. Let’s go back to CandyBowl. It has a getter, checkCandyCount() that “gets” the count of candy left. We used checkCandyCount() in our testTake5Refill7() test, but imagine if there was an error in checkCandyCount() that caused the test to fail. We might start debugging by examining takeCandy and refillCandy since it might not occur to us to check checkCandyCount at first. If we had tested our getter first, we would be able to save time searching through multiple functions for the error. Let’s look at how we could test checkCandyCount() using no other methods and knowing the initial count is 10:

@Test 
public void testCheckCandyCount() {
    assertEquals(b.checkCandyCount(),10);
}

Check out the appendix for the full code and another example getter test! You might think these tests will be short and simple, but some might require some thinking. Either way, don’t skip them! You’ll save yourself so much time and energy if you don’t have to trace through all the methods you use in a test to find why it’s failing.

Testing complex methods

Now, assuming you’ve tested the simple stuff first, how do you test the complex stuff?

Complex methods often call simpler methods as helpers. Thankfully, you’ve tested your simpler methods already, so you’ll be able to fix those smaller bugs early on before they propagate to complex methods. That lets you focus on what each complex method does overall, instead of getting lost in the weeds of its simpler method calls.

To test complex behavior, run the complex method(s). The complex methods can produce an output or change the state of your program, so you’ll now want to check that the method’s output and the program’s state matches with your expected values. The best way to test the program’s state is by using assertEquals and getters. You may want to confirm that parts of the program that should not have changed did not change.

Testing Objects

Encapsulation

Encapsulation is a concept that comes up multiple times in CIS 120. For the purposes of testing, to test encapsulation is to test that you can’t change an internal value willy-nilly. Test that you can only change a value through the appropriate methods.

Consider a simple class called MyNumbers:

public class MyNumbers {
    private int[] myNumbers;

    public myNumbers(int[] x) {
        myNumbers = x;
    }

    public addNumber(int x) {
        myNumbers[0] = x;
    }

    public removeNumber() {
        myNumbers[0] = 0;
    }

    public getNumbers() {
        return myNumbers;
    }
}

In this class, we should only be able to change myNumbers by calling addNumber or removeNumber. In other words, we are encapsulating the information in myNumbers - it should only be modifiable via addNumber or removeNumber. We also have a getter, getNumbers.

If I am able to modify myNumbers without using addNumber or removeNumber, that would violate encapsulation. Clearly I can’t access myNumbers directly, because it’s a private field. Is there any other way I can modify it without using addNumber or removeNumber?

Consider the current implementation of getNumbers. The current implementation will return a pointer to the myNumbers object on the heap. If I wrote:

int[] arr = getNumbers();
arr[0] = arr[0] + 1;

This would also modify myNumbers, since arr and myNumbers are aliased and referentially equal. A better implementation would be:

public getNumbers() {
    return Arrays.copyOf(myNumbers);
}

Now, understanding that example, to test encapsulation we could write the following:

@Test
public void testGetNumbersEncapsulation() {
    int[] x = new int[2];
    x[0] = 0;
    x[1] = 1;
    MyNumbers n = new MyNumbers(x);
    int[] y = n.getNumbers();
    y[0] = 5000;
    assertEquals(n.getNumbers()[0], 0);
}

When this test passes, we can be certain that the only way to modify myNumbers is through the appropriate methods. We have preserved encapsulation.

Structural equality

When you create a class in Java, you should always override the equals method. To evaluate whether two objects are structurally equal, you must call equals. For example, object1.equals(object2).

By default (if you do not override equals), your classes will use reference equality for comparison when you call object1.equals(object2).

To ensure you have overridden equals and to ensure you have done so correctly, test that comparing two referentially unequal but structurally equal instances of a class returns true.

compareTo

When you create a class in Java and you want to store instances of that class in a Set, you need to implement the Comparable interface. (If this doesn’t make sense right now, you can skip this until it comes up in lecture.)

The Comparable interface requires that you implement one method, compareTo(T o), where T is the type of the class implementing the Comparable interface.

Make sure to test the compareTo(T o) function thoroughly, as it is a common source of bugs when working with TreeSets or TreeMaps.

Model-View-Controller Testing (for user interfaces)

What is Model-View-Controller

Model-View-Controller (MVC) is a way of structuring the framework of a program typically used for user interfaces. As the name suggests, it’s broken down into three parts: the model, the view, and the controller. The model contains all of the data or knowledge of the program, the view displays the model and is actually seen by the user, and the controller links the two by accepting user input, translating it to the model, then updating the view to reflect the change in the model. As a result of this structure, the model and controller can exist without the view – everything a person could do to interact with the user interface could be written in code.

Our Paint application is an example of this framework. The model can be seen most easily in paint.ml. paint.ml contains all of the data for the internal state: definitions of the types point, shape, state, etc, the current state, the deque of shapes, all of the widgets, and more.

The view is most easily seen in eventloop.ml. We didn’t ask you to write any of this code, and you might not have looked at the file, so let’s look at some of the comments for the functions in eventloop.ml:

(** 
 * This function takes a `widget`, which is the "root" of the `GUI` interface. It creates the "top-level" `Gctx`, 
 * and then it goes into an infinite loop.
 * 
 * The loop simply repeats these steps forever:
 * - wait for a user input event
 * - clear the graphics window
 * - forward the event to the widget's event handler 
 * - ask the widget to repaint itself
 *)

This file is what creates the initial window that the user sees and can interact with. If it receives user input, it passes it off to the controller and repaints itself with the updated model.

The controller is most easily seen in widget.ml. Here, we define event_listeners, value_controllers, etc that take in user input and translate it to updating the internal state of paint.ml. To put it all together, if a user clicks the thick lines checkbox in the Paint window, eventloop.ml passes this to the checkbox’s event handler which we designed in widget.ml. The checkbox widget specifies generally what happens if a MouseDown event happens inside the widget. In paint.ml, we’ve defined specifically what happens when the thick lines checkbox is clicked, so the internal state of paint.ml is updated to have thick lines when the handle of the widget occurs. Since the value of the checkbox also gets updated inside of paint.ml, when the window repaints, the checkbox widget draws the box differently in it’s repaint field (checked vs unchecked). This is then translated to the way the checkbox looks to the user.

In this example, user input to the view was translated by the controller to update the model. The updates were then translated by the controller to reflect in the view.

What does Model-View-Controller mean for user interface testing

All of the Java homeworks have some kind of user interface and internal state, so understanding how they are designed as MVC is very important for testing. One feature of MVC is that everything designed to happen in the model and controller can happen without the view. For example, clicking on a checkbox and passing a MouseDown event to a checkbox through code are handled the same way by the controller and the model.

Let’s look at a test from widgetTest.ml for an example of this:

let gc = Gctx.top_level
let click55 = Gctx.make_test_event Gctx.MouseDown (5,5)

;; run_test "checkbox click" (fun () ->
    let w, cc = checkbox false "checkbox" in
    w.handle gc click55;
    cc.get_value())

click55 is a MouseDown event and w, cc are the widget and value controller for a checkbox. Sending click55 to the handle of w should act the same as clicking in w in the Paint window. Since this checkbox was initialized to false, and the handle of w should update the value_listener of the checkbox to be true, so cc.get_value() should now be true, causing the test to pass.

When you write tests in Java, you should be testing how the controller updates the model. Instead of testing how the user interacts with the interface, you should test that the internal state updates correctly by stimulating user events through code. For example, in ManipulateTest in HW06, the tests make sure the methods you write in SimpleManipulations and AdvancedManipulations work by comparing the result of calling each method to a hard-coded expected result.

public static PixelPicture smallSquare() {
    return new PixelPicture(new Pixel[][] {
        {Pixel.BLACK, Pixel.BLUE},
        {Pixel.RED,   Pixel.GREEN}
    });
}

@Test 
public void rotateCwSmall() {
    assertEquals(
        0,
        PixelPicture.diff(
            new PixelPicture(new Pixel[][] {
                {Pixel.RED, Pixel.BLACK},
                {Pixel.GREEN,Pixel.BLUE}
            }),
            SimpleManipulations.rotateCW(smallSquare())
        ),
        "rotateCW 2x2 image"
    );
}

This test compares the result of calling SimpleManipulations.rotateCW on smallSquare() to what we expect smallSquare() to look like. In Pennstagram, this action would be performed by clicking the RotateCW button, but we can just call the method without needing to interact with the GUI.

The JUnit tests do not need the user interface at all, so the MVC framework allows you to thoroughly test the model and controller without using the view. The next section will explain when testing through the GUI is helpful, but when you write JUnit tests, you should not need to involve the GUI at all.

Testing with tests vs the GUI

Think back one more time to Paint and how you tested it. We gave you some tests in widgetTest.ml, but a large part of your testing was running paint, gdemo, or lightbulb and playing around with them to make sure your code worked. Your checkbox was tested in widgetTest.ml to make sure clicking it would change the value, but your specific checkbox for thick lines was only tested by checking it and drawing some shapes.

Testing with JUnit tests will typically be more exhaustive than testing with the GUI and will make it easier to pinpoint which functions are causing an error. Playing around with the GUI is still important to ensure everything is displayed properly since some visual changes are difficult or impossible to test through JUnit tests. For example, if we wrote a program to draw out the pieces of candy left in our CandyBowl, we can write JUnit tests to ensure the internal count is updated properly, but the only way to check that we draw the right number of pieces of candy is by running the program and seeing what it looks like.

It will be important for you to figure out what parts of your code are best tested through tests and which are best tested by playing around with the GUI. Thinking of the Model-View-Controller framework might help with this! This will become more important in later Java homeworks, so keep this in mind for the future.

Happy testing :)

Appendix

You can also download the below code files here.

CandyBowl.java:

public class CandyBowl {
    private String candy; // initializes values
    private int count;

    public CandyBowl(String candyName) {
        candy = candyName; // sets candy as string input
        count = 10; // always sets count to 10
    }

    public void takeCandy(int taken) {
        count = Math.max(count - taken, 0); // subtracts from count
        // maintains invariant that count must be non-zero
    }
    
    public void refillCandy(int refill) {
        count = count + refill; // adds to bowl
    }

    /**
     * Getter for count
     */
    public int checkCandyCount() {
        return count; 
    }
    
    /**
     * Getter for candy
     */
    public String getCandyName() {
        return candy;
    }
}

MyCandyTests.java:

// import tools to write JUnit tests
import org.junit.jupiter.api.*;
import static org.junit.jupiter.api.Assertions.*;

public class MyCandyTests {
    private CandyBowl b; // global CandyBowl variable to be used by all tests
    
    @BeforeEach // will run before running every test
    public void createNew() {
        b = new CandyBowl("Milky Way"); // creates a new candy bowl every time
    }

    @Test // will run the following method as a test
    public void testCheckCandyCount() {
        // checks the two values are equal
        assertEquals(b.checkCandyCount(),10); 
    }
    
    @Test
    public void testGetCandyName() {
        assertEquals(b.getCandyName(), "Milky Way"); // checks name of b
        // creates a second candy bowl to test
        CandyBowl b2 = new CandyBowl("Butterfinger"); 
        // you can have multiple assert statements in one test
        assertEquals(b2.getCandyName(), "Butterfinger"); 
    }
    
    @Test
    public void testTake5Refill7() {
        b.takeCandy(5);
        b.refillCandy(7);
        assertEquals(b.checkCandyCount(), 12);
    } 

}

MyNumbers.java:

import java.util.Arrays;

public class MyNumbers {
    private int[] myNumbers;

    public MyNumbers(int[] x) {
        myNumbers = x;
    }

    public void addNumber(int x) {
        myNumbers[0] = x;
    }

    public void removeNumber() {
        myNumbers[0] = 0;
    }

    public int[] getNumbers() {
        // Bad implementation
        // return myNumbers;
    
        // Good implementation
        return Arrays.copyOf(myNumbers, myNumbers.length);
    }
}