GUI Programming: Building A Paint Program from Scratch!

Getting Started

The supplied code for this project is an extremely primitive GUI library and paint program, comprised of a few files:

The three modules—Gctx, Widget, and Eventloop—are the three libraries that form our GUI toolkit. The Gdemo and Lightbulb programs test some of the functions you must add to the Gctx and Widget modules. Finally, Paint is where we implement our paint program.

Important Setup Instructions

When setting up your OCaml Managed Project, be sure to add the graphics.cma build flag. (See the graphics project configuration instructions if you don't remember how to add this flag.)

Be sure to import all of the given files, and set the project to build executables for widgetTest.ml, gdemo.ml, lightbulb.ml, and paint.ml.

Assignment Progression

We've covered the overall design of this GUI toolkit in class. (An overview of this design is also available in the lecture notes .) Your job in this homework assignment will be to extend both the GUI toolkit and the paint program itself. As a reminder of how the different parts fit together, here is a diagram of the software architecture:

gui architecture
The online documentation for the GUI toolkit gives a concise overview of its contents.

Feel free to use any function in the OCaml standard library in your solution. We've provided some useful guidelines in this description and in the comments in the source code. The documentation for the OCaml standard library is available online. In particular, you will need to read the documentation for the Graphics module.

Note that the early tasks of this homework have lots of explanation, but tasks 5 and 6 not so much. As a result, the earlier tasks are much easier than tasks 5 and 6. This is intentional; a rewarding component of this assignment is learning to program without a script. If you feel lost at first, don't panic. Think about what you've learned in terms of concepts and programming conventions from earlier tasks, and try some stuff out.

Grading

Although some components of the widget library are automatically graded, it is notoriously difficult to automatically test an entire user interface. Furthermore, we want to give you the freedom in Task 6 to implement whatever you like. We will thus also be testing most of your homework by hand. You can submit your homework as many times as you like, but we will only look at your last submission. (Note that the submission system will not run any tests; it will only verify that your code compiles. We will however run the test cases provided to you in testWidget.ml when we grade your program and use that as part of your score.)

The grade breakdown is as follows:

A quick note on style: while earlier assignments have not had too much emphasis on commenting (since we already gave you specifications for most of the code), you should keep in mind that you should be commenting your code appropriately for this assignment, especially for Task 6.

Submission Instructions

You will need to submit the following 5 files for this assignment:

Do not edit any other file! We won't see your changes, and the testers may not work. In particular, we will use the versions of Deque and Eventloop provided to you in the assignment download for testing, so there's no need to submit those.

Task 0: More Drawing

The first (tiny) part of this assignment is to read over the file Gctx.ml and become familiar with graphics drawing before you dive into the main GUI library and paint program. For this problem, fill in the main implementations for Gctx.draw_rect and Gctx.draw_ellipse.

The draw_rect function takes in a graphics context, a point, and a dimension. Given these, it will draw a rectangle with the dimensions's width and height in the given graphics context, with the lower-left corner situated at the given point. The type of draw_rect is therefore gctx -> position -> dimension -> unit.

You might be asking yourself: what color do we draw the rectangle? When you looked at gctx.ml, you may have noted that a graphics context stores a color value as one of its fields. When you pass the graphics context to a drawing function, it should call the helper function set_graphics_state. This function will set the color stored in the graphics context as the as the "active" color used by the OCaml Graphics library. Look at Gctx.draw_line for an example of how to do this.

Note that calling any native drawing method (such as Graphics.draw_rect) also requires conversion of the coordinates from our widget-local top-left-origin system to the bottom-left-origin system that the Graphics library assumes. The Gctx.ocaml_coords function will be helpful here. Again, look at Gctx.draw_line to understand how this should be used.

The Gctx.draw_ellipse function is similar. Looking at the documentation for Graphics.draw_ellipse should help you here.

You will know that this works correctly when running the gdemo.exe program. If it produces this layout below, you have done the task correctly.

gdemo
If you are on Windows, make sure that you always exit gdemo.exe by pressing a key. In the paint program you can use the "Quit" button. Some students have had trouble when closing the window using the X in the title bar.

Introducing: The Paint Application!

Here's what our finished Paint application looks like:

rawr dinosaur

Yours looks a little simpler at first. And even worse, you'll have to draw your own dinosaur. (No points for that.)

your paint app

At first, your GUI for this application consists of a number of controls and a drawing canvas, but it's pretty minimal.

You can draw lines with two clicks: the first click sets the start point of the line; the second click sets the end point. The color can be changed by clicking one of the color buttons; the other two buttons, Undo and Quit, do as their names suggest.

An Overview of paint.ml

Before explaining the main tasks in this homework, we briefly describe how paint.ml works. You should read through these instructions and the provided files before writing any code. (Remember Step 1 of program design: Understand the problem.)

As you read through paint.ml, browse the documentation for the primitive GUI toolkit.
The paint program uses a mutable record (called state) to store its state. This record stores the sequence of shapes that the user has drawn (state.shapes), the input mode of the paint program (state.mode), and the currently selected pen color (state.color). The interface of the paint program starts with three parts: the mode_toolbar, the color_toolbar, and the paint_canvas. These three widgets are laid out horizontally at the end of paint.ml (see below).

Shapes

At first, our paint program can only draw lines. The shape type has one constructor, Line, which takes a color, a start point, and an end point.

We record the sequence of shapes the user has drawn in state.shapes, which is of type shape Deque.deque. A deque is a good data structure for this because we want to record the order in which the user drew the shapes (so that we can display them properly layered), and so that we can support Undo by removing shapes from the tail of the deque.

Paint canvas repainting

We do the actual drawing in the repaint function, which is used to create the paint_canvas.

Every time the canvas is repainted, we go through the deque of shapes, drawing each line (and later other shapes) on the canvas in the order they were originally drawn by the user. (Deque.iter applies a given command to each element of a deque, in order from head to tail.)

Since later drawings are "on top of" older drawings, they are "higher in the z-order", where z refers to the z-axis of a three-dimensional coordinate system.

Paint canvas event handler

We handle clicks inside of the canvas using the paint_action function, which is added as an event_listener to the canvas via the its controller, paint_canvas_controller.

In the version of the code we provide you, users draw lines with two clicks, one for each endpoint.

To implement this, we define two drawing modes: LineStartMode and LineEndMode. The user of our paint program draws a line with two clicks: the first click sets the start point of the line; the second click sets the end point. We keep track of the first point by sticking it into the LineEndMode constructor; after we get the second click, we can go back to LineStartMode after adding the line to state.shapes.

We look at state.color to see which color is currently selected, and store it in the Line value we add to state.shapes.

Toolbars and layout

The rest of the file, marked TOOLBARS, sets up the various buttons and their event handlers.

To undo drawings, we simply remove entries from state.shapes; this is implemented in the Paint.undo function. To quit, we call the exit function. (The integer argument can be used to indicate that we're exiting with some kind of error—0 means there was no problem.)

The color buttons aren't like normal buttons: we don't want to draw a label, but instead we want to draw a colored box. To accomplish this, we define the widget-producing function Paint.color_button. Whenever a color button is clicked, it sets state.color appropriately. The current color is also displayed via the color_indicator. The repaint function of this colored square always displays itself with the currently selected color.

The last bit of this section just creates the toolbars and sets them up with Widget.hpair layouts. Once this is all done, we just need to run the eventloop on the top-level layout widget to run the program.

But don't just take our word for it: run it! (Try to draw a dinosaur, if you have the patience…)

Task 1: Better Layout

Suffice it to say, the paint program we've designed so far is pretty ugly. The first problem is with the layout: we only have Widget.hpair, so everything just goes off to the right. Half of the canvas is cut off!

We can solve this problem in a few steps: first, we'll add a new vertical layout, Widget.vpair, then we'll extend both of our pair layouts so they can be used with lists of more than two widgets.

Define Widget.vpair

For your first task, fill in the implementation of the function Widget.vpair, so we can stack our widgets vertically rather than horizontally. It has the same type as Widget.hpair: that is, widget -> widget -> widget. Obviously, however, the logic in the implementation of a vertical pair will differ slightly from the horizontal pair.

In particular, the translations in the repaint and handle functions will need to be swapped: rather than translating on the x-axis, we should translate on the y-axis. We'll also need to adjust the size function: instead of summing along the x-axis and taking the maximum y-axis value, we should take the maximum x-axis value and sum along the y-axis.

You may have noticed that our hpair widget uses the functions fst and snd. These useful functions take a tuple of two elements as input and return either the first or second element, respectively. For example, fst (3, 2) yields 3, and snd (3, 2) yields 2.

Once you have completed your implementation, you can verify its correctness by testing it! We have given you some test cases for a few functions in the Widget library; these can be run by executing widgetTest.exe. Note that we can't automatically test the repaint function of a widget, so you will have to verify that this part works correctly by running a GUI program. (But keep reading before you do that!)

Define Widget.list_layout

Before we reconfigure paint.ml to use our swanky new vertical pair widget, we should go one better: why not have horizontal and vertical list layouts, so we can make a row or column of a group of widgets? We'll define a function Widget.list_layout, which has type (widget -> widget -> widget) -> widget list -> widget. This higher-order function should take a pair layout function and a list of widgets, and use that function to arrange the widgets in order.

That is, list_layout hpair [w1;w2;w3;…;wn] should be same the as hpair w1 (hpair w2 (… (hpair wn-1 wn) …)). So we can implement list layouts as a fold over the list of widgets. (You may find the function List.fold_right useful here!) As our base case, we can use a trivial spacer, space (0,0), which takes up no space and won't change anything.

Create Widget.vlist and Widget.hlist

Using Widget.list_layout, we can now define two functions of type widget list -> widget, called Widget.vlist and Widget.hlist. Add the implementations of these functions to widget.ml.

Change the toolbars to use vlist and hlist

Now we can lay out our paint program much more cleanly. First, change the color toolbar and mode toolbar in paint.ml to use a Widget.hlist. Then lay out the toolbars and the canvas using a Widget.vlist, so that the canvas is above the undo and quit buttons, which are above the color buttons.

You can use the screenshot of our program above as a guide, though yours does not need to look exactly the same.

Task 2: Improving the Interface for Drawing Lines

The next step in making the paint program a little more usable is adding drag-and-drop drawing of lines. This is the way most drawing editors allow users to enter lines (rather than the click-two-endpoints method we currently have).

We'll make this change in two steps. First, we'll add previewing. Currently, when drawing a line, you don't see anything until both clicks have been made, which makes it difficult to draw a masterpiece like our dinosaur. By adding previewing, we will draw a "preview" line from the first click's location to the current mouse position.

After we've added previewing, we'll make it so that lines are drawn by dragging and releasing the mouse.

Add line previewing

Previewing for line drawing shouldn't be too difficult: if the first click in a line has already been made, then paint.mode = LineEndMode (x, y) for some point (x, y).

We just need to modify Paint.paint_action so that we know the mouse's position—the preview line should be drawn from (x, y) to the current position of the mouse over the canvas.

One thing to bear in mind as you complete this part is that many of the OCaml Graphics functions that take in a width and height parameter (such as Graphics.draw_rect) will react very badly when given a negative or zero value for either of these parameters. If you are having bizarre crashes on this part, make sure you are only passing positive values to these functions!

There are two things to consider when implementing preview functionality:

  1. How do we get the current position of the mouse over the paint canvas?
  2. How do we draw the preview line?

Notice that in the first line of the Paint.paint_action function, we define a variable p, which is the position in which the event occurs. For mouse events e.g. MouseDown , MouseMove, is the position of the mouse.

To draw the preview line, we simply need to modify the commands when we receive a Gctx.MouseMove event type. In this case, we want to draw a preview line when we're in LineEndMode.

To do this, we need to change the Paint.repaint function. We don't want to add the preview line to state.shapes, since the line shouldn't be permanent. Instead, we need to draw a different preview line every time the mouse moves. Our solution will be to add a new component to the state type that keeps track of the shape currently being previewed, if any. A shape option type is a good candidate here.

Extend the state type definition in paint.ml to include the following:

type state = {
  ...
  mutable preview : shape option;
}

Updating Event Handling

We want to create a Line (c, p1, p) and add it the "history" (state.shapes deque) each time we click while in LineEndMode, but not if we're in LineStartMode). We implement the necessary logic by pattern matching on event_types and paint modes within the Paint.paint_action function. Before you move on, read over this function and make sure you understand the logic in each case.

There are several changes to make to paint_action in order to implement shape previewing:

  1. Make sure that Paint.repaint draws whatever is being previewed. It should draw the previewed item last, since it's the most recent action and thus highest in the z-order. So whenever repaint is called, you'll want to draw all of the shapes already present in the shapes deque, and after that draw the shape stored in paint.preview, if it is not None.
  2. Change Paint.paint_action to manage paint.preview. When we receive the second mouse click (which is an event of type Gctx.MouseDown) and we are in LineEndMode, we need to remove the line preview:
    paint.preview <- None
  3. When the mouse moves without clicking (Gctx.MouseMove) in LineEndMode, we need to set paint.preview to some appropriate value. In this case, the preview should be a line from the start position (associated with the LineEndMode (x, y) data constructor) to whatever the current mouse position is (the position associated with the MouseMove event).
  4. We also need to modify the function associated with the Undo button to reset the paint.preview shape to None. In addition, if the paint.mode is LineEndMode, then reset it to LineStartMode—this prevents strange behavior when the user starts drawing a line, drags the mouse off the canvas, and then clicks the Undo button.

Drag-and-Drop Line Drawing

To add drag-and-drop line drawing, we need to change the way we handle previewing and drawing. A "drag" is when the user clicks and holds the mouse button; a "drop" is when the user releases the mouse button after dragging. We want to draw a line by clicking at the start point, dragging the line, and dropping at the end point.

The necessary logic changes are straightforward:

Task 3: Drawing Points

Our primitive program only allows us to draw lines, so we will now extend it to draw points as well. To make things simple, points will always be one pixel in size. This task is a little more complex than the one above. Here are the steps to implement drawing points.

Update the interface files

We will be adding two functions, draw_point and draw_points. Let us update the gctx.mli file and add the functions' signatures to this file.

An mli file acts as an interface constraint for an ml file, so other files can only access values and functions which are explicitly added to a module's interface. (Think back to homework 3 which used a set interface to constrain which values defined in listset.ml and treeset.ml could be seen from outside the module.)

In this case, you'll want to add the following lines to the mli for Gctx:

val draw_point : gctx -> position -> unit
val draw_points : gctx -> position list -> unit
For most of the later functions and values we ask you to implement (as well as any you write yourself), you will need to add the function or value's type signature to the appropriate mli file.

Implement Gctx.draw_point

First define a draw_point function in gctx.ml. This is a function that takes in a graphics context and a point, and draws a point in that location in the context. The type is therefore gctx -> position -> unit.

Your draw_point function should be similar to the functions in Task 0. In this case, to do the actual point drawing use Graphics.plot, a function in the native OCaml Graphics library.

Implement Gctx.draw_points

We also want to support drawing multiple points concurrently, so our next step will be using our new draw_point function to write another function which iterates over a list of points and draws each of them inside the given graphics context. Define the function Gctx.draw_points, of type gctx -> position list -> unit. (You may find the function List.iter useful here!)

Add a point-drawing mode

We now have a function that draws a point or list of points in some graphics context. Our next job is to hook this into the paint program by adding a mode to distinguish when we want to draw lines versus points. Add a PointMode constructor to the mode type.

The PointMode constructor doesn't need to carry any data: unlike line drawing (which needs to remember the first point of the line), we can draw a point in a single click, so we don't have to track anything in our mode.

Extend the shape type to include points

Recall that we are storing the "history" of the canvas in the shapes deque, where each item in the list represents some kind of basic drawing command. We need to make it so that "drawing points" is one of these possible commands.

Open paint.ml and modify the shape type to include a constructor Points of Gctx.color * point list.

We want to allow a user to draw a series of points at once by dragging their mouse with the button down. After the user releases the mouse button, we need to add all of the points that the user drew to our deque of shapes. (A command to draw a set of points just requires knowledge of the current color, and the points' locations.)

Update Paint.repaint

Now update repaint so that when it sees Points (c, ps) in shapes, it will appropriately call the Gctx.draw_points function we defined earlier and actually draw each point in ps on the canvas. Don't forget to set the color in the graphics context!

Update Paint.paint_action

We want to create a Points (c, ps) and add it the "history" (shapes) each time we click while in PointMode, but not if we're in some other mode, like LineStartMode).

Because we want to the user of our application to be able to drag the mouse to draw several points at once, we can't just add every new point we encounter to our shapes deque. Instead, while the user has not yet let go of the mouse button, we can use our paint.preview feature to store all the points they've drawn on the canvas so far.

Update paint_action so the following event handling takes place:

Add buttons to the toolbar

The final step is to add buttons to the toolbar that allow us to toggle between Point and Line modes.

You will need to create two buttons: one for Lines and another for Points (see the reference image above). When you click on the Line button, it will set the current mode to LineStartMode; when you click on the Point button, it will set the current mode to PointMode. You can use mouseclick_listeners to set up these actions, following the examples in undo and quit.

Once the buttons have been added and you can switch modes, you should be able to draw points when running your application! (Make sure to give it a try!)

To recap the steps you took to implement drawing points:

  1. Add some functions in Gctx that allow us to draw points on a canvas.
  2. Add a new mode, which allows us to figure out if we should be drawing points or lines when the user clicks.
  3. Update the type shape, so we can store the color and location of drawn points.
  4. Update Paint.repaint to link the points to be drawn (step 3) with the drawing operation (step 1).
  5. Update Paint.paint_action so when we click in the new mode (step 2), it stores the desired point by creating a new shape item (step 3) and adding it to the deque of shapes.
  6. Add a button to the toolbar so we can switch to the new mode (from step 2).

Task 4: Drawing Ellipses

For your next task, you must add the ability to draw ellipses via dragging and dropping to your paint program. As with lines, you will need to display a preview of the shape to be drawn as the user drags on the canvas; the shape should only be saved to the canvas when the user releases the mouse. You'll need to make a number of changes to get this working, following a similar pattern to the one we used for point drawing:

There are different ways to draw ellipses. We'd like you to implement ellipse drawing using a "bounding box" method. The starting and ending points of the mouse drag are treated as two corners of a rectangle, with the ellipse drawn so that it fits exactly inside of this rectangle. Reference the GIF below for an illustration of how this works.

your paint app

Task 5: Checkboxes and Line Thickness

Task 5 is significantly more difficult than tasks 0-4. The first tasks were meant to familiarize you with the project structure. This one will require you to synthesize what you have learned into entirely new functionality.

To do this task, you must extend the widget library with a checkbox widget. This widget can be checked and unchecked on click, and should update the state of the application based on the checkbox's state. In the case of the paint application, you will be using it to toggle line thickness! After you complete implementing this feature, both lines and ellipses should be able to be drawn using thick lines.

While you are going to be using the checkbox for thickness, we want the checkbox widget to be a general purpose component that could be used in any GUI application. Therefore, it will have the following signature:

val checkbox : bool -> string -> widget * bool value_controller

The first argument indicates the initial state of the the checkbox (i.e. should it start clicked or not?), while the second is the initial string for the label of the checkbox. Along with the checkbox widget itself, this function returns a 'a value controller of the type below...

type 'a value_controller =
  { add_change_listener : ('a -> unit) -> unit;
    get_value : unit -> 'a;
    set_value : 'a -> unit }

What is a 'a value_controller? The type definition for these can be found in widget.ml, but the main idea is that a 'a value_controller is a generic OCaml object that is responsible for storing and monitoring some mutable value of type 'a, in our case it will store the state of our widget. Each controller provides a set_value and get_value function, which can be used to update and query the value of the 'a stored by the controller. The most interesting function of the 'a value_controller, however, is the add_change_listener function. Every 'a value_controller stores a list of change_listeners, which are ('a -> unit) functions, that are associated with that controller. Whenever the controller value is updated via the set_value function, it should call all of the associated change_listeners with the newly-updated value as the argument to the listener.

Using a 'a value_controller, implement a generic checkbox widget. The checkbox should have associated with it a bool value_controller which keeps track of whether or not the checkbox is checked, and stores a list of change_listeners that should be executed whenever the checkbox is toggled. In this case, the checkbox's state is stored by the bool value_controller, and will be updated when appropriate by the checkbox via the controller's set_value and get_value functions.

To help you in testing your checkbox code, we have given you an adaptation of the lightbulb code presented in lecture (lightbulb.ml) that uses the checkbox widget you must define for this task. There are two versions of the lightbulb within the given code. The first one, labeled "STATE LIGHT", just directly uses the get_value function of the bool value_controller to determine whether it should be turned on or not; this will test whether or not you are toggling state correctly within your checkbox. The second, labeled "LISTENER LIGHT", registers the change of lightbulb color as a change_listener within the program; this one will test that you got the change_listener functionality working properly within your program. We advise you first make sure that toggling the state works (i.e. get the first example working), and then get the item listener support working (i.e. get the second example working)

The aesthetics of your checkbox are a matter of preference; simply make sure that your checkbox has a label, and that it clearly displays whether or not it is toggled. For example, our own implementation draws an X within itself when toggled.

Checkboxes function similar to buttons, but with the addition of a piece of local state that must be maintained and updated between clicks. While understanding how our given button widget works will give some insight into how to complete this task, we strongly advise you not to simply copy over the code and tweak random things until it works. Take the time to understand the how the differences in how these widgets are specified will translate to implementation differences between them!

Once you have successfully created the checkbox type, you will need to put one into your paint program and configure it so that can be used to toggle line thickness. This task will require that you modify the paint program such that it knows whether it should be drawing thin or thick lines at any point in time. You'll want to consider how the paint program we provide handles information about colors, and implement line thickness in a similar fashion. You can control the line thickness of figures drawn by the OCaml Graphics library through the Graphics.set_line_width function. You can use the function get_value defined in the 'a value_controller type to actually get the status of the checkbox on which the thickness toggle will be based.

Task 6: A Cool New Widget!

Your final required task is to add one more cool widget of your own choosing and use it in your paint program! Below are a few things of appropriate coolness and difficulty...

These are not the only possible options for your cool new widget! If you have an alternate idea, feel free to run it by us using a private Piazza post. Remember that any suggestions should involve a creating a new type of widget, and should be of reasonable difficulty; if you think it would be substantially easier than anything we suggested above, it is probably too easy.