Immediate Mode User Interface

Tom - Tuesday, August 18th, 2020 Comments Off on Immediate Mode User Interface !

Hello!  Tom here for another tech post.  In this entry I will be talking about an Immediate Mode GUI library we’re using for parts of Blightmare’s editor, and what makes it special.

We’re all familiar with Graphical User Interfaces, or GUIs.  It’s the thing you interact with on a modern computing device where you can see and click on or touch the things you’re interacting with.  This includes things like buttons, text boxes, images, stuff like that.

The classic way of creating a GUI is using some kind of GUI library.  There are many of these around, including Windows’ built in library, Cocoa on macOS and iOS, Qt, Gtk, etc.  Most of these are what’s called a retained mode user interface.  In the retained model, you create widgets, and then pass them copies of your data.  You then set up callbacks that will be called if the data is ever changed in the UI, or if the user presses a button, or any other interaction happens.  This works, and has been used successfully for decades.  It does, however, have some disadvantages.  The main disadvantage is that now there’s a separation between your actual representation of your data and the user interface version of the data.  The UI has its own copy, and tells you when it changes, and it’s up to you to then update the real copy.  Likewise, if your copy changes, its up to you to update the UI copy.  While not a huge deal, it does introduce additional work which can be prone to errors.

Another relatively recent way of implementing user interfaces is called immediate mode.  In this model, rather than creating your widgets up front, and then updating them as data changes, the widgets only exist ephemerally, created/drawn once per frame, and only if explicitly told to.  The data they contain is the same data that you’re already storing, and the UI updates it directly, right then and there where the widget is invoked.  This may sound a little strange, but hopefully an example will help.

For Bligthmare’s editor, I’m currently working on the camera system which lets the game designers select how the camera will track the player as they move through the level.  To do this, we needed a way of adding, removing and modifying various camera zones in the level.  A camera zone is just a rectangle that defines the bounds within which the camera affects the player.

I’m personally not a huge fan of Unity’s built in GUI system, so I thought I’d try an experiment and see how hard it would be to use an immediate mode GUI library to implement this feature.

I chose a library called Dear ImGui since it has pretty good support, a fair number of widgets, and can work with Unity.  To actually create a GUI using Dear ImGui is very simple.  Just define a function that gets called on each frame, and within that function create whatever widgets you happen to need during that frame.  A simple example:

// Start a new window
ImGui.Begin("Camera Editor");

// Add a button.  If the button is clicked, the function returns
// true and we can immediately perform the corresponding action
if (ImGui.Button("Add Zone"))
{
    // This button creates a new zone, so let's create one and
    // add it to our list of zones.
    CameraZone newZone = new CameraZone();
    // ... initialize newZone ...
    cameraZones.Add(newZone);
}

// Add another button on the same line.
ImGui.SameLine();
if (ImGui.Button("Remove Zone"))
{
    // This button removes a zone, so let's just remove the currently
    // selected zone.
    cameraZones.RemoveAt(currentCameraIndex);
    if (currentCameraIndex == cameraZones.Count && currentCameraIndex != 0)
    {
        currentCameraIndex--;
    }
}

// We need to make a list of the names of each zone so we can display
// them in a list
string[] cameraNames = new string[cameraZones.Count];
for (int i = 0; i < cameraZones.Count; ++i)
{
    cameraNames[i] = cameraZones[i].name;
}

// Now show the list of zones.  We pass it currentCameraIndex which
// is the currently selected zone.  If the user changes the selection,
// this variable will automatically reflect that change.
ImGui.ListBox("", ref currentCameraIndex, cameraNames, cameraNames.Length);

// And we're done with this window
ImGui.End();

The above function is our entire user interface, including handling all the interactions.  It draws something like this:

In our GUI function, just calling ImGui.Button is enough to have a button appear.  If that function returns true, we know that the user has, during that exact frame, pressed the button, so we can update our internal data accordingly.  In this case, in the “Add Zone” button handler, we create a new zone and add it to our cameraZones list.  In the “Remove Zone” handler, we remove the current zone from the list.

The currently selected zone is stored in currentCameraIndex.  ImGui.ListBox shows us a list of the entries we’ve given it, in this case it’s a list of the camera names.  We also give it the currentCameraIndex which it will use to draw the highlight over the currently selected camera name.  If the user happens to select a different entry in the list, the ImGui.ListBox call will automatically and immediately update our currentCamreaIndex variable to hold the newly selected index.

This is the beauty of an immediate mode user interface.  Interactions happen exactly in the same place we define the UI itself, and the data the UI is operating on is the same data that we’re storing anyway.

Here’s another example.  Now that the user has selected a camera zone, we can show them some more details about it:

if (selected)
{
    // Begin a new window
    ImGui.Begin("Camera Detail");

    // Show the name of the camera
    ImGui.Text(cameraZones[currentCameraIndex].name);

    // Set up two columns, we'll show x and y on one row, and
    // width and height on the next row.  Each of these InputInt
    // functions displays the current value, but also updates
    // the actual value if the user happens to change it.
    ImGui.Columns(2);
    ImGui.InputInt("x", ref cameraZones[currentCameraIndex].x);
    ImGui.NextColumn();
    ImGui.InputInt("y", ref cameraZones[currentCameraIndex].y);
    ImGui.NextColumn();
    ImGui.InputInt("width", ref cameraZones[currentCameraIndex].width);
    ImGui.NextColumn();
    ImGui.InputInt("height", ref cameraZones[currentCameraIndex].height);
    ImGui.Columns(1);

    // Since we're doing immediate mode, we can sanitize our data like
    // this, and the UI just draws the correct value during the
    // next frame.  No callbacks, no validators, it just works.
    if (cameraZones[currentCameraIndex].width < 3)
    {
        cameraZones[currentCameraIndex].width = 3;
    }

    if (cameraZones[currentCameraIndex].height < 3)
    {
        cameraZones[currentCameraIndex].height = 3;
    }

    // And we're done with this window
    ImGui.End();
}

That example results in the following UI:

Once again, simply declaring the ImGui.InputInt fields is enough to display them, and to read back any changes the user inputs.  And that’s true whether they type in a value, or use the +/- keys to change the value.  In both cases, the provided variable is updated for us, right then and there.

Notice that we’re using currentCameraIndex in this code block.  In the previous screen, if the user clicked on a different camera zone, that would update currentCameraIndex, and we would now be drawing the contents of a different camera.  All automatically, just because we’re doing it right then and there.  There’s no UI to remember to update when the camera changes, we’re redrawing everything every frame, so it just works.

Making changes is also super simple.  For example, if we wanted the user to be able to change the camera name, we just change the following:

// Replace this...
ImGui.Text(cameraZones[currentCameraIndex].name);

// With this
ImGui.InputText("name", ref cameraZones[currentCameraIndex].name, 64);

This changes the widget from a simple display widget into an edit widget.  It still takes the same data, but now it can update it as well.  In addition, as the user updates this data, we’re updating the real zone info, so the UI we had above that shows a list of camera zone names gets updated as well, automatically!

Making user interfaces in this manner can significantly reduce the time taken to implement them, and the number of potential bugs that can happen.  This is because everything happens in one place, and is always up to date.  If you’re seeing it on the screen, it has to be using real live data, otherwise it wouldn’t show up on the screen, since it’s redrawn fresh from live data on each frame.

In a future post I’ll show a demo of how the camera zone editor works, and how the level designers can change how the camera will work at various parts of the level.  Until then, stay safe, stay happy, and don’t forget to add Blightmare to your Steam wishlist!