What is “UI testing”?

UI testing is testing via the user interface. This is nothing new; we do it all the time, manually, by running an app and tap-tapping through its UI.

But manually testing for regressions is dull. You might remember to test some of the functionality you suspect your latest changes impact, but you’re unlikely to test presumably unrelated functionality. Repeating those other tests takes time—if you even remember what they were. Remembering things and doing them for you while you have a coffee—this is why we have computers, right?

Automating unit tests is relatively straightforward: we can test a “unit” (read “class”) via its programmatic interface, so testing it is as easy as writing some code and making sure it has the intended effect.

Automating UI tests is not so straightforward: you need to pretend to be someone tapping and dragging and pinching at your app’s UI to trigger changes, and you need to be able to “look” at the UI to see if it’s changed how you expected it to.

How does Xcode 7 help?

As of Xcode 7, we have a programmatic interface to our app’s UI, thanks to the newly introduced XCUI namespace.

Prior to Xcode 7, we sort of had this, but it was exiled to Instruments and used JavaScript. This put it solidly outside our main focus when developing, which is working in Xcode (not Instruments!), writing Swift (not JavaScript!). No bueno. The old UI testing system was also iOS-only; the new XCUI APIs apply to both iOS and OS X.

This new API provides an arm’s-length interface to the UI elements of our running app. Talking through proxies, we can locate buttons and table rows and interact with them much as a user would.

Accessibility: The Secret Star

The XCUI API lets us interact with our app much as a user would, but not necessarily any user: We view the app, not through our eyes, but through the metadata exposed by the accessibility infrastructure to drive VoiceOver interaction with the iOS system.

Fortunately, the system-provided UI components go a long way toward making our apps accessible. If you’re using only UIKit components for their intended purposes, you’ve got little to worry about. If you’ve gotten clever, UI testing might be the spur that drives you to make your app accessible to more of the world. We’re not going to go into detail here on making an app accessible; if you’d like to learn more, check out Accessibility for iPhone and iPad apps.

UI testing kills two birds with one stone: It tests that our app works, and it also tests our app is accessible.

Example: Todo.app

Let’s give these general ideas concrete form through a worked example. We’ll be walking through adding UI tests to the oh-so-uniquely named Todo.app.

What’s it do?

You might wonder what this app does. The short answer is “not much,” which is normally a bad thing, but in the context of this blog post, we’re focusing on UI testing rather than the app’s behavior.

Todo.app supports these capabilities:

• View a list of tasks and due dates.
• Mark an unfinished task as finished.
• Mark a finished task as unfinished.
• View a list of finished tasks and due dates.
• View a list of tasks due today.

Concretely, that means we have three main views:

• Today’s Tasks
• Todos (unfinished tasks)
• Dones (finished tasks)

A final view enables navigating between these views. This first draft of the app uses the venerable drill-down table-view interaction style. To toggle a task between being marked finished and unfinished, we’ll use a long-press action. We’ll indicate that a task is finished by displaying its title with strikethrough applied, so it looks crossed out, just as you might do with a pen on paper.

Hold onto your hats, we’re going to UI test this jewel!

UI Testing Todo.app

Go and grab yourself a local clone of this app so’s you can follow along with me:

git clone https://github.com/bignerdranch/blog-ios-xcui-todo.git xcui-todo
cd xcui-todo
git checkout ui-tests-coming-soon
open */*.xcodeproj

Note that you’ll need to open this up in Xcode 7. This is the first version of Xcode to include the XCUI system we’ll be using. It’s also the first version to understand Swift 2, which is what the app has been written in.

If you get a ton of compiler errors when trying to build, double-check that you’ve opened it in Xcode 7 and not Xcode 6. (As of the time of writing, Xcode 7 Beta 6 was current.)

Recording Your First Test

Xcode supports writing UI tests by capturing your interaction with the app. It won’t write any test assertions for you, but it will record your actions as test code, at which point you can write assertions around the recorded app interactions.

Let’s start by testing that we can drill down to the “Due Today” table and toggle the first item’s finished state. Along the way, we’ll verify that exactly two items show as due today.

Start by opening the XCUITodoUITests.swift file. Drop your cursor in just before the end of the testExample function so that Xcode will write the test code in a meaningful location. Select the iPhone 6 simulator, then click the little red dot at the bottom left corner of the editor pane, which is your “Record UI Test” button.

If the “Record UI Test” button disabled, then verify you’re in the XCUITodoUITests.swift file. If you are, and it’s still disabled, then try running the tests by selecting the menu item Product < Test. This worked around the issue for me.

Once the simulator is up and running:

• Tap the “Due Today” row
• Long-press the first table row

Notice how the row reloads to show the to-do item no longer crossed out. This means it’s now considered unfinished: still to do, rather than already done.

Stop recording by clicking the Record button again. The Stop Recording button’s icon differs slightly from the Start Recording icon, but it includes the same little red dot icon.

Recording Is Limited

You’ll see that Xcode inserted a line of code to perform the navigation:

        XCUIApplication().tables.staticTexts["Due Today"].tap()

In fact, a portion of that shows up inside a clickable token, similar to the placeholders Xcode uses in snippets and code completion. Unlike those inert placeholder tokens, when you click this token, a menu pops up to allow to you select a different way of writing the recorded action:

        XCUIApplication().tables.cells.staticTexts["Due Today"].tap()

This other version has a cells. in the middle. We’ll stick with the shorter one: Drag-select the token text and hit Return on your keyboard to accept the selected option.

Unfortunately, the recorder completely failed to capture our long-press interaction! We’ll have to write that ourselves.

UI Testing Concepts

Elements

UI tests exist outside the app. They interact with it at arm’s length by using proxy elements. These proxies represent the actual in-app UI elements. Your UI test can get a handle on an XCUIElement with type .Button that represents a UIButton, but this XCUIElement exposes only certain properties of its represented object—title, but not title color; label, but not button type—and it interacts with the proxied view through a limited API—tap or double-tap, but not sendActionsForControlEvents.

Elements nest in a tree and have various properties, such as a label and a value. They get a reference to the root of the tree by calling XCUIApplication().

Queries

You navigate the tree using queries. Aside from the tree node-based queries you’d expect, like “how many children have you got?”, you can also run queries across an entire subtree based on filtering by UI element type, accessibility identifier or an NSPredicate.

The UI element type is represented by an enumeration, XCUIElementType, with members like .StaticText, not by classes like UILabel—remember, arm’s length!

There are short-form accessors for common types that let you ask for all tables rather than manually building a query for descendantsMatchingType(.Table).

There’s also a gotcha here, in that the element types are not separated based on platform, so you’ll find the iOS .Cell snuggled up alongside the OS X .TableRow and .TableColumn.

Simulated Events

Being able to inspect the tree of elements is great, but to get anywhere writing tests, you’ll need to poke and prod them. Xcode’s got you covered here with actions like tap() and doubleTap(). There’s no longPress(), though—you’ll have to build that yourself using pressForDuration(_:).

Run the UI Test

The test doesn’t test anything now, but if you run it, you will be able to watch the Simulator fire up and the app navigate to the Due Today page.

Run the tests using the Product < Test menu item, or by pressing the shortcut Cmd-U (presumably “U” for “Unit Tests”).

You’ll notice that it first runs the logic tests, then starts the UI tests, then launches the app. In the Report Navigator, you will actually see both a “Test XCUITodo” entry and a “Debug XCUITodo” entry, rather than just a “Test XCUITodo” entry, as is usual when running only logic tests.

UI Test Faster: Skip the Empty Logic Test Target

Since we don’t have any logic tests, we can speed up our UI testing by editing the Test scheme to run only the XCUITodoUITests target:

• Navigate to menu item Product > Scheme > Edit Scheme…, or press Cmd-< (that’s Cmd-Shift-, on a Qwerty or Colemak keyboard).
• Select the Test action from the left pane.
• Select the Info tab from the right pane.
• Ensure that, in the Test column, the checkbox in the XCUITodoTests row is not checked, while the checkbox in the XCUITodoUITests row is checked.

Now, when you mash Cmd-U, it will run only the UI tests.

(On a real project, I sincerely hope you don’t have an empty logic test target. In that situation, you might want to create a new scheme and configure your two schemes so the test action of one runs just the logic tests and the test action of the other runs just the UI tests.)

Locating the First Item Due Today

Conceptually, the first item due today is the first (“zeroth” with zero-based indexing) cell in the only table we see after navigating to the Due Today table.

So you’d think this would work:

• Use the recorded code to tap the Due Today cell
• Get the list of tables from the app (there should be only one)
• Get the list of cells from the tables
• Get the first element of that list (this is our first Due Today item)

You’d expect code like this to work, then:

        XCUIApplication().tables.staticTexts["Due Today"].tap()

        let cells = XCUIApplication().tables.cells
        XCTAssertEqual(cells.count, 2, "found instead: (cells.debugDescription)")

        let firstCell = cells.elementBoundByIndex(0)

Gotcha: A Race Condition

Go ahead and run that. Watch it crash and burn.

Turns out that if you just charge straight ahead, there’s a race condition: The query will find both the top-level navigation table and the Due Today table at the same time, so it will report five cells rather than two.

If you wait a bit, things settle down, and only the table we’re actually looking at will be found by the query:

        XCUIApplication().tables.staticTexts["Due Today"].tap()

        /* For a bit, both the old and the new table will be found.
         * This leads to us finding 5 (3 + 2) rather than just 2 cells. */
        _ = self.expectationForPredicate(
            NSPredicate(format: "self.count = 1"),
            evaluatedWithObject: XCUIApplication().tables,
            handler: nil)
        self.waitForExpectationsWithTimeout(5.0, handler: nil)

        let cells = XCUIApplication().tables.cells
        XCTAssertEqual(cells.count, 2, "found instead: (cells.debugDescription)")

        let firstCell = cells.elementBoundByIndex(0)

We’ve introduced an explicit delay while we wait for the query (self in the predicate) to find exactly one (self.count = 1) table.

Thanks to this delay, we can now pick out the first Due Today cell using the query we thought should work in the first place.

It’s surprising that we have to do this. It’s all too easy for a machine to race ahead of a UI intended for humans, especially with the growing use of transition animations. The XCUI system is designed to cope with this by automatically introducing a “Wait for app to idle” step after each interaction. Most of the time, that wait-for-idle step is enough to ensure the app is in a consistent state before continuing. For whatever reason, it failed in this case. Fortunately, XCUI gives us the tools we need to work around the problem.

Gotcha: Viewing the Test Steps

You can view the test report by opening the Report Navigator, selecting a “Test XCUITodo” row and clicking the “Tests” tab.

If you expand out a UI test, Xcode is supposed to provide a nice list of the UI actions and queries it’s running on your behalf in the test report. For me, that worked fine once my test was passing. If it failed, though, I saw only the first line, “Start Test ()”; to view all the tests, I had to flip over to the “Logs” tab and expand the log messages for the failing test. It’s the same information, but with not so pretty a presentation, and at the cost of a few more clicks.

Gotcha: The Label Is Plaintext

The app uses strikethrough on the cell label text to indicate when the to-do is finished. We can read this label out pretty easily:

        let staticTextOfFirstCell = cells.elementBoundByIndex(0)
            .staticTexts.elementBoundByIndex(0)
        let beforeLabel: String = staticTextOfFirstCell.label

Note the type there, though: String. We’d need an NSAttributedString to be able to pull out the strikethrough info. Uh-oh! That means we can’t actually verify whether the finished state toggles on long-press without some changes to our app code.

Making the Finished State Accessible

On the bright side, UI testing through this API has forced upon us the realization that our current UI is not exposing task finished state to the accessibility layer. Someone relying on VoiceOver would not be able to tell which of their Due Today tasks were finished and which weren’t!

Luckily, this is easy to work around. We will take advantage of the accessibilityLabel property of the cell’s textLabel. This property allows us to provide a label specifically for the accessibility system (and our UI tests!) to read.

We’ll make the finished state accessible by prefixing finished items with done: .

Gotcha: Sharing Code Between App and UI Tests

What if we change our mind later, though? It would be great to be able to reuse the same code we use to prefix the to-do title when we check whether the to-do is finished from our UI test.

Since UI tests are an entirely separate module from the app and are not run inside the app as logic tests are, the only way for them to share code is to compile in all the app files we need to share between the two.

If we add TodoCellView.swift to the UI test target, we’ll also have to pull in Todo.swift because the cell view uses that class, even though our test code doesn’t need access to the app’s internal model classes. This is because it can’t access them, only the UI.

To work around this, we’ll put the shared code in a new file, Accessibility.swift:

import Foundation

class Accessibility {
    static func titlePrefixedToIndicateFinished(title: String) -> String {
        /* Genstrings uses first argument verbatim when generating
         * Localizable.strings files,
         * so interpolating a constant prefix here like "(prefix): %@"
         * would not be terribly useful. */
        let template = NSLocalizedString("done: %@",
            comment: "accessibility label for finished todo item")
        let label = NSString.localizedStringWithFormat(template, title)
        return label as String
    }

    static var FinishedTitlePrefix: String {
        /* Exploit that prefixing the empty string returns just the prefix. */
        return titlePrefixedToIndicateFinished("")
    }
}

Then our TodoCellView can grow a method to format the accessibility label:

    private func accessibilityLabelFor(todo: Todo) -> String {
        guard todo.finished else { return todo.title }

        return Accessibility.titlePrefixedToIndicateFinished(todo.title)
    }

and use that method from its configure(_:, afterToggling:) method:

        textLabel!.attributedText = (todo.finished ? strikethrough : normal)(todo.title)
        textLabel!.accessibilityLabel = accessibilityLabelFor(todo)

Now our UI test code can test whether a todo is finished by checking the label text with this helper:

    func isFinishedTodoCellLabel(label: String) -> Bool {
        return label.hasPrefix(Accessibility.FinishedTitlePrefix)
    }

Long Pressing

We’re so close:

• We can navigate to the Due Today table
• We can find the first Due Today cell.
• We can read out that cell’s to-do title and tell if it’s finished.

All we need to do to finish testing that a long press actually toggles the finished state is to perform a long press.

You might think there’d be a longPress() method right next to tap() on XCUIElement, but there isn’t. Perhaps it’s because a “long press” can be any of various lengths; check out UILongPressGestureRecognizer.minimumPressDuration.

Regardless, we know what “long press” means in the context of our app, so we’ll simply teach XCUIElement how to long press via an extension:

extension XCUIElement {
    func bnr_longPress() {
        let duration: NSTimeInterval = 0.6
        pressForDuration(duration)
    }
}

A Fully Functional UI Test at Last

Now we can finish our test:

    func testLongPressTogglesFirstTodayItemFinished() {
        XCUIApplication().tables.staticTexts["Due Today"].tap()

        /* For a bit, both the old and the new table will be found.
         * This leads to us finding 5 (3 + 2) rather than just 2 cells. */
        _ = self.expectationForPredicate(
            NSPredicate(format: "self.count = 1"),
            evaluatedWithObject: XCUIApplication().tables,
            handler: nil)
        self.waitForExpectationsWithTimeout(5.0, handler: nil)

        let cells = XCUIApplication().tables.cells
        XCTAssertEqual(cells.count, 2, "found instead: (cells.debugDescription)")

        let staticTextOfFirstCell = cells.elementBoundByIndex(0)
            .staticTexts.elementBoundByIndex(0)
        let beforeLabel = staticTextOfFirstCell.label

        staticTextOfFirstCell.bnr_longPress()

        let afterLabel = staticTextOfFirstCell.label
        let finishedStateDidChange = (isFinishedTodoCellLabel(beforeLabel)
            != isFinishedTodoCellLabel(afterLabel))
        XCTAssert(finishedStateDidChange, "before: (beforeLabel) -> after: (afterLabel)")
    }

Review

As it says on the tin in the final version (if you un-CamelCase the test name and apply some formatting, that is), it tests that “long press toggles the first today item .finished.”

We met some rough spots along the way, but in the course of debugging them, we have the home court advantage now: we’re working in Swift within Xcode, rather than in JavaScript within Instruments, as would have been the case using UIAutomation.

Using queries, elements and actions, we can simulate interacting with our application through the narrow view of the accessibility API. This lets us simultaneously create UI-level tests and encounter accessibility gaps before our users do.