A Framework For Speedy and Scalable Development Of Android UI Tests

Fast feature development and productivity for mobile engineers has long been held up by mobile UI testing, which is a slow but essential process. While new automated UI testing technologies like UI Automator or Espresso have helped developers write UI tests, these tools do not keep the code clean, organized, and easy to read. This ultimately hurts productivity and scalability and continues to make UI testing a development bottleneck. Fortunately, companies struggling with UI testing can enhance UI test automation tools by using a Fluent design pattern to create easy to read, manageable tests that are fast to implement and will enable scalability. At DoorDash, testing all the UI scenarios of a new build would take two full days for three developers and one QA, which slowed our development cycle to one release every two weeks. This process, while essential to catching harmful user bugs, hurt the team’s morale and productivity overall. To solve this problem we built a Fluent design pattern-based framework that enabled us to utilize UI automation tools by making tests easy to read and scalable. To demonstrate how we increased our testing velocity we will first go through the problems with using UI Automator and other approaches to testing. Then, we will introduce design patterns, Fluent design patterns, and how we implemented these at DoorDash

The challenges of UI testing

Fast, scalable UI testing is a key challenge in ensuring mobile app development is bug-free, because test automation tools do not produce easy to read, scalable tests and alternatives are equally time consuming. While tools like UI Automator or Espresso using Android Studio have made it easier for engineers to start writing tests on Android to simulate user behavior, on their own the tests are difficult to understand and manage at scale. While testing may be fine at the start, increasing the number of tests makes it more difficult to understand the test code, causing a maintenance problem in the long run. Test automation tools can produce test code where each action is described in three to four lines of instructions, rather than having concise lines with clear descriptors, using business language, as shown in Figure 1, below:

Figure 1: Automation tests created without a design pattern can result in obscure code that does not state the intent of the test.

The alternative to using automated testing platforms is to outsource to manual testers. Unfortunately, this does not really save time because manual testers require knowledge transfers and managing the delegation takes almost as much time and effort as having developers do the testing themselves. Additionally, manual testing is not as accurate as automated testing, as it allows for more human errors, and it is not cost effective for high volume regression.

How a design pattern can help with UI automation

The issues with manual UI tests can be addressed by choosing a good design pattern and the right framework for UI testing. A good automation test framework should allow engineers to write tests that are:

Easy to understand and read
Quickly written
Maintainable
Scalable

There are several design patterns that are commonly used for web automation tests, the most popular being the Page Object pattern described by Martin Fowler in 2013. Applying this design pattern to the example in Figure 1, above, we can see a definite improvement in readability of the test code, as shown in Figure 2, below:

An automation test using the Page Object pattern produces a better organized and more concise test then in our previous example. — Figure 2: An automation test using the Page Object pattern produces a better organized and more concise test than in our previous example.

The code in Figure 2 looks a lot better than what we started with because:

Each action can be performed in one line
Details are extracted within a function
This function can be reused whenever this action is required again

However, there are still some challenges to adopting a design pattern like this:

The test is still not clearly showing it’s intent; instead it looks more like coded instructions
There will still be a lot of code duplication, which is not ideal

Using a Fluent Interface to highlight the business logic

A Fluent design pattern provides us with the best of both worlds, as it demonstrates clear intent by using domain-specific language. A Fluent Interface is an object-oriented API whose design relies extensively on method chaining. The goal is to increase code readability by using domain-specific language, allowing the relay of the instruction context of a subsequent call.

How a Fluent design pattern demonstrates clear intent

Design patterns should have clear intent and should use a domain-specific language that can almost be read like conversational language. A Fluent Interface fits the bill because it allows us to use API names that flow and domain-specific language. The benefits of using a Fluent design pattern include:

When test code is easy to understand, it is easy to extend and reuse
The ease of use will help developers work more quickly and be more confident when writing tests
The design pattern is agnostic to underlying tools like UI Automator or Espresso

Utilizing a Fluent design pattern to build the Dasher app UI test automation

Here at Doordash, we had been using TestRail for manually testing the app before every release. It used to take three software engineers and one quality assurance engineer to go through the TestRail tests and half a day each to run the tests, taking two full work days. This process limited our app releases to every two weeks. Establishing a new UI automation framework for Android eliminated these pain points for release cycles. We will now go into a bit more detail to explain the approach and tooling we used, the overall high level solution architecture, and share some best practices.

Our approach to utilizing Fluent design patterns

Generally, every UI test scenario involves interactions with activities and screens; on each screen the user will take some action and expect some behavior as a result. We then use assertions to verify the results. To conduct these tests, we structure the test code in such a way that each screen encapsulates the actions that are performed on that screen and can verify the behavior after performing those actions. All interactions are named in domain-specific language using the Fluent design pattern interface, as shown in Figure 3, below:

Figure 3: In our UI automation framework for our Android Dasher app, we use a Fluent design pattern to name interactions in our domain-specific language.

Testing suite

Tooling choice plays a big role in improving developer productivity. We find that our tooling proves easy to work with and has good online support. UI testing tools: Before developing our UI test procedures, we considered different tools for writing UI tests, such as Appium, a third-party tool. However, we found that Android’s native tools were easier to use and had better support. There are two UI testing tools supported by Google, which can be run separately or together, since they run under the same instrumentation test runner:

UI Automator - UI Automator is a UI testing framework suitable for cross-app functional UI testing across system and installed apps.
Espresso - A key benefit of using Espresso is that it provides automatic synchronization of test actions with the UI of the app being tested. Espresso detects when the main thread is idle, so it is able to run test commands at the appropriate time, improving the reliability of the tests.

IDE:

Android Studio: For developers who believe that UI testing is an integral part of app development, Android Studio will make their lives easier. It allows running the unit tests, Android UI tests, and the app itself from the same development environment. It also allows the package structure such that the app code and its corresponding tests (unit tests and UI tests) can reside in the same repository, making it easy to maintain app code versions and their corresponding tests.

Target test devices:

UI tests are generally run on either a real device or an emulator to imitate the test scenario. For most of our test cases, we use emulators for common device configurations and sizes.

CI/CD:

Bitrise is one of the most popular CI/CD tools on the cloud that allows scaling and ease of use for setting up test environments. Especially for UI testing it allows integration for both a device farm and virtual devices and has become an easy tool to set up a build and testing environment for developers.

Testing process

We write test scenarios, shown in Figure 3, above, in a domain-specific language following a behavior-driven approach. These tests use the test setup API to create the environment for a particular test and use screen objects that interact with the screens and verify actions. Screen interaction and verification are ultimately performed through a test automation tool, such as UI Automator, Espresso, or any other similar tool. To understand this process, let’s look at an example for the login flow of an app using a Fluent design pattern and our testing architecture described above. Login test:

Figure 4: Using the Fluent design pattern in our automation test results in easy to read, simplified code.

The test in Figure 4 is written using the Fluent design pattern, and the base class that enables this pattern is called Screen.kt. The code for Screen.kt is shown in Figure 5, below:

*Figure 5: The Screen.kt base class enables the Fluent design pattern.*

All the screen classes extend this class and follow the pattern of returning itself for each interaction/verification function, thereby passing the context along. The inline generic method “<reified T: Screen> on()” is used to switch context from one screen to another. An example of the “Screen” implementation is shown in Figure 6, below:

Figure 6: While the LoginScreen class is being implemented with UI Automator in this example, it could easily be replaced with another tool for the same action but with a different approach.

The above implementation uses the underlying tool, UI Automator, to actually interact with the screen. While this example uses UI Automator, it can be replaced with Espresso or any other similar tool without affecting the business logic or test expectations.

Reviewing the folder structure

For clean coders and software crafters, the primary property of a package is the possibility to have a meaningful name that describes its purpose and its reason for existence. Therefore, we have organized our packages as follows:

test: Holds all the tests for various scenarios
screen: Holds all the screens within the app and the corresponding interactions
UI Automator/Espresso: Holds tool classes for performing screen interactions and verifying behaviors
utils: Common API to setup environment for a test execution, e.g. creating and assigning orders before Dashing starts. It also holds other common utility functions.

Figure 7: We organize our packages in a manner that allows for meaningful naming.

Best practices for using this approach

While writing these tests we developed a few best practices that helped us keep our code clean, readable, and extensible. Here are a few that we follow:

Naming convention: Even for the UIAutomator class we continue using the Fluent design pattern, which reads like a domain specific language.
- UiAutomator.kt: This class will essentially have two kinds of functions, any action that the user takes on the screen and verification of the behavior.
  - Verification function name uses this pattern: hasViewBy<Class,Text,ContenDesc,ResourceId etc that identifies the view>
  - Action function name has this pattern: <click,swipe,scroll,set><Button/View>By<Button/View identifies>
- Screen: It is very important to use the Fluent design pattern here, and figure out the correct naming of the functions that flow well while reading the test.
  - Name of the screen class is as per what that screen does, eg. PickUpItemScreen()
  - Verification function name is in domain specific language, eg. verifyAmount(), verifySignatureStepComplete(), verifyCompleteStepsGetCxSignature() etc
  - Action function name is also in domain specific language, eg. clickStartCateringSetup(), slideBeforeCateringSetupComplete() etc
We should always add a log within each function of the screen class, which helps in troubleshooting faster on CI/CD logs.
Any dialog/bottom sheet that is relevant to a screen is defined as a nested class of the parent screen.
All the verifications should have been asserted with a log message that clearly states the reason for the failure of the assertion, as shown in Figure 8, below:

Figure 8: Including a failure message in assertions makes it easy to troubleshoot failed tests.

Fluent design pattern increases developer velocity

Once the initial framework was set up, we finished 70% of our regression tests in two months. Here are some of the results:

Our code coverage moved from 0% to ~40%.
Our manual testing got four times faster, going from 16 hours to four hours.
Release cycles have gone from one every two weeks to weekly releases, and we can run regression tests any time we want.
The team is more productive because we only need to write the tests for new features or update the existing ones, which is much faster to develop.

Conclusion

Using the Fluent Interface for UI testing freed engineers from repetitive and time-consuming tasks, allowing for more time to solve tricky edge cases. Improved code coverage and running the automated test for regression testing ensured the robustness of our Android Dasher app. Since the code structure is agnostic to the underlying testing tool (UI Automator or Espresso), we can easily adopt any better tools released in the future.

A Framework For Speedy and Scalable Development Of Android UI Tests

The challenges of UI testing

How a design pattern can help with UI automation

Using a Fluent Interface to highlight the business logic

How a Fluent design pattern demonstrates clear intent

Utilizing a Fluent design pattern to build the Dasher app UI test automation

Our approach to utilizing Fluent design patterns

Testing suite

Testing process

Reviewing the folder structure

Best practices for using this approach

Fluent design pattern increases developer velocity

Conclusion

About the Author

Related Jobs

Similar Blogs

Path to high-quality LLM-based Dasher support automation

How DoorDash is pushing experimentation boundaries with interleaving designs

Growing Your In-House Legal Career