Test Scenarios To Ensure Your Mobile App Runs Seamlessly

In order to create high-quality software, you must thoroughly test your mobile apps. Testers can find test values by using boundary values and equivalence partitioning. Testing requires lots of time and effort, therefore, many software developers use automated testing to make the process more efficient.

What is a Test Scenario?

A test scenario is a collection of high-level test cases describing the functionalities of an application. A test scenario is usually used by testers for end-to-end feature testing to help testers know the flow of the application. As testers, we must put ourselves in the perspective of the end-user and figure out the scenarios that the end user would use when interacting with the application. Test scenarios will give an idea of ​​what needs to be tested.

Test scenarios can serve as the basis for lower-level test case creation. A single test scenario can cover one or more test cases. Therefore a test scenario has a one-to-many relationship with the test cases.

Test scenarios are usually created to get the following benefits:

• It ensures complete test coverage because each requirement should have test scenarios leading to new test cases.
• a quick tool to estimate testing work effort.
• Helps to understand end-to-end business and functional flow of the application.
• Provide a high-level overview of what conditions need to be tested.

Differences Between Test Cases and Test Scenarios

In this part, we are going to understand the essential differences between the Test Cases and Test Scenarios, both test cases and test scenarios are often used while testing, but also often create confusion among the testers. Therefore, it is important for testers to know the differences. Below are the differences between test cases and test scenarios. 

In general, test scenarios are high-level documentation of what you are going to test in your application, usually, it is just a short sentence whereas test cases are low-level documentation, derived from test scenarios with a more detailed description of how you will test the application, test cases consists of multiple components and each component has multiple steps, by using test cases, a person who has no knowledge of the project can run the application because the test cases have multiple steps to reproduce. That’s the major difference between test scenarios and cases. 

As an example, consider a test scenario – “Verify that the user is not able to login with invalid credentials”. This test scenario can be further broken down into multiple test cases such as :

1. Checking if users that logged in with the correct email and incorrect password should not be allowed to log in.
2. Checking if users who log in with an incorrect email and correct password should not be allowed to log in.
3. Checking if users that logged in with the incorrect email and incorrect password should not be allowed to log in.
4. Checking if users who log in with the correct email and an expired password should not be allowed to log in.

Types of Test Scenarios on Mobile Applications

The following are the most common types and usually mandatory scenarios for mobile applications, which we will discuss later on.

1. Functional Test
2. UI Test
3. Compatibility Test
4. Performance Test
5. Security Test

Functional Test Scenario

Functional testing of the application should be the first thing testers do. Functional Testing is a testing process to make sure the application functionality such as user interaction or other actions that the users might do are already working properly based on the requirements. For example: inputs, outputs, tappable buttons and navigation, ensuring all the functions perform as they should. Sometimes functional testing is also called black box testing because the tester focuses on the application functionality specification without concern for the actual code.

The main purpose of functional testing is to ensure that the application fulfils the requirements of the stakeholders that the user has defined at an early stage, also reducing the risk of errors and customer satisfaction.

During the process of functional testing, two major techniques are used by the team of testers. Each of these caters to different aspects of the software and works together to improve the quality and performance of the software. These techniques are:

1. Requirement-Based Testing: During this type of testing the requirements are prioritized by the team on the basis of the risk criteria, after which the most critical and important features and functions of the software are evaluated and tested.
2. Business Process-Based Testing: This type of functional testing relies on the knowledge of business processes, based on which the scenarios for testing are described.

Along with the usual test scenarios which test the basic functionality, there are some mobile application testing scenarios which should be considered while performing mobile application functional testing.

• Verify the app can be installed and has been launched correctly.
• Verify whether the application works as per the requirements whenever the application starts/stops.
• Verify functionality is not impacted for users in various timezones & localisation setup.
• Verify functionality is not impacted during light or dark mode if supported.
• Verify functionality is not impacted in various display modes (i.e. landscape and portrait).
• Verify that the app behaves as designed if the mobile device is shaken.
• Verify that the app still functions as designed when “battery low” notifications appear on the screen.
• Verify the page scrolling and swipe scenarios are being enabled in the application as expected.
• Verify that the app works in multitask mode when needed. For example, if the user gets a call while using the app, the application should perform the expected actions.
• Verify whether the phone is able to store, process and receive SMS whenever the app is running.
• Verify that the user receives an appropriate error message like “Network error. Please try after some time” whenever there is any network error.
• Verify that the application resumes at the last operation in case of a hard reboot or system crash.
• Verify that the application performs auto start facility according to the requirements.
• Verify that the application performs according to the requirement in all versions of Mobile that is 2g, 3g and 4g.
• Verify that the app’s social networking options such as sharing, posting, and login are working correctly.
• Verify that the application provides enough user guide (or onboarding steps) for new users.

Testing the functionality is one of the most important aspects of the application. Depending on your app, functional testing will be different in every project.

UI Mobile Test Scenario

UI testing can be tricky to test for mobile applications because it has to check the position of components on several different screens from large screen sizes to small screen sizes. As a user who uses the application, it is not only performance that is considered but also an attractive UI. UI-friendly apps sell a lot more when compared to the best-developed apps with poor UI. While testing UI on a mobile application, various characteristics need to be verified.

• Verify your app’s UI with some of the widely used mobile screen resolutions and aspect ratios, ie :  360×800, 414×896, 390×844, 412×915, 360×640, etc.
• Verify that the app display is adaptable and amenable to the various display modes (i.e. landscape and portrait).
• Verify that the app display (text, button, image) is clear and visible in both light & dark modes.
• Verify that the UI components are fitted on the screen and not cut off, especially when it has long text.
• Verify navigation through the menu, carousel, and swipe functions are working as expected.
• Verify that the page load duration is not more than five seconds. If the page takes longer to load, it must have a progress bar allowing the user to have an idea about the scenario.
• Review the company logo, image size, and rendering.
• Run a check for grammatical and spelling errors.
• Check the overall color scheme and theme of the app on the device.
• Check the consistency of the UI, for example, the same colour & font for the cancel button on all screens, also for each error/warning/success message, etc.
• Ensure all buttons & menus are comfortably clickable, the area clickable should not be too small.
• Ensure there is no lag or glitch in content loading when the user scrolls up.
• If the screen enables editing without saving, validate displaying a message to save details when the user navigates to another screen. 
• Check that data does not get deleted while moving the mobile app to the background while carrying out other functions like: filling out a form.

UI-friendly applications make the user experience more enjoyable and contribute to better app rating reviews, so UI testing is highly recommended and important especially when it comes to smart devices because here the screen size is comparatively small and a lot of variations of the devices are available in the market.

Compatibility Mobile Testing Scenario

Compatibility is about the ability of the system under test to function as required on all the supported platforms. Mobile compatibility testing is performed to safeguard against mobile application errors as devices have different OS’, sizes, resolutions, etc. Some compatibility failures will result in survivability and recovery issues for mobile devices. 

Mobile app compatibility testing can be further categorized into the following types:

1. Cross-platform testing: Testing your mobile application’s compatibility with different types of mobile operating systems: Windows Mobile, iOS, Android, BlackBerry, etc.
2. Cross-browser testing: Testing your mobile application’s compatibility in different types of mobile phone browsers: Apple Safari, Google Chrome, Native Android Browser, Mozilla Firefox, Opera Mini, Samsung Browser etc.

Below is an example of compatibility test case scenarios:

• Check the interface of the application on various screen sizes.
• Check the application supports various mobile device brands & types.
• Check the application’s compatibility with various OS versions.
• Check the compatibility of the application with a combination of different devices and different OS.

Performance Mobile Test Scenario

Mobile performance testing can significantly impact user satisfaction when mobile users expect to launch an application within 2 seconds but, the actual application needs a longer than expected time may cause the user to delete or uninstall the application. There are 2 major performance tests that are commonly performed:

1. Backend performance test to check how the servers or backend systems perform when handling large workloads to meet the expected result.
2. UI performance test to check the rendering time of the UI components after the mobile app received required data from the backend, check resource (battery, memory, CPU) consumptions on the mobile.

Performance testing is performed to determine the operability, stability, and resource utilization of mobile application quality, in different usage scenarios and workloads.

Various testing scenarios can be tested on performance mobile testing :

• Check launch time when launching the application.
• Check loading time when loading content.
• Validate the performance of the application by changing the network to 2G,3G, 4G, 5G, and wifi.
• Check resource consumption while using the app. For example, testing using a fully charged battery, after installing the application and leave it on standby mode so the application can run in the foreground/background then wait for a couple of minutes (or hours) to see whether the app is consuming lots of battery or not. Several tools are available to check & monitor resource consumption.
• Perform load testing on individual API calls and validate how the application behaves under a certain load.

Security Mobile Testing Scenario

Security testing is how to find security weaknesses in the application, it’s a complex test and best done by experts.

Various testing scenarios can be tested :

• Validate the flow for security vulnerabilities that involve the user’s personally identifiable data, such as mobile number, email, etc.
• Validate that the data is secured and encrypted, sent to and back from the server over the network.
• Validate that no confidential data is stored without encryption and visible/accessible.
• Check the application certification.

Furthermore, OWASP Mobile Security Project can be a starter guideline for security testing.

Key Takeaway

• Mobile App Testing often comes with its own challenges. It can be challenging to design an effective mobile testing strategy due to the high combination of devices’ brands, types, screen sizes and os versions.
• UI-friendly applications also have good compatibility and can result in satisfied customers but poor brand imaging may cause lost sales opportunities, therefore testers should also be aware of this and find out the best strategy to cover the issue.
• Testing the app against data usage and power consumption should also be considered, as high resource consumption will make users leave the app.
• Performance and security testing in mobile applications are ones of the most crucial types of testing.
• Proper guidelines for the testers helps them to make testing more effective and efficient. 

Reference

• Daniel Knott, “Hands-On Mobile App Testing: A Guide for Mobile Testers and Anyone Involved in the Mobile App Business”, Pearson Education, Inc, 2015
• Test scenario examples
• Test Scenario: Definition, Purpose, and How to Create – QA Madness Software testing company
• Functional Testing Complete Guide 2021
• https://www.guru99.com/testing-mobile-apps.html
• https://www.testrigtechnologies.com/25-test-scenarios-for-mobile-app-testing/
• https://www.professionalqa.com/mobile-app-testing
• https://www.professionalqa.com/functional-testing 
• https://www.softwaretestinghelp.com/ios-android-ui-testing/#Guidelines_What_to_Test_in_Mobile_App_UI_Testing
• https://www.softwaretestinghelp.com/ios-android-ui-testing/ 
• https://www.testbytes.net/blog/test-cases-for-mobile-applications/