Mobile Application Testing
Overview: Smart Phones are must haves! No one wants to switch on their PCs for every small operation they need to do. It is essential that your mobile app offers a perfect user experience and nothing less. According to Compuware, 79% of users would only retry an app once or twice if it failed to work the first time, not more than that. This calls for thorough mobile application testing.
What is Mobile Application Testing and Why is it needed?
With the constant advent of handheld devices, businesses naturally wanted to leverage them. Hence, they started building mobile apps to make their services more accessible to customers. To deliver an optimal solution to customers, apps are built across all the major platforms and devices. Hence, Mobile app testing comes into picture.
Mobile application testing in most general terms refers to the process of verifying whether an app designed for mobile devices meets certain mobile testing requirements. Mobile apps need to be tested separately as mobile apps work a lot differently than web apps. Few are the listed differences-
- There is a wide variety of operating systems and component configurations like Android, iOS etc. and device types are also in hundreds unlike web applications.
- Mobile phone OS/specifications change very frequently and become obsolete quickly. Hence, a compatibility testing plays a much bigger role which is not the case for desktop web applications.
- Mobile devices use over the air network (4G, 5G, Wi-Fi) whereas desktop web applications usually use broadband connections. Hence, mobile applications need to be tested with different data rates as well.
- Making and receiving calls is primary function of a mobile device. Hence, applications should be tested for how they impact calling which is not the case with desktop web applications.
- User’s way of interaction with application is different in mobile apps (scrolling by touch, pinching to zoom) which is way different from web applications.
Types of Mobile Applications
At a high level, mobile applications can be classified as:
- Native Applications
- These are platform-specific apps, and users can install them from specific app stores (like Google Play Store or App Store).
- These applications are developed using specific programming languages for a particular operating system like Android or iOS.
- Mobile Web Applications
- Standard web applications that can be accessed using mobile browser.
- They sometime look like mobile apps but operate very differently.
- Hybrid Applications
- They are a combination of above two- they are basically web apps put into native app bottle.
- Once downloaded, they use the mobile browser embedded in app to operate.
How to test a Mobile Application?
Mobile applications can be tested manually and through automation as well. A lot of mobile app testing is done manually using the GUI of mobile apps as the testing team needs to test various interrupts or events which may occur when application is running like incoming call, message, email, notification, battery alert, change of network from home to roaming, etc.
- Testing with Simulators and Emulators (Virtual Testing Device)
- A Virtual testing device is nothing but a computer program that mimics most important features of an actual device.
- This helps to run the mobile apps and get an idea on how they will run on the real device.
- Virtual testing devices can be categorized as
- Simulators: A simulator creates a replica of a device’s UI and does not represent its hardware.
- Emulators: An emulator is a software that mimics the hardware and software of the target device on your computer.
- Testing using Simulators and Emulators is helpful in early stages of development cycle. It is in no way replacement of testing with real devices.
- Issues related to the OS kernel code, the amount of memory on a device, the Wi-Fi chip, and other device-specific features can’t be replicated on an emulator.
- Testing on real devices
- Best option for mobile app testing is to test on real devices. Real testing devices are the various models of mobile handsets that the end user would use to run the website or app.
- However, this is a costly solution as we cannot buy all types of devices available in market for testing purposes.
- This problem can be solved by cloud solutions which offer real devices for testing like AWS device farm, Sauce labs etc.
- These cloud solutions allow you to run manual and automated tests just by uploading your test and selecting the configuration needed for your scenario.
Types of Mobile Application Testing
- Functional Testing:
- Functional testing checks whether the functionalities are working as per requirements or not.
- It is often applied to screen adaptation of app, basic user journeys.
- It also covers installation test, application update testing, local settings testing, etc.
- Functional testing interacts with app’s UI elements, database layer, network layer and more and is hence time-consuming and complex.
- Operational Testing:
- Operational testing tests the app’s response to changes in certain devices operations such as data cable connection, Wi-Fi connection, and airplane mode status.
- Interrupt Testing:
- Interrupt testing tests all possible interruptions, such as incoming calls, SMS, pop-up notifications, reminders etc.
- It ensures the app can handle all interruptions by going into a suspended state and restarting afterward.
- Security Testing:
- Security testing intends to test security weaknesses in an application.
- It is a complex area and needs knowledge on various areas like how client-server communication happens, app architecture, system architecture etc.
- Usually, a Pen Test is performed to find security weaknesses in the app.
- Performance Testing
- Mobile app performance testing is the process of determining how a system responds under a particular workload or task.
- It includes testing the speed, stability, and scalability of an application.
- It is performed on both the client side and server side.
- On the server side, it checks variations in response times, delays in delivery of messages, application crashes, etc.
- On the client side, it checks the usual discrepancy of application behavior on various platforms and handsets, memory leakage, CPU consumption, loading speed and battery issues.
- Compatibility Testing
- There is a huge diversity of mobile devices which leads to problem of Fragmentation and hence compatibility testing is must.
- Creating the Compatibility matrix helps in compatibility testing-
- Take every device or model available in the market
- Map the information of platform details, technology features supported by the device, hardware features included in the device, and network and other technology features supported by the device.
- Divide all devices into two lists: fully compatible vs. partially compatible devices.
- Fully compatible devices support all technology features required to make all the application functionalities work seamlessly, while partially compatible devices may not support one or a few features and therefore cause error messages.
- It is recommended to cover at least 70% of compatibility spectrum in testing for an app to be successful.
- Usability Testing:
- Known as user experience testing, lets you observe the user-friendliness of a mobile application.
- Regression Testing:
- Regression, as the name suggests, checks whether new feature updates, patches or configuration changes do not introduce any new bugs in functionality of mobile app.
- It should be done continuously with every new change introduced.
Challenges of Mobile Application Testing
- Device Fragmentation:
- Testing every combination of device, OS and network settings creates many test cases. This requires test teams to perform the work of sourcing and maintaining a growing pool of mobile devices.
- Multiple Connection Types:
- Most of the apps use mobile data connections (3G, 4G,5G) and Wi-Fi.
- When users move around there’s a possibility that their connection type will change.
- Apps should be tested for all connection types for performance and functionality and hence a huge effort is involved.
- Integration with other Apps:
- All mobile apps require integration with other applications like SMS for authentication, crash reporting etc.
- This makes test teams switch between multiple devices throughout the course of the test scenario, hence making testing complex.
Mobile Application Testing Best Practices
- Define the clear testing strategy for app testing. Context is very important while deciding different testing strategies regarding cross-device and cross-browser testing.
- Carry out UI testing on real devices. Emulators/Simulators will not give the right output.
- Make sure testing is integrated throughout the development process. Initial tests can be conducted using virtual test devices whereas final tests should focus on real devices.
- Identify OS and device type of your user base and prioritize tests accordingly.
Conclusion: Mobile testing sounds easy in the beginning as everyone is using a mobile device. However, it is multi-layered. Designing the right test strategy is the key. Selecting the right mobile simulators, devices (based on user base) and testing tools helps to ease the challenges. It is important to think from holistic perspective while testing the apps. Mobility testing is relatively new field, but it is here to stay!
*********