With the emergence of mobile app development, companies often rushed to be the first to market with their new apps, often overlooking steps critical to ensuring a successful application. As this market is more established, businesses now realize that, just as any other media for reaching customers, a mobile app must be a well planned, properly designed, and quality piece of software in order to be successful.
Quality Assurance (QA) testing is one of these previously overlooked steps that companies now realize is critically important. QA activities take on a new level of complexity with mobile applications that can make this a daunting task. With a wide variety of devices, manufacturers, operating systems, mobile browsers, screen sizes, screen resolutions, and screen proportion, there are literally thousands of unique combinations. Each combination can result in a different user experience or functional impact, making the scope of testing needs tremendous.
An important first step related to the QA efforts that should happen early in, or prior to, the design and development phase, and before any formal QA testing begins, is to identify a matrix of the target devices and Operating Systems/browsers. The business should determine a list of all of the different devices that they feel are important, and provide a priority ranking for each. Similarly, they should provide a list of all mobile operating systems and browsers targeted and provide a priority ranking for each. Data is available online that gives an overview of the market usage and prevalence of operating systems, browsers, and devices (see http://www.netmarketshare.com/). Along with the general market information, the business should evaluate their target market, demographic usage patterns and requirements, and business needs in determining their device/platform prioritization. As there is a direct cost associated with compatibility development, changes, and defect resolution specific to individual devices, this prioritized matrix allows the business to evaluate where money should be spent to enable devices, and which devices are of lower priority and not worth the investment.
Although the application is meant for mobile, the easiest testing method is on the desktop through a standards browser (for mobile website) or development tools (for mobile applications). Basic functional testing to ensure the code is defect free, meets business requirements, and data entry is appropriately handled can be addressed on the desktop. Providing feedback to developers is simplest when tested on the desktop, and the easiest to resolve. This ensures the application is functional in its own state and allows future device specific testing to focus on defects and issues related specifically to the device and software. Some browsers, such as Firefox, have a setting allowing control of the identified browser agent, which allows the server to provide mobile versions to the desktop browser.
Emulators provide a convenient and easy way to provide pre-device testing. As nearly all mobile manufacturers provide development device emulators, developers and QA resources should utilize these services for initial testing across the full spectrum of prioritized devices. Emulators don't always exactly mimic the device, and certain usability testing can be difficult to test, however this can provide an ability to initially eliminate errors related to the device.
The final step, which is critical to fully ensuring compatibility and usability, is to test on the actual devices. The tester should perform tests across the full spectrum of categories (functionality, usability, data entry, security) directly on the devices using a combination of manual testing and automated testing directly against the device.
Testing on each of the target devices can be a logistical challenge. Especially if a large variety of devices has been identified, securing devices with mobile data connections across the spectrum could be very costly. Depending on the nature and frequency of apps being created, outsourcing device testing could be a viable option. There are three major categories of outsourcing: outsourced testing, device outsourcing, and crowd sourcing.
Outsourced testing is, as it sounds, completely outsourcing the testing of a mobile application to a vendor specializing in mobile testing. The testing organization is responsible for test case creation and full execution of testing on mobile devices. Results of the testing are provided back to the development organization for correction.
Device outsourcing retains the QA activities within the development organization, however utilizes devices owned/maintained by the outsourcing vendor. Depending on the vendor, this could involve actually renting devices which are shipped to the development organization, utilizing devices within a lab setup provided by the outsourcing vendor, or remote access to the actual devices. In the later scenario, you have full control of the actual devices via remote control, and can view the screen through both a screen share, and an actual camera feed of the device.
Crowd sourcing represents an interesting option for testing. Vendors coordinate real-world users based on the types of devices and configurations they use for their personal mobile communications. Development organizations can specify the exact category of users to test, and the vendor will provide the application to users to test. This is generally not thorough testing – simple user interaction testing to provide user feedback and error logs from each type of device. Although this testing does not provide a full QA implementation, it is an effective method to get initial field results.
Although mobile testing represents significant additional challenges above traditional software testing, it is critically important to ensure thorough testing. Mobile users have little tolerance for defects, and will not continue to use an application that does not function well or intuitively on their device. The best chances for success in your mobile application involve creating a well planned and designed, fully tested application.
No comments:
Post a Comment