Blog

iOS 7 Shakes Testing Tool Market

Jamo Solutions envisages the new iOS7 will shake the entire test automation tool market, making some tool vendors unable to deliver support for iOS7 because the technology used are incompatible to test iOS7 applications or to perform regression testing. Companies that have invested in an automation test bed using these tools should expect high maintenance for adapting their scripts to perform test automation on their iOS7 apps, moreover, delayed support or even no support will be given by their tool vendors for iOS7.

iOS7 New User Interface Design

With iOS7, Apple is releasing a new look and feel of the iOS system. The new interface is applied for all apps including your existing apps when upgrading to iOS7.

It is obvious that all apps will need to be re-tested. Since many years, companies have invested in test automation of their iOS apps and now they will be able to verify if their automation efforts can be used for regression testing against iOS7. In the test automation tools market, many tool providers are using bitmap-based techniques, pattern-based techniques, and OCR techniques to recognize text and areas on the iOS screen. Executing regression testing using scripts made by these tools will be costly and thus reducing the ROI of test automation significantly. Even if the tool provides ‘libraries’ of images, the QA engineer will have to update these libraries. For these tools, the effort of testing the app on iOS7 is no longer testing a new version on an existing mobile operating system, but rather, testing on a new mobile operating system!

Conversely, test automation scripts based on real object recognition by accessing the user interface on the device inside the app under test will not be affected by the new design of iOS7. These tools will continue to recognize all UI elements and the effort of regression testing will remain minimal.

Touch The Screen From The Testing Script, Still Possible?

Besides the changes in the new design interface, Apple has also changed the private API in iOS7.

What is private API?

Private API is a set of methods and classes that are implemented by Apple for iOS but these methods and classes are not documented by Apple. Documentation of these classes and methods is available on the internet by reverse engineering actions or by leaks of information from R&D centers. Apple does not guarantee that these private API classes and methods will be available in new releases.

This is in contrast with public API. Public API is a set of methods and classes that Apple documents and releases officially so that app developers can create applications for iOS. Apple has to guarantee upwards compatibility of public API.

Apps that use private APIs are not allowed to be sold through the Apple Store. For security reasons, Apple does not provide a public API to simulate “a touch”. If your testing tool is touch-based, for example, it recognizes a pattern and then touches an area on the screen, it means that the testing tool is using a private API for executing the touch.

Even UI object-based scripts are using the touch to replay the action from the test scripts.

All these tools where the replay is based on sending out a touch, will have problems in releasing their support for iOS7 since Apple changed this private API, an action to which they are entitled to do so.

As a QA engineer or a QA manager, you have to ask yourself if you want to continue using tools that are based on private API? How would you reach your company objectives if your automation test bed does not meet the quality requirements of the company’s apps running on iOS7?

Cloud-Based and Private Labs for iOS Devices in Danger?

For the tool vendors providing access to cloud-based devices or devices in your private central lab also implemented the touch simulation so that the QA Engineer can ‘touch’ the remote device by clicking the visualized screen on his PC. Some of these cloud-based or private lab software companies will have difficulties to enable this environment for iOS7 since they rely on software that uses the private API to simulate a touch.

M-eux Test

Jamo Solutions has a beta version of M-eux Test available to test apps on iOS7. Ever since the first release for iOS, the tool M-eux Test has been applying public available API for replaying the test actions. As the M-eux Test directly recognizes all UI elements of the app under test on the device, the existing iOS6 scripts can be re-used as such for testing on iOS7.

A video of our support is available here: http://goo.gl/mVhQKL

M-eux Test allows its iOS users to test their apps on iOS7 during the beta phases of iOS7 so that its customers’ apps are ready when iOS7 will be officially released by Apple.

More information on tools affected by the new iOS7 build and how M-eux Test can be applied to test iOS7 is available by mailing your inquiries to: [email protected].