The dataset includes 20 mobile applications. These applications are from different categories named according to the Play Store categories, with a large variation of size and test code size. For instance, applications vary from 455 lines of code (MoonShot) to more than 347,000 lines of code (WordPress-Android). Similarly, test code size vary between 464 lines of code (MoonShot) and 3,732 (Mixin-Messenger). Moreover, the number of test cases vary from 4 (Ground) to 164 (AnkiDroid). The number of commits vary between 21 (Threema) to 68,148 (WordPress-Android).
| Name | Category | LOC | Test LOC | Test Cases | Resources | Commits |
|---|---|---|---|---|---|---|
| AnkiDroid | Education | 158,607 | 2,770 | 164 | Camera, Mobile Data, Wi-Fi | 13,643 |
| CovidNow | Medical | 2,269 | 540 | 21 | Mobile Data, Wi-Fi | 85 |
| Ground | Productivity | 19,906 | 525 | 4 | Camera, Location, Mobile Data, Wi-Fi | 4,936 |
| Iosched | Books, Reference | 27,824 | 473 | 9 | Location, Mobile Data, Wi-Fi | 3,101 |
| Lockwise | Productivity | 14,535 | 1,184 | 38 | Mobile Data, Wi-Fi | 503 |
| Mixin-Messenger | Finance | 168,080 | 3,732 | 160 | Bluetooth, Camera, Location, Mobile Data, Wi-Fi | 8,086 |
| Moonshot | Tools | 455 | 464 | 28 | Mobile Data, Wi-Fi | 351 |
| Nekome | Productivity | 1,084 | 2,097 | 64 | Mobile Data, Wi-Fi | 2,742 |
| Nl-covid19 | Medical | 65,839 | 1,114 | 20 | Bluetooth, Mobile Data, Wi-Fi | 1,293 |
| Openscale | Health, Fitness | 27,781 | 1,451 | 14 | Bluetooth, Location | 2,027 |
| OwnTracks | Travel, Local | 14,499 | 889 | 27 | Location, Mobile Data, Wi-Fi | 1,995 |
| PocketHub | Productivity | 29,001 | 1,663 | 107 | Mobile Data, Wi-Fi | 3,512 |
| RadioDroid | Music, Audio | 22,815 | 1,735 | 23 | Bluetooth, Mobile Data, Wi-Fi | 1,186 |
| Scarlet-Notes | Productivity | 4,260 | 770 | 52 | Mobile Data, Wi-Fi | 656 |
| Showly-2.0 | Entertainment | 2,547 | 952 | 55 | Mobile Data , Wi-Fi | 3,251 |
| SpaceXFollower | News, Magazines | 7,664 | 940 | 30 | Mobile Data, Wi-Fi | 356 |
| Threema | Communication | 238,045 | 1,931 | 54 | Bluetooth, Camera, Location, Mobile Data, Wi-Fi | 21 |
| Vocable | Communication | 13,188 | 499 | 14 | Camera | 863 |
| WooCommerce | Business | 156,962 | 1,367 | 27 | Mobile Data, Wi-Fi | 26,527 |
| WordPress-Android | Productivity | 347,897 | 3,674 | 115 | Camera, Mobile Data, Wi-Fi | 68,148 |
1 - Settings:
The settings were defined for five testing strategies (Random, One-Disabled, One-Enabled, Most-Enabled-Disabled, and Pairwise)
2 - Build scripts and the Manifest files:
The gradle build scripts and the Android Manifest files were adapted to our instrumentation, for instance, to include suitable API dependencies and to allow control permissions (such as for accessing Wi-Fi state). Each application has the test classes of androidTest folder extended with our instrumentation.
3 - Test suite instrumentation:
Our instrumentation was implemented for Java and Kotlin. There are particularities according to the screen device, since we use the UI Automator testing framework.
4 - Test execution:
Requirements: A device running Android 10. This device must be rooted in order to install Sensor Disabler app to manage the sensors. Also, the Lens Cap app must be installed to allow camera control. The host machine must have Python 3 installed and the Android SDK and the Java SDK environment system variables properly set.
The Python script runExpAll-MultiExec must be executed according to the proper gradle verification task, often connectedDebugAndroidTest, for installing and running Android instrumented tests on the connected device.
5 - Test analysis:
All test reports are available in TestResults.
There are deitaled instructions for replicability and a video tutorial illustrating a test suite execution for vocable.