Study Design

Despite mobile apps widely rely on connections to back-end services/resources via WiFi or cell network, there is no previous work that analyzes the practices followed by Android developers. Therefore, we carried out an empirical study in which we manually analyzed 50 open source Android apps to build a taxonomy and an online catalogue of the most common issues/bad practices in Android apps that are related to eventual connectivity. That taxonomy was the foundation for answering the following question:

RQ1: What are the eventual connectivity issues observed in Android open-source apps?

Context Selection

To answer RQ1, we selected 50 popular open-source Android apps with features relying on Internet connection. The apps were manually selected from publicly available lists of open source Android apps: (i) the Wikipedia list of free/open source Android apps, and (ii) the GitHub repository of open source Android apps. The selected apps are distributed among different categories as presented in following image:

More information related to the process of app selection can be requested to the authors via email while the paper is under revision.

Data Collection

The testing and analysis process of the selected apps was organized in three-steps: (i) scenarios design, (ii) scenarios execution, and (iii) taxonomy building. The process was carried out by three authors with experience in mobile apps development (hereinafter referred as ``analysts”) and took about six months of work with partial dedication. Details about each of the three steps in the data collection are reported in the following.

Scenarios Design

Each application was assigned to one of the three analysts, who had as first task to explore the application and the features it provides. Then, the analyst had to fill-in a template composed by two parts: (i) App information, and (ii) Scenarios.

There are six different types of scenarios the analysts followed for defining the execution steps; the scenarios were identified by letters: a, b, c, d, e, f.

Scenarios Execution

Once the scenarios for an app were designed, an analyst executed each of them at least twice by running the app on a physical device. This re-execution process was performed to avoid issues related to flaky tests. All the executions were made under the same WiFi connection, and the offline cases were obtained by activating the airplane mode. During the execution, each analyst collected (i) the specific steps followed to execute a scenario (including screenshots of the app and a video of the execution), (ii) the results obtained when executing the scenario, and (iii) the issues exhibited in the app.

Taxonomy Building

After executing all the scenarios, we analyzed the assigned tags to build a taxonomy categorizing the issues exhibited in the apps. The taxonomy was defined by the three analysts through multiple open meetings.

Impacted Quality Attributes

We enhanced our results by analyzing the impact of the identified issues over different quality attributes. In order to do this, two authors used the list of quality attributes depicted in the ISO/IEC 25010 standard to evaluate separately the impact of each ECn category using a 4 values scale (\ie High, Medium, Low, None)