research paper mobile devices

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 14 May 2021

Mobile Health: making the leap to research and clinics

• Joy P. Ku   ORCID: orcid.org/0000-0003-4785-6044 1 &
• Ida Sim   ORCID: orcid.org/0000-0002-1045-8459 2  

npj Digital Medicine volume  4 , Article number:  83 ( 2021 ) Cite this article

6060 Accesses

14 Citations

9 Altmetric

Metrics details

• Health care
• Research data
• Translational research

Health applications for mobile and wearable devices continue to experience tremendous growth both in the commercial and research sectors, but their impact on healthcare has yet to be fully realized. This commentary introduces three articles in a special issue that provides guidance on how to successfully address translational barriers to bringing mobile health technologies into clinical research and care. We also discuss how the cross-organizational sharing of data, software, and other digital resources can lower such barriers and accelerate progress across mobile health.

Introduction

Mobile devices have been a disruptive technology in many industries, but their impact on healthcare has yet to be fully realized. This is not due to a lack of interest. There are ~85,000 health apps 1 , 2 available for download, and over $8 billion was invested in “digital health” in 2018 3 . Novel miniaturized sensors are being developed to continuously detect biomarkers (e.g., from sweat 4 , tear fluid 5 ) that have traditionally been measured within a clinic. These developments are creating new possibilities for a vision of medicine that is more data-driven and personalized (e.g., ref. 6 ). In this article, we will refer to the use of such sensors and apps to collect personalized data for health in a ubiquitous manner as mobile health (mHealth).

As has been noted elsewhere 7 , 8 , data collection is only the first step in developing mHealth solutions that improve health outcomes. Clinicians and other stakeholders need to be convinced of the benefits of mHealth, and to-date it has been challenging to draw clear conclusions about the efficacy of these solutions, given the conflicting outcomes and heterogeneity in the implementation of mHealth interventions. This holds true whether assessing the impact of mHealth on hospital admission rates among patients with heart failure, adherence to prescribed rehabilitation exercises or lifestyle changes, or health outcomes like weight and blood pressure 9 , 10 . Myriad other factors, such as integration into the clinical or research study workflow, cost of implementation, usability of the device, and adequacy of privacy protections, also affect the likelihood of a solution transitioning from the prototype stage to routine use within research and clinics. Coming out of a multi-disciplinary workshop called mHealth Connect, three articles in this issue explicate some of these factors and provide guidance on how to successfully address translational barriers for different use cases. Specifically, the articles describe considerations when (1) selecting a suitable wearable sensor for a given application; (2) analyzing observational health behavior data generated by mHealth apps and devices; and (3) integrating these technologies into the clinical environment.

Their recommendations demonstrate the critical role data has in this new paradigm, so in addition to introducing the three articles, this paper calls for cross-organizational sharing of digital resources to accelerate progress within mHealth. Drawing on examples from other biomedical domains, we describe the positive impacts of sharing for three different types of resources and identify early efforts to encourage this behavior within mHealth. Thus, the insights offered through this and the other three articles in this issue can catalyze diverse activities to bring mHealth capabilities into clinical research and care.

mHealth Connect workshop

Despite the growing body of literature on consumer-oriented mHealth devices, there is a paucity of strong evidence for their benefit 9 . Few applications have made the leap from prototype to routine use for research or clinical purposes. mHealth Connect ( http://mobilize.stanford.edu/mhealthconnect/ ) was a workshop that brought together key stakeholder leaders across industry, clinical systems, and academia to collaboratively identify and overcome barriers to this translation. The workshop was launched in 2016 by two of the National Institutes of Health’s (NIH) Big Data to Knowledge (BD2K) Centers of Excellence–the Mobilize Center 11 and the Mobile-Sensor-to-Knowledge Center (MD2K) 12 —in response to concerns voiced by many BD2K researchers that many commercial mobile devices and apps on the market are poorly validated, without compelling clinical use cases, and are opaque and restrictive about data sharing. mHealth Connect enabled discussions around these and other critical issues to take place with a balance of stakeholders at the table and seeded collaborations to advance the field. The three mHealth papers in this issue arise from those discussions and the needs identified during them.

Scope of mHealth covered

While mHealth comprises a broad range of topics, as an outgrowth of two NIH Big Data to Knowledge Centers, mHealth Connect’s focus is on accelerating the use of data collected from mobile and wireless devices, such as wearable sensors, in clinical research and care. Because of the personal ubiquitous nature of mobile devices, the greatest new opportunity is in using mHealth to directly measure and improve patient health and health states outside the traditional confines of the hospital and clinic. The scope for this and the accompanying three papers thus excludes the following topics: (1) sensors and devices designed exclusively for the hospital or clinic setting and are intended solely to inform clinical decision-making (e.g., a Holter monitor, which would be excluded, versus AliveCor’s KardiaMobile device, which would be included), (2) strictly educational apps that are one-way channels for fixed media, (3) electronic health records (EHR) apps, and (4) apps for navigating the healthcare system (e.g., finding doctors, scheduling appointments) rather than for managing health or disease.

What these three papers do focus on are mobile apps and sensors used by patients in their daily lives to manage their health, with or without co-management by clinical team members or friends and family. These include devices measuring novel biomarkers, as well as consumer versions of traditional clinical equipment, such as blood pressure cuffs and spirometers, which an individual can use to collect measurements whenever and wherever they desire independent of clinical indications. The devices may be integrated into a clinical healthcare workflow, but they are not designed exclusively or primarily for that environment. The emphasis is on the availability of dynamic personalized data captured either passively or through active self-report, and the consequent value of this data for informing patient and clinician action to improve health and manage acute or chronic disease.

Guidelines for developing and deploying mHealth solutions

Recent years have seen a rise in resources providing guidelines to evaluate mHealth solutions, including from the U.S. Food and Drug Administration (FDA) 13 , 14 , 15 , 16 . Evaluation criteria assess a broad range of factors, including adherence to privacy laws, data security, interoperability with existing infrastructure and workflows, cost, usability, and validity of the content or intervention. Nascent efforts, such as Express Scripts’ planned digital health formulary, a list of approved digital health technologies to guide consumers and payers, are emerging to reinforce these guidelines 17 . While efforts to increase rigor in the evaluation of mHealth solutions are still taking shape, many questions remain on best practices and frameworks for mHealth development upstream of final regulatory or formulary approval.

The Clinical Trials Transformation Initiative (CTTI) provides one of the more comprehensive sets of guidelines for developing a mobile device-based solution, including the development of novel endpoints from mobile device data and the design of protocols that use mobile devices for data capture 18 . CTTI’s guidelines are intended for the relatively controlled conditions and limited durations of clinical trials, and therefore, necessarily exclude considerations for broad-scale clinical deployment. Nonetheless, they provide a useful path for individuals launching mHealth endeavors in general. Below we introduce a collection of articles based on our series of mHealth Connect meetings that augment existing guidelines provided by CTTI and others 18 , 19 , 20 .

Device selection for wellness, healthcare, and research applications

Regardless of the application, defining the target use case is critical for success. This definition is a fundamental tenet of many mHealth guidelines 18 , 19 , 20 , and it requires a process of user-centered design incorporating clinical, engineering, behavioral science, ethical, and disparities considerations (e.g., language, numeracy, literacy, and disabilities). All mHealth projects, even noncommercial ones, should have a clear business case detailing how continued use of the solution will be financially and logistically sustainable. The paper by Caulfield, et al. presents a framework for optimizing the match between sensors and classes of use cases, for refining the use case requirements, and then evaluating available devices against those requirements 21 .

Analysis of digital biomarkers for predictive models and unique insights

Digital biomarkers are clinically meaningful measures derived from mobile and wearable devices that correlate with or predict disease states. They can be analogues of traditional clinical quantities, such as heart rate, or novel indicators of health states. The full impact of mHealth comes from simulation or predictive models that combine digital biomarkers potentially with other data sources. An example is the cStress model, which blends real-time data streams on heart rate, heart rate variability, and interbeat interval data to derive a probability of stress in a given 1-min time window 22 . Developed using MD2K’s Cerebral Cortex, a cloud infrastructure for big data analysis of high-volume high-frequency data streams 12 , cStress utilized a prospective approach and actively recruited participants to collect data for its development.

Data analysis and model building can also be done retrospectively on observational datasets to gain insights that are challenging to obtain through traditional studies. In some cases, these datasets contain upwards of hundreds of thousands of individuals, enabling analysis about health and behavior on an unprecedented scale 23 , 24 . While such datasets can be a windfall, they present their own set of unique challenges for obtaining reliable results. The paper by Hicks, et al. presents a set of best practices for analyzing these large-scale, observational digital biomarker datasets from commercial personal technologies 25 .

Deploying mHealth solutions within clinical care

The necessity of a well-defined use case and business case becomes especially evident when it comes to the adoption and scaling of a mHealth solution. Is the mobile technology to be used by people with or without their clinicians? Is the intent to deploy locally in one care setting or to scale to global use? Particularly where clinician use is envisioned, integration into the clinical workflow is a prerequisite for adoption. To help guide expansion of mHealth technology into clinical care delivery, the paper by Smuck, et al. presents common factors driving successful utilization of wearables in the clinical care environment, as shown by two examples 26 .

Resource sharing to accelerate mHealth adoption

These papers aim to increase the likelihood of mHealth projects to achieve their aims, whether that is integrating mHealth technologies into the clinical workflow or developing a model to accurately predict health outcomes from mHealth data. The recommendations are intended to advance the work of individual groups, but they also point to opportunities for collective efforts that would advance activities across the entire community. In particular, we highlight the impact of sharing digital resources. Echoing Hicks, et al., we encourage “sharing models, software, datasets, and other digital resources whenever possible” 25 . Below we describe three categories of shared resources that can accelerate mHealth’s leap to research and the clinics: raw and processed data from devices; software and models used to analyze and interpret data; and evaluation results. We call attention to the positive impact the sharing of such resources has had in other biomedical domains and highlight initial efforts to bring these practices to mHealth.

The benefits of sharing experimental data, software, and models are well-delineated: enhanced transparency, the ability to more rapidly and easily extend existing efforts, and decreased duplication of efforts 27 , 28 . Large biomedical datasets that have been established specifically as research resources, such as the UK Biobank and the Osteoarthritis Initiative, have demonstrated the value of sharing, having supported hundreds of published research studies 29 , 30 . Smaller datasets from independent research labs can also positively impact a field. Previously shared data have served as benchmarks for comparing algorithms and aided in the validation of new models 31 , 32 , 33 . And pooling these datasets, such as in the Enhancing Neuro Imaging Genetics through Meta-Analysis (ENIGMA) project, has increased the statistical power of analyses and led to discoveries that would not have been possible from a single dataset alone 34 .

Such capabilities are critical for advancing mHealth. Having readily accessible datasets, both large and small, will facilitate the development of new digital biomarkers or robust mHealth-based predictive models, such as those described by Hicks, et al. We are starting to see some organized efforts to promote data sharing among the mHealth community. The International Children’s Accelerometry Database (ICAD) has created a pooled dataset, similar to ENIGMA 35 . Vivli, a platform for sharing data from clinical trials including trials with digital biomarkers, was launched in 2018 36 , and SimTK, a repository for the biocomputation and movement communities, recently added support for the sharing of mobile and other experimental data 37 .

Although invaluable, shared data alone will not propel mHealth applications into routine research and clinical use. Software methods and models are needed to glean insights from the data and thus, it is just as important that they also be shared, ideally with an open-source license to encourage modification and reuse for new applications. The programming language Python is a testament to the power of open-source with over 100,000 community-developed extensions 38 that make it a popular tool within bioinformatics and scientific computing. In biomechanics, researchers are sharing software for analyzing movement data within the open-source OpenSim simulation platform, extending the community’s ability to derive new insights 39 . While some mHealth software is being shared 40 , 41 , large, active communities have yet to develop around them. Initiatives, such as Open mHealth 42 as well as Shimmer and Nextbridge Exchange’s industry-based open-source effort to share analysis tools for wearable sensor data 43 , may help change the culture.

Similar initiatives would be useful throughout the mHealth development process, including the sharing of evaluation results, for example when evaluating devices during study design, as described by Caulfield, et al. 21 , and also when developing a reimbursement model to implement wearable technology into patient care, as mentioned by Smuck, et al. 26 . If individuals made their evaluation results available for others to leverage, we could appreciably streamline these processes. The CTTI Feasibility Studies Database 44 is a step towards this. The database compiles a list of devices, along with relevant evaluation criteria such as outcome measures and sample size, from publications examining the feasibility of mHealth in clinical trials. In a similar vein, the Digital Medicine Society provides a crowdsourced library of digital endpoints being used in industry-sponsored studies 45 . While there are some concerns about resource sharing—for example, potential misuse of shared resources and privacy breaches—technological and policy solutions can be implemented to mitigate them 46 , 47 . Compiling mHealth knowledge, data, and methods with such safeguards will accelerate the widespread adoption of mHealth for research and clinical care, and we urge individuals to contribute to such efforts.

It has been 13 years since the first iPhone was released, and 11 years since the first FitBit. In the intervening years, smartphone adoption has skyrocketed, fitness bands and smartwatches are commonplace, and “mobile health” NIH grants have grown from tens per year to over 610 in 2019. It has been said that digital health is now at “the end of the beginning” 48 . The mHealth Connect events have highlighted ways to go beyond the beginning: develop cross-disciplinary collaborations, pay attention to purpose, and consider factors beyond the technology itself. The papers in this series are intended as a guide for mHealth’s journey ahead and highlight ways in which we can collectively accelerate our progress along the path to clinical research and care.

Appfigures. Number of mHealth apps available in the Apple App Store from 1st quarter 2015 to 2nd quarter 2019 . https://www.statista.com/statistics/779910/health-apps-available-ios-worldwide/ (2019).

Appfigures. Number of mHealth apps available at Google Play from 1st quarter 2015 to 2nd quarter 2019 . https://www.statista.com/statistics/779919/health-apps-available-google-play-worldwide/ (2019).

Rock Health. 2018 Year End Funding Report: Is digital health in a bubble? https://rockhealth.com/reports/2018-year-end-funding-report-is-digital-health-in-a-bubble/ (2019).

Bandodkar, A. J., Jeang, W. J., Ghaffari, R. & Rogers, J. A. Wearable sensors for biochemical sweat analysis. Annu. Rev. Anal. Chem. 12 , 1–22 (2019).

Article   Google Scholar  

Tseng, R. C., Chen, C.-C., Hsu, S.-M. & Chuang, H.-S. Contact-lens biosensors. Sensors 18 , 2651 (2018).

Ginsburg, G. S. & Phillips, K. A. Precision medicine: from science to value. Health Aff. Proj. Hope 37 , 694–701 (2018).

Sim, I. Mobile devices and health. N. Engl. J. Med. 381 , 956–968 (2019).

Slovensky, D. J. & Malvey, D. M. Introduction to focused issue on mHealth infrastructure: issues and solutions that challenge optimal deployment of mHealth products and services. mHealth 3 , 52 (2017).

Noah, B. et al. Impact of remote patient monitoring on clinical outcomes: an updated meta-analysis of randomized controlled. Trials npj Digit. Med. 1 , 20172 (2018).

Lunde, P., Nilsson, B. B., Bergland, A., Kværner, K. J. & Bye, A. The effectiveness of smartphone apps for lifestyle improvement in noncommunicable diseases: systematic review and meta-analyses. J. Med. Internet Res. 20 , e162 (2018).

Ku, J. P. et al. The Mobilize Center: an NIH Big Data to Knowledge Center to advance human movement research and improve mobility. J. Am. Med. Inform. Assoc. 22 , 1120–1125 (2015).

Kumar, S. et al. Center of excellence for mobile sensor data-to-knowledge (MD2K). IEEE Pervasive Comput. 16 , 18–22 (2017).

Xcertia. Xcertia mHealth App Guidelines . https://www.xcertia.org/ (2019).

Agarwal, S. et al. Guidelines for Reporting of Health Interventions Using Mobile Phones: Mobile Health (mHealth) Evidence Reporting and Assessment (mERA) Checklist. BMJ 352 , i1174 (2016).

Anonymous. Report of the Working Group on mHealth Assessment Guidelines. Digital Single Market - European Commission https://ec.europa.eu/digital-single-market/en/news/report-working-group-mhealth-assessment-guidelines (2017).

U.S. Food & Drug Administration. Digital Health Innovation Action Plan . https://www.fda.gov/media/106331/download (2017).

Express Scripts. Express Scripts Simplifies Digital Health Technology Marketplace for Consumers and Payers . https://www.prnewswire.com/news-releases/express-scripts-simplifies-digital-health-technology-marketplace-for-consumers-and-payers-300851128.html .

Clinical Trials Transformation Initiative. Mobile Clinical Trials (MCT). https://www.ctti-clinicaltrials.org/programs/mobile-clinical-trials (2016).

Fillo, J. et al. Simplified Novel Application (SNApp) Framework: A Guide to Developing and Implementing Second-generation Mobile Applications for Behavioral Health Research. Transl. Behav. Med. 6 , 587–595 (2016).

Iii, C. C. W., Peeples, M. M. & Kouyaté, R. C. A. Evidence-based mHealth chronic disease mobile app intervention design: development of a framework. JMIR Res. Protoc. 5 , e25 (2016).

Caulfield, B., Reginatto, B. & Slevin, P. Not all sensors are created equal: a framework for evaluating human performance measurement technologies. npj Digit. Med. 2 , 7 (2019).

Hovsepian, K. et al. cStress: Towards a gold standard for continuous stress assessment in the mobile environment. Proc. ACM Int. Conf. Ubiquitous Comput. UbiComp Conf. 2015 , 493–504 (2015).

Google Scholar  

Althoff, T. et al. Large-scale physical activity data reveal worldwide activity inequality. Nature 547 , 336–339 (2017).

Article   CAS   Google Scholar  

Serrano, K. J., Yu, M., Coa, K. I., Collins, L. M. & Atienza, A. A. Mining health app data to find more and less successful weight loss subgroups. J. Med. Internet Res. 18 , e154 (2016).

Hicks, J. L. et al. Best practices for analyzing large-scale health data from wearables and smartphone apps. npj Digit. Med. 2 , 1–12 (2019).

Smuck, M., Odonkor, C. A., Wilt, J. K., Schmidt, N. & Swiernik, M. A. The emerging clinical role of wearables: factors for successful implementation in healthcare. npj Digit. Med . 4 , 45 (2021).

Lucraft, M., Baynes, G., Allin, K., Hrynaszkiewicz, I. & Khodiyar, V. Five essential factors for data sharing. figshare https://doi.org/10.6084/m9.figshare.7807949.v2 (2019).

Bull, S., Roberts, N. & Parker, M. Views of ethical best practices in sharing individual-level data From medical and public health research: a systematic scoping review. J. Empir. Res. Hum. Res. Ethics 10 , 225–238 (2015).

UK Biobank. Published papers | UK Biobank . https://www.ukbiobank.ac.uk/enable-your-research/publications (2019).

Osteoarthritis Initiative. Publications—Osteoarthritis Initiative . https://nda.nih.gov/oai/publications (2019).

Fregly, B. J. et al. Grand challenge competition to predict in vivo knee loads. J. Orthop. Res. 30 , 503–513 (2012).

Andronescu, M., Bereg, V., Hoos, H. H. & Condon, A. RNA STRAND: the RNA secondary structure and statistical analysis database. BMC Bioinformatics 9 , 340 (2008).

Seth, A., Matias, R., Veloso, A. P. & Delp, S. L. A biomechanical model of the scapulothoracic joint to accurately capture scapular kinematics during shoulder movements. PLoS ONE 11 , e0141028 (2016).

Thompson, P. M. et al. The ENIGMA Consortium: Large-scale collaborative analyses of neuroimaging and genetic data. Brain Imaging Behav. 8 , 153–182 (2014).

Sherar, L. B. et al. International Children’s Accelerometry Database (ICAD): Design and methods. BMC Public Health 11 , 485 (2011).

Center for Global Clinical Research Data. Vivli. https://vivli.org (2019).

SimTK Development Team. SimTK . https://simtk.org/ (2019).

Wikipedia. Python Package Index. https://en.wikipedia.org/wiki/Python_Package_Index (2019).

Seth, A. et al. OpenSim: Simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement. PLoS Comput. Biol. 14 , e1006223 (2018).

Hees, V. T. van et al. GGIR: Raw Accelerometer Data Analysis . https://cran.r-project.org/web/packages/GGIR/index.html (2019).

Onnela Lab. Beiwe Research Platform . https://www.hsph.harvard.edu/onnela-lab/beiwe-research-platform/ (2017).

Open mHealth. Open Source Data Integration Tools . https://www.openmhealth.org/ (2019).

Anonymous. Shimmer Announces Launch of Healthcare Industry Open Source Initiative for Wearable Sensor Algorithms . http://7thspace.com/headlines/968056/shimmer_announces_launch_of_healthcare_industry_open_source_initiative_for_wearable_sensor_algorithms.html (2019).

Clinical Trials Transformation Initiative (CTTI). CTTI Feasibility Studies Database . https://feasibility-studies.ctti-clinicaltrials.org/ (2019).

Digital Medicine Society. Library of Digital Endpoints—Digital Medicine Society (DiMe) . https://www.dimesociety.org/index.php/knowledge-center/library-of-digital-endpoints (2019).

Ursin, G., Sen, S., Mottu, J.-M. & Nygård, M. Protecting privacy in large datasets—first we assess the risk; then we fuzzy the data. Cancer Epidemiol. Prev. Biomark. 26 , 1219–1224 (2017).

Sardanelli, F. et al. To share or not to share? Expected pros and cons of data sharing in radiological research. Eur. Radiol. 28 , 2328–2335 (2018).

Rock Health. 2017 Year End Funding Report: The end of the beginning of digital health. Rock Health https://rockhealth.com/reports/2017-year-end-funding-report-the-end-of-the-beginning-of-digital-health/ (2018).

Download references

Acknowledgements

This work was supported by U54EB020404 and U54EB020405.

Author information

Authors and affiliations.

Department of Bioengineering, Stanford University, Stanford, CA, USA

Division of General Internal Medicine, University of California San Francisco, San Francisco, CA, USA

You can also search for this author in PubMed   Google Scholar

Contributions

J.K. led the drafting, and J.K. and I.S. contributed equally to the conception, writing, and review of this paper.

Corresponding author

Correspondence to Joy P. Ku .

Ethics declarations

Competing interests.

J.K. declares no competing interests. I.S. declares the following competing interests: financial support from Myovant, Audacious Inquiry, and Vivli, as well as non-financial support from Open mHealth, 98point6, Myia, and Google.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ku, J.P., Sim, I. Mobile Health: making the leap to research and clinics. npj Digit. Med. 4 , 83 (2021). https://doi.org/10.1038/s41746-021-00454-z

Download citation

Received : 09 January 2020

Accepted : 22 July 2020

Published : 14 May 2021

DOI : https://doi.org/10.1038/s41746-021-00454-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Methods for evaluating the efficacy and effectiveness of direct-to-consumer mobile health apps: a scoping review.

• Carley Prentice
• Kimberly Peven
• Sonia Ponzo

BMC Digital Health (2024)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: Translational Research newsletter — top stories in biotechnology, drug discovery and pharma.

Advertisement

Mobile learning: research context, methodologies and future works towards middle-aged adults – a systematic literature review

• Track 5: Multimedia and Education
• Published: 20 August 2022

Cite this article

• Syahida Mohtar   ORCID: orcid.org/0000-0002-4462-8890 1 ,
• Nazean Jomhari 1 ,
• Mumtaz Begum Mustafa 1 &
• Zulkifli Mohd Yusoff 2  

6495 Accesses

5 Citations

Explore all metrics

Over the past several years, mobile learning concepts have changed the way people perceived on mobile devices and technology in the learning environment. In earlier days, mobile devices were used mainly for communication purposes. Later, with many new advanced features of mobile devices, they have opened the opportunity for individuals to use them as mediated technology in learning. The traditional way of teaching and learning has shifted into a new learning dimension, where an individual can execute learning and teaching everywhere and anytime. Mobile learning has encouraged lifelong learning, in which everyone can have the opportunity to use mobile learning applications to gain knowledge. However, many of the previous studies on mobile learning have focused on the young and older adults, and less intention on middle-aged adults. In this research, it is targeted for the middle-aged adults which are described as those who are between the ages of 40 to 60. Middle-aged adults typically lead very active lives while at the same time are also very engaged in self-development programs aimed at enhancing their spiritual, emotional, and physical well-being. In this paper, we investigate the methodology used by researchers based on the research context namely, acceptance, adoption, effectiveness, impact, intention of use, readiness, and usability of mobile learning. The research context was coded to the identified methodologies found in the literature. This will help one to understand how mobile learning can be effectively implemented for middle-aged adults in future work. A systematic review was performed using EBSCO Discovery Service, Science Direct, Google Scholar, Scopus, IEEE and ACM databases to identify articles related to mobile learning adoption. A total of 65 journal articles were selected from the years 2016 to 2021 based on Kitchenham systematic review methodology. The result shows there is a need to strengthen research in the field of mobile learning with middle-aged adults.

Explore related subjects

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

Adulthood can be categorized into early, middle and late adulthood. Middle-aged adults come between the ages of 40 to 60, in other words is when one is in between the younger and older generations [ 42 , 62 ]. This stage of age, notably is the age period that Hall [ 31 ] referred to as aging, where the signs of cognitive and physical ageing start to be noticeable, from the age of 40 and rapidly increase after the age of 65 [ 6 , 54 ]. According to Yaffe and Stewart [ 94 ], a large part of adult life is made up of the mid-life period. This has been associated with many descriptive terms: mid-life syndrome, mid-life crisis, middlescence, empty nest syndrome, second adolescence, second honeymoon, age of fulfillment, and menopause. Aging population contributes to healthcare issues, not only amongst the older adults but towards middle-aged adults too. As mentioned earlier, the healthcare issues amongst the middle-aged adults are related to the decline in physical abilities, relational, and psychological capacities. For example, women in their middle age experience menopause and perceived personality change, which lead to severe depression, physical, and emotional problems [ 80 ]. According to Yaffe and Stewart [ 94 ], the most frequently identified events or concerns among middle-aged adults were: increased personal concern for health, death of a friend or relative, change in wage/salary, and concern for change in physical appearance.

When middle-aged adults enter their 60s, their reaction time starts to slow down further, and they experience a significant declination in their performance. The brain may also no longer function at its optimal level, leading to problems like memory loss, dementia, and may have issues with other cognitive functions such as language, attention, and visuospatial abilities [ 35 , 61 ]. It has been widely assumed that the midlife period is a critical period, thinking about death and mortality, as well as experiencing decline in physical abilities, relational, and psychological capacities [ 80 ]. Therefore, early prevention should therefore be looked upon at the middle age stage to help with memory impairment, as well as emotional control.

Middle-aged adults typically lead very active lives while also engaging in self-development programs aimed at enhancing their spiritual, emotional, and physical well-being. Muslim adult, for instance, will prefer to go to the mosque, surau, or Islamic center to seek for Islamic education [ 42 ] to enrich their knowledge and gain serenity through the command of Islam. This indicates that an individual in the middle-aged is inclined to reflect and improve the quality of one’s daily practice. Unfortunately, during the Covid-19 pandemic outbreak, many lectures at the mosques and other institutions could not be held, resulting in many people having to work from the home. As a result, many have taken the initiative to hold religious lectures online through video conferences such as via Zoom, WebEx, Jitsi Meet, Google Meet applications, and many more [ 1 ]. There are also those who watch religious lectures that have been prerecorded on certain channels, such as YouTube or podcasts. However, the enthusiasm and motivation for online and prerecorded learning is not the same and less encouraging as compared to face-to-face lectures.

Health management apps have shown to be useful for treating a variety of illnesses such as chronic illnesses caused by obesity, high blood pressure, diabetes, and so on [ 32 ]. As middle-aged adults are smartphone and tablet active users, they can use these portable devices to track their healthy lifestyle habits, maintain social communication, prevent accidents, and seek information [ 91 ]. In addition to chronic illness management using mobile applications, there is also a concern on how middle-aged adults can utilize mobile technology in fulfilling their spiritual journey towards a quality lifestyle. For example, they can learn how to acquire a literal understanding of the Quran through a spiritual mobile application. This will help a Muslim to elevate their understanding, motivation, and devotion towards Islam, which eventually leads them to become a better person emotionally and psychologically. All of these exhibit many important experiences associated with middle-age adults, most involving work and family, and self-development [ 53 ].

Mobile devices such as smartphones have gained popularity because they allow people to stay in touch and provide easy access to information anywhere and anytime [ 89 ]. Therefore, investigating the acceptance and adoption of mobile learning by the middle-aged adults through a systematic literature is important in highlighting the gap for any future work.

This review paper presents the fundamentals of mobile learning and the utilization of mobile technology in the learning environments. Mobile learning theories are also highlighted to show the significance of mobile learning towards middle-aged adults. Based on the research context found in the selected literature, the researchers here provide a systematic mapping of the employed methodologies in the area of mobile learning research. The purpose of the systematic mapping is to determine the most appropriate methodology for future research on middle-aged adults in areas of mobile learning.

2 Mobile learning

M-learning is a subset of ‘e-learning’ while ‘e-learning’ is the subset of distance learning that focuses on learning across context and learning with mobile devices, which can take place anytime, anywhere [ 43 , 62 ]. For example, learning may happen at the workplace, at home, and at places of leisure. The learning may be related to work demands, self-improvement, or leisure; and it is mobile with respect to time where it happens at different times during the day, on working days, or on weekends [ 68 ].

According to Ozdamli and Cavus [ 70 ], learners, teacher, environment, content, and assessment are the basic elements of mobile learning. The core characteristics of mobile learning are ubiquitous, portable size of mobile tools, blended, private, interactive, collaborative, and instant information. They enable learners to be in the right place at the right time, that is, to be where they can experience the authentic joy of learning.

Since learning can be performed anywhere and anytime using electronic devices, Traxler [ 85 ] defines that mobile learning is a learning process that is delivered through the support of mobile devices such as personal digital assistants, smartphones, wireless laptops, and tablets. This understanding is supported by Keegan [ 45 ] who suggested that m-learning should be restricted to learning on small and portable devices as mobile devices that could be carried everywhere.

According to Nordin, et al. [ 69 ], the requirements for mobile learning environment include technology, that is, (1) highly portable (to support learning whenever and wherever), (2) individual(the design should be able to support individual learning, cater for individual learning styles and be adaptable to learners’ abilities), (3) unobtrusive(where learners should be able to retrieve knowledge without the technology becoming a deterrent), (4) available(enabling communication with friends, experts and/or teachers), (5) adaptable(the context of learning should be adaptable to situations and the individual’s skills and knowledge development), (6) persistent( able to manage the learner’s learning despite the changes in the technology itself), (7) useful(useful to learners for everyday chores), and (8) user-friendly(easy for people to use and must not create technophobia among new users).

3 Mobile technology

Today, it is fortunate that mobile technology’s on-demand capability puts learning back into the learner’s hands by allowing users to take the initiative in diagnosing their learning needs, formulating learning goals, identifying human, and material resources for learning, choosing and implementing appropriate learning strategies, and evaluating those learning outcomes [ 50 ].

Mobile technology covers a wide range of mobile devices such as portable electronic devices used to perform a wide variety of communication, business, productivity, and lifestyle tasks such as parenting [ 26 , 66 ]. It is also connected through a cellular communication network or a wireless connection. The common mobile technologies that allow these tasks are cellular phones, PDAs, handheld computers, tablets, laptops, and wearable devices. A standard mobile technology device, such as a cellular phone, may have one or more features such as a GPS, a web browser, an instant messenger system, an audio recorder, an audio player, a video recorder, and gaming systems [ 4 ].

In the area of healthcare, numerous studies have been conducted on the use of mobile devices with wearable devices [ 21 , 39 , 87 ] to monitor the health of the elderly and individuals with disabilities. By using mobile apps, the health of the elderly and young adults can also be tracked [ 12 , 20 , 40 ] and diagnosed using mobile game-based screening tools [ 34 ], especially when facing challenges and stressful time during the Covid-19 pandemic [ 79 ] , or during post-college life transition [ 27 ]. Not all older people are proficient in using mobile devices. Therefore, there are researchers who make studies related to how older and young adults (university students) manages their mobile device security and privacy settings of their mobile devices in the context of social interaction and motivation [ 64 , 67 , 90 ].

Besides the usage of mobile devices in the healthcare area, the growth of mobile devices is significant and impactful in the education area such as in teaching history using 3D [ 57 ] and safety education [ 13 ], personal learning and workplace learning [ 29 ]. The use of mobile devices such as smartphones and tablets has become truly ubiquitous and has a potential for improving student learning, which can happen in collaborative, authentic settings, i.e., real life contexts and use active learning approaches [ 18 ]. As smartphones have become popular devices among youth nowadays [ 36 , 65 ], these devices can be utilized and embraced in the classroom teaching environment. By having a smartphone with wi-fi connectivity, Bluetooth, camera, color display, audio/video recording capability, it is already suitable for a person to adopt m-learning [ 36 ]. Majority of students spend most of their time (6 to 24 hours) on the Internet using their smartphones [ 8 ].

Smartphones also have become essential communication tools for older adults to stay connected with their family and peers [ 93 ]. Compared with younger adults, older adults tend to be more likely to use mobile phones for their original design purpose—that is, making calls for instrumental reasons such as arranging plans and other instrumental activities rather than playing games, surfing the internet, or using auxiliary applications [ 91 ]. The intervention of mobile technology in older adults’ lifestyles can improve their well-being and keep their mind and body active as well as prevent or slow down cognitive decline. For instance, mobile games can be used to capture cognitive learning outcomes and the process of knowledge acquisition [ 92 ]. Through activities such as interacting with easy games [ 71 ], taking and managing photographs, sending messages via SMS, video or audio calls, and reading newspapers via webpages may help cognitive and noncognitive stimulation of older adults.

Mobile computing devices become more situated, personal, collaborative, and lifelong and these innovations will become embedded, ubiquitous, and equipped with enhanced features for rich social interaction, contextual awareness, and access to the Internet. Hence, extending learning outside the classroom and into the learner's environment, mobile learning can have a significant impact on middle-aged adults. However, based on the research context in the areas of mobile learning, existing studies have concentrated exclusively on aspects of the mobile device use, such as accessibility, usability, and adoption, among young and older adults, while middle-aged adults have received less attention. Thus, the use of mobile devices among middle-aged adults should be further investigated to determine how mobile devices can assist them in acquiring knowledge and developing themselves while leading hectic lifestyles and having to deal with the Covid-19 pandemic, towards their long-life wellbeing.

4 Multimedia in Mobile learning application

Using mobile device as a learning tool is a new way for learners to learn as they like, anywhere and anytime. Moreover, an application that contains multimedia elements such as text, animation, graphic and video will engage and attract the attention of the student. Mobile learning application used in mobile learning environments varies, such as Learning Management System (LMS), Short Messaging services (SMS), Podcasting, Social Networking, Instant Messaging, Blogging, Facebook, Microblogging, Wiki, QR, 3D and Augmented Reality [ 81 ].

SMS and videos have long been used as language learning tool through the use of mobile phones and personal digital assistants (PDAs) [ 68 ], and today, many have benefits from using WhatsApp, flashcards and mind maps, on-line videos, and social networks in learning. Recently, Duolingo is said to be a popular application for new language learning where learners can interact with intelligent chatbots that give corrective feedback and awards at the same time [ 49 ].

In the fast-aging population countries like China, senior users have become a significant new growth point that cannot be ignored in social network sites to keep continuous competitiveness. In China, WeChat is the most popular social software for senior citizens. This is due to the good user experience and operability, where some senior users manage to operate the application although they have no computer skills or they know little about the network [ 11 ].

On the other hand, instant messaging apps such as WhatsApp and Line have become a popular mobile app amongst students. In a classroom environment, the student may use these apps to interact with teachers outside the class and using smartphones to manage their group assignments. The use of instant messaging applications promotes collaborative learning [ 7 ] and flexible learning, improves student participation, increase communication and interaction between lecturers and students, as well as improve the performance of teaching and learning [ 10 ].

Text editors such as the Mobile MS office, content management systems such as Learning Management Systems (LMSs), and audio-video recording of lectures did not get much attention by the students in terms of its usage via the smartphone. The reason for the low usage of these functions and features could be due to the limited screens space, which makes it difficult to read large documents, and the small sized keypad makes data entry cumbersome [ 36 ]. To make mobile learning more interesting, game-based elements have been used to improve the students’ engagement and enjoyment in learning. For instance, Kahoot is a game-based technological platform that can be accessed from, for instance, smart devices or a laptop. The game-based learning application (app) can benefit working adults who are adult learners with diverse learning abilities. Chunking method was used to break down complex concepts into smaller parts in the form of multiple-choice questions. The students’ learning process is tested and corrected, in real time, through the statistics which are generated from this chunking process. Kahoot creates a safe environment for students to make mistakes through multiple choice questions, and yet relearn it without being judged by their peers. However, the drawback of Kahoot is, it does not adequately support the learning experience of adult learners [ 74 ].

To achieve a successful ageing life, positive spirituality indeed has a close relationship with physical and mental abilities. There have been studies that develop an Empathic-Virtual Coach (VC) to involve senior users in enjoying a healthy lifestyle with respect to diet, physical activity, and social interactions, while in turn supporting their carers [ 41 ]. Furthermore, in addition to physical support, adults also require emotional and spiritual help for a balanced lifestyle. For example, Sevkli, et al. [ 75 ] in their study had designed and developed mobile Hadith Learning Systems (HLS) that were able to encourage and promote hadith learning for young and middle-aged Muslims. Hence, mobile apps appear to be one of the tools that can be used to promote a balanced well-being lifestyle for the older people such as their social status, independence in their everyday activities, health status, standard of living, or leisure activities of the aging population.

5 Mobile learning theory

According to Lee, et al. [ 56 ], there is an increasing number of adult learners entering or returning to university. Despite the growing number of nontraditional adult students in online higher education, little is known about the dynamic processes of adult distance learning, through which adult students struggle to develop their learning ability to balance their life and study, and to become self-regulated learners, and ultimately as competent selves and lifelong learners. The implementations of mobile learning are supported and guided by theories such as Behaviorism, Cognitivism, Constructivism, Situated Learning, Problem-Based Learning, Context Awareness Learning, Socio-Cultural Theory, Collaborative Learning, Conversational Learning, Lifelong Learning, Informal Learning as well as Activity Theory, Connectivism, Navigationism, Location-based learning [ 46 , 68 ]. The classification of activities around the main theories and areas of learning relevant to learning with mobile technologies are shown in Table 1 .

Lifelong learning happens not only in learning institutions such as community colleges or higher learning institutions, but can also happen anytime and anywhere according to the needs of the individual [ 69 ]. Informal and lifelong learning are often referred to adult education or continuing education, which means a learning process that occurs as blended learning with everyday life unobtrusively and seamlessly [ 73 ]. The unique characteristic of lifelong learning is the fact that it is centered on the learner. Because of that, the use of technology in offering a flexible learning framework is often favored by adult learners [ 69 ]. In addition, when compared to conventional methods such as textbooks, mobile learning tools, especially learning through mobile apps, are intrinsically inspiring, provide greater satisfaction, increase student well-being, and have positive implications for long-term student persistence [ 78 ].

Lifelong adult learners are different from young learners (school or university students) who may devote significant amounts of time to study each day, as their learning time is scattered due to family responsibilities, work obligations, and other social obligations [ 44 ]. However, the keys to unlocking the secrets to successful adult learning online are embedded in the basic principles that guide adult learners. The subsequent six principles upon which Knowles [ 51 ] constructed his formal and andragogical concept are shown in Table 2 .

6 Methodology

This study carried out an extensive literature review to identify the research gap, focusing on the related literature published within the period of 2016 to 2021. The aim of this systematic review is to investigate the trend of previous research on the acceptance and adoption of mobile learning by middle-aged adults. In order to justify the research gap based on the previous studies, this article will also provide views on the existing mobile learning usage targeted at solving user’s adoption of mobile learning towards young and older adults.

To conduct the systematic review, the researchers followed the procedure defined by Kitchenham [ 48 ], which is one of the most complete and suitable methods for reviewing studies in computer science. We carried out this review in three main phases: 1) planning of systematic mapping; 2) conducting the review; and 3) reporting the review. The phases of this systematic review and the related activities are shown in Fig. 1 .

Phases of conducting this systematic review

Planning of the Systematic Mapping

Activities involved in this stage were aimed to identify the objectives of the review. These activities are as follows:

Discovering the gap of the existing systematic reviews

In this step, a comprehensive search was performed in the cyberspace to locate the related review studies in mobile learning. Some of the bibliographic databases accessed included EBSCO Discovery Service, Science Direct, Google Scholar, Scopus, and IEEE.

Specifying the research questions

The research questions we have formulated for this review attempt to acquire the understanding and to determine the research gap on mobile learning usage in assisting lifelong learning in the context of spiritual among middle-aged adults. These questions are related to the acceptance and adoption of mobile learning towards middle-aged adults. The research questions are:

What are the fundamentals and background of mobile learning in the learning environment, including its adoption, acceptance, and available applications?

What are the research methodologies employed in the current studies carried out in mobile learning field?

What are the core research gaps should be further investigated by researchers in mobile learning towards middle-aged adults?

Identifying the relevant bibliographic databases

To answer the research questions and find relevant studies, bibliographic databases that cover majority of journals and conference papers associated with the field of human-computer interaction and mobile learning were selected. Related literatures published within the period of 2016 to 2021 were chosen in this research and the relevant bibliographic databases are ACM ( https://www.acm.org/ ), Emerald ( https://www.emerald.com/insight/ ), EBSCO Discovery Service ( http://search.ebscohost.com ), Science Direct ( http://sciencedirect.com ), Google Scholar ( http://scholar.google.com ), Scopus ( http://scopus.com ), and IEEE ( http://ieee.com ).

Conducting the Review

Activities involved in this stage were aimed to selecting related studies. These activities are as follows:

Identifying the Relevant Studies

In identifying the relevant studies, a search using key words such as “human-computer interaction”, “mobile learning”, “middle-aged adults”, “us- ability” was conducted. Accordingly, Boolean OR was used for alternative spellings, synonyms, or alternative terms, and Boolean AND was applied to connect the main terms. The complete list of search keywords of the review is provided in Table 3 .

Two additional search strategies were applied to retrieve the maximum number of relevant papers. The first strategy was reviewing the reference list of selected papers to find more related papers. The second strategy was googling the authors of selected studies to find potential related research.

Defining Selection Criteria

For selecting the primary papers, the following criteria based on the purpose of this study are defined.

Inclusion Criteria:

Studies containing mobile learning, acceptance, and adoption among mobile devices users.

Studies dealing with factors that contribute to the adoption and acceptance of mobile learning in the educational environments or working environments.

Studies utilizing mobile learning applications related to education, health care, data collection, and engineering that motivate users to use mobile learning.

Studied involving mobile learning users in category young adults, middle-aged adults, and older adults.

Exclusion Criteria:

Studies in learning environments that do not relate to the mobile learning context.

Studies of mobile learning that involve children such as kindergarten students or users with special needs.

Studies that are reluctant to serious mobile learning.

Papers that are only available in the form of abstracts or PowerPoint presentations.

Papers that are not written in English.

Selecting Primary Studies

The titles and abstracts of searched papers were reviewed based on the inclusion and exclusion criteria. Every paper that met at least one of the criteria and without any of the exclusion criteria was included in the review. For papers that could not be excluded based on reading of the titles and abstracts, the full texts of the papers were reviewed. Through this process, 65 articles were selected from the 531 papers initially found. 292 papers were excluded only by reading the topics, 105 papers by reading the abstracts, and 65 papers by reading the full text.

Validation control of the Primary Studies

In order to maintain the quality of the selected studies, the primary studies chosen by the first reviewer were double-checked by a second author. The evaluation of the selected paper was based on the evaluation questions as follows:

Whether a proposed mobile learning solution is implemented in the research context?

Whether the methodology of mobile learning solution is suitable for middle-aged adult?

To what extent the proposed solution effects the middle-aged adult in mobile learning?

The procedure of selecting the primary papers is illustrated in Fig.  2 .

Selecting the primary papers

Data Extraction and Synthesis

In order to extract and synthesize the data to answer the research questions, the selected studies are classified into five categories as follows:

Mobile learning and their research context: This categorization answer the first research question and helps to find the fundamentals and background in mobile research based on research context such as acceptance, adoption, effectiveness, impact, intention of use, usability, and readiness.

Methodology in the mobile learning research area : In order to answer the second research question and find the methodologies employed in the related context, the research context with the methods employed by the researchers was mapped as shown in Table 7 . Based on this mapping, the instruments that have been used in mobile learning research involving middle-aged adults can be identified.

Instruments used in Mobile learning research context: This category answers the second research question. From the systematic mapping done, it was found that the common research instruments used were Questionnaire, Interview, Experiments and Task Analysis. Here, the most preferable instruments used in mobile learning research were highlighted.

Mobile learning solutions in general: This category answers the third research question in order to find the gap in mobile learning research. Articles found in this study include mobile learning articles for young and older adults to show the trend of research towards adulthood. Since the focus of this systematic mapping is on identifying mobile learning technology applied to the middle-aged adults, thus those works focusing on the application of mobile learning not on adult learners or studies on users with special needs were excluded.

Solution for middle-aged adult in mobile learning: This category also answers the third research question in presenting the future works related to mobile learning involving middle-aged adults. This article begins by explaining the use of mobile technology in a learning environment, and the mobile learning theories that form the basis for the comparison of the existing mobile learning solutions for middle-aged adults.

Effects of mobile learning on middle-aged adult: This category answer the importance of the mobile learning towards middle-aged adults for a healthy well-being by assessing the number of studies related to middle-aged adults.

Reporting the Review

In the following section, the outcomes of reviewing the selected studies were reported and the results were discussed in detail, to respond to the defined research questions.

7 Results of the systematic mapping

From the search procedure and criteria, a total number of 65 articles are extracted. The distribution of the primary studies according to the publishing year is shown in Table 4 and Fig. 3 . The articles searched for this systematic review study are from 2016 to 2021. The reason is that this study aims to identify the latest research trends in the field of mobile learning with middle-aged adults. Finding shows that there are several studies from 2016 to 2018 that focus on this topic. The number of articles on mobile learning increased significantly from 2019 to 2020, which may be due to the outbreak of the Covid 19 pandemic. In education, for example, many institutions and organizations have drastically shifted from the traditional teaching and learning approach to online platforms. As a result, there is a considerable amount of research on mobile learning focusing on students in schools, universities, and academic staff. Meanwhile, a lot of study has been done in the field of healthcare with the elderly and middle-aged individuals, because their health begins to decline at this age.

Distribution of reviewed studies by year

It would also be interesting to find out the distribution of studies by countries, as shown in Table 5 . This shows that China had contributed the most research articles in this area of mobile learning. In article [ 13 , 21 , 72 , 77 , 81 , 88 ], the country where the study was conducted was not specified.

8 Participants

The categories of participants in the selected studies consist of young adults, middle-aged adults, and older adults. The number of studies based on age category is illustrated in Table 6 and Fig. 4 . It is found that the number of studies involving young adults is higher compared to studies involving older adults and middle-aged adults. This is due to the fact that young adults are frequent users of smartphones and are more adept at using mobile apps. Furthermore, since they are unable to attend college or universities due to the Covid-19 outbreak, many students are required to study online from home using mobile devices.

Number of studies based on participants’ age category

The details of the reference pertaining to the articles based on participants’ categories (older adults (OA), middle-aged adults (MA), and young adults (YA)) are listed in Table 8 .

9 Research context in Mobile learning

The articles obtained for this study were categorized by research area, as shown in Table 7 . Based on the results, mobile learning was studied in the following areas: Education, Healthcare, Usability, Transactional Services, and Social and Communication. Figure 5 illustrates the number of articles published on each research topic. The finding shows that many researchers prefer to conduct research in the field of education. This is because computers and mobile devices are widely used in educational institutions among young adults. On the other hand, studies that focus on middle-aged and older adults are usually concerned with language or vocabulary learning. The healthcare field is also receiving a lot of attention from researchers, and studies on mobile learning in this field are usually related to elderly and middle-aged people because older people and middle-aged people tend to be more vulnerable to health problems. The number of articles from other fields is low because studies on middle-aged adults and mobile learning did not match the scope and range of years defined for this study.

Number of articles in the research domain

Because the study related to mobile learning is very broad, therefore the article obtained has been classified into research context. Research context was determined based on the previous and current research in the field of mobile learning. It was found that many researchers in the field of mobile learning have studied the acceptance, adoption, effectiveness, impact, intention of use, readiness, and usability of mobile learning. The categorized articles are listed in Table 9 in section 11, with additional information on the methodology used in each study. Figure 6 shows the number of articles obtained by research context.

Number of papers by research context

10 Mobile learning towards the middle-aged adults

From these articles, not many researchers have examined the adoption of mobile learning by middle-aged adults. As mentioned earlier, a person in his or her forties is already inclined to focus on and enhance the standard of daily practice while also finding serenity. At this stage, many people have developed an inclination and willingness to gain more religious knowledge. Adult Muslims who work during the day, would rather choose to visit a mosque or surau to learn about Islam through religious lectures in the evening or at night. During the Covid-19 pandemic outbreak, many people were forced to work from home, and many lectures at the mosque were cancelled. As a result, many have taken the initiative to hold religious lectures via video conferences over the internet (e.g.: Zoom, WebEx). Others tend to watch religious lectures that have been posted on YouTube or other related platforms. However, as opposed to face-to-face seminars, the excitement and encouragement to attend online and prerecorded learning is lacking. Midlife brings with it a multitude of significant life experiences, the majority of which revolve around work, family, especially parenting, and self-development. Tablets are being used more commonly by middle-aged adults to monitor healthy lifestyle behaviors, maintain social contact, avoid injuries, and search information.

Many middle-aged and older adults are using the Internet to obtain information about health conditions and treatments, to get social support and advice from others with similar health-related experiences, and to access apps to help them manage their health [ 28 ]. For instance, Huang, et al. [ 32 ], studied on the attitude of middle-aged adults towards health app usage. From the study, they discovered that middle-aged adults who have no habits in health management tend to consider health applications as valuable tools and have a positive impact on them, while those who already have the habit, do not tend to consider health applications as valuable tool to be used in their daily routines. There are also some middle-aged adults who decide not to use health apps due to some sentimental reasons and the confidence of middle-aged adults in using a smartphone influences their cognitive assessment of health apps.

Table 8 shows the list of studies that are related to middle-aged adults. The age range of the middle-aged adults by each researcher varies. In this study, the age range of the adult is between 40 to 60 years old, which means the selected articles involve participants in this age range. A total of 22 articles were selected that involved middle-aged adults. In the field of language learning, two papers were identified. From these articles, it is found that the study of mobile learning with middle-aged adults is widely conducted in education area. The use of mobile apps in healthcare is also considered important, as this area is also the focus of researchers. The remaining articles are related to the study of user requirements, usability, and the design and development of mobile apps for middle-aged adults.

11 Research methodology

Research methodology is the main key to perform academic research and the strength of a research. The research methodology found used in the selected articles are Questionnaire, Interview, Systematic Literature Review, Literature Review, Reporting, Task Analysis and Experiment. Figure 7 shows the most popular research method used by a researcher in the field of mobile learning is questionnaire (n=24). This methodology has been used in studies that require a large amount of data from many respondents. The second most popular research method used in mobile learning research area is the Interview (n=9). There are also studies that require the use of multiple research methods to answer research questions.

Number of articles by research methodology

Table 9 shows the methodologies employed in the selected articles. However, articles [ 23 , 49 , 63 , 72 , 81 , 83 , 88 ], and [ 77 ] are not included because these articles are review articles.

In the literature, the questionnaire was found to be the most common method used by researchers for data collection involving many participants among young adults and middle-aged adults. On the other hand, the interview method only involved small groups of participants and was carried out in a short time period. Task analysis with interview method was used in three research studies to evaluate the usability, acceptance, and adoption. The studies were done towards young adults and older adults.

In the quantitative research method, the questionnaire instrument was used by the researchers to understand users’ motivation to use e-learning as a medium of learning [ 60 ]; the use of mobile technology and means of internet access [ 24 ]; awareness in using mobile devices towards mobile learning [ 14 ]; investigate the perception of students related to educational use of mobile phones [ 36 , 55 , 76 ]; investigate students’ behavioral intentions [ 3 ] and knowledge transfer among adult workers [ 52 ]; identify factors that affect the intention to use m-learning by learning the experience of the m-learning system by the participants [ 84 ], measure usability [ 19 , 75 ]; use and engagement with m-learning [ 2 ]; collaborative learning experience in social media environment [ 7 ], students’ immersion in the game and their perceived learning outcomes [ 33 ], and the use of mobile application [ 11 , 65 ].

Almost all researchers have formally collected demographic data such as gender, age, degree program, year of study, and race of the participants. There is only one study that collects data on working background because the participants in the study involved working adults. Amongst the selected articles , Al-Adwan, et al. [ 3 ] and Lazar, et al. [ 55 ], validated the content of the survey using experts before the questionnaire was distributed to participants. Dhanapal, et al. [ 19 ] and Huizenga, et al. [ 33 ] carried out a pilot test to identify the flaws and improves the questionnaire. All but one of the researchers used Point Likert scale, while MICAN [ 65 ] uses short answer questions, multiple choices with 1 or n answers, single or two-dimensional questions. The duration of data collection was less than 40 weeks depending on the targeted number of participants.

For the qualitative research method, data was collected via task analysis and interviews. Data were captured through multiple channels including video data analysis and interview content analysis. From the selected articles, it is found that task analysis and interview method were employed in the mobile learning domain to understand participants’ actions, performance, and usability towards mobile apps. The task activities that have been examined by researchers are navigation tasks (with task activity duration of 1.5 hours for older adults to complete searching and navigating using several mobile applications) [ 58 ], quiz activities using Kahoot application (held within 13 weeks for working adult and the task activities were perform in a classroom environment) [ 74 ], mobile devices usage training ( duration of 9 months of training intervention involving older people), and the task activities (e.g.: sending messages, video and audio calls ) was performed in a hospital [ 15 ]; Vocabulary learning [ 86 , 97 ]; games application with task duration of 5 to 20 minutes [ 71 ]; and usability testing [ 30 ]. Open-ended questions were used in the interview sessions [ 71 ] and all the audio recordings of the interviews were transcribed verbatim for analysis purposes [ 58 ].

In the experimental research design, two groups were created with specific condition applied. The treatment group and the control group involved in the experiment and questionnaire research approach can be seen in articles [ 9 , 16 , 38 , 92 ] as listed in Table 9 . For instance, in Bensalem [ 9 ], aims at investigating students' perceptions about the use of WhatsApp in learning vocabulary and in the study, twenty-one participants were randomly assigned to the experimental group. Participants from the experimental group are required to complete and submit their vocabulary assignments via WhatsApp. In the assignment, students are required to search the meaning of new words in a dictionary and build sentences using each word. On the other hand, participants from a control group need to submit the same homework assignment using the traditional paper and pencil method. Later, a questionnaire was distributed to the participants and the collected data was used to measure the participants’ perception of the use of WhatsApp in vocabulary learning.

12 Discussion

In this article, a systematic review was conducted to provide a thorough analysis on the methodologies adopted by researchers in mobile learning. The number of research papers in the year 2020 exceeds the number of research papers in the previous year. This could be due to the outbreak of the Covid-19 pandemic that triggered higher number of papers. During the pandemic, everyone had to work from home, and many organizations, including public and private higher learning institutions, were unable to carry out traditional teaching and learning activities. As a result, many studies or meetings were required to be conducted online.

The country with the highest number of research papers in the field of mobile learning is China with 11 articles. There is a lack of study in mobile learning that focuses on middle-aged adults. Out of 65 research papers, a total of 22 research papers are related to middle-aged adults whereby the distribution of research can be seen in countries such as in Czech Republic (n=1), United States (n=5), China (n=3), Germany (n=1). Singapore (n=2), Turkey (n=1), Brazil (n=1), Poland (n=1), Bangladesh (n=1), United Kingdom (n=2) and 2 articles did not mention the country in which the research was carried out. Studies related to middle-aged adults in Malaysia are not very encouraging, therefore the study of middle-aged adults in the field of mobile learning needs to be given more attention.

The articles selected in this systematic review were classified by research context to identify the focus of previous researchers on the use of mobile learning by middle-aged individuals. Overall, it was found that studies related to the adoption of mobile learning, mobile applications and mobile devices have gained significant attention among researchers, followed by studies related to the acceptance and mobile learning usage. However, studies on examining the adoption and effectiveness of mobile learning usage towards middle-aged adults are still lacking. Examining the effectiveness of mobile learning usage is crucial to provide guidance towards decision making and development work in the future.

The field of education is a popular field for researchers as it involves teachers and young adults who are mainly engaged in the learning environments. Research on middle-aged adults in the educational field is found in seven articles, where two of the articles focused on vocabulary learning. One study on Hadith learning for middle-aged adults, which has been classified as a study on spiritual learning under the educational research domain was also identified. The remaining four articles are respectively related to the use of game applications in teaching adults, the use of mobile devices in sharing information among adult workers, and the readiness of the teachers in adopting mobile learning in a classroom. Besides that, there is also a lack of research towards middle-aged adults in the area of mobile usability and user requirements. Research in the healthcare domain mostly involves older adults where most researchers extensively investigate the use of mobile devices and mobile applications towards healthy ageing and wellbeing.

The coding of the research methods was based on the methods reported by the researchers in their methodology section. Questionnaire is a popular instrument used across quantitative and mixed research approaches for data collection. The questionnaire developed by the researcher will be validated by the experts and tested before it was distributed accordingly to the targeted participants. Task analysis and interview approach can be used to observe the behavior of the users and to evaluate users’ feedback in the learning environment. Even though the method was not extensively used by the researchers from the selected literature focusing on middle-aged adults, this method to be employed in the mobile learning research to gain more insight on the effectiveness of mobile technologies in the learning environment of middle-aged adults was suggested.

Nowadays, almost everyone owns a smartphone, as smartphone prices have dropped significantly, making them affordable for more users. All smartphone users are capable to use most of the basic features of the mobile device, such as downloading applications from the Apple Store or Google Play. Given that middle-aged individuals are heavy smartphone users, it is critical to understand how users utilize mobile technology such as smartphones not just for work, leisure, and entertainment, but also for knowledge acquisition.

Middle-aged adults are self-directed, able to take responsibility for their learning, have a variety of experiences and backgrounds, and are motivated and willing to learn while effectively managing real-world situations. Hence, middle-aged adults can benefit from webinars and short courses delivered online. Therefore, more research should be conducted on mobile learning for middle-aged adults.

13 Conclusion and future work

The novelty of this study is that it contributes to the understanding of the research trends based on research context and methods used in research related to middle-aged adults in mobile learning. It is noted that there are still few studies that address the adoption and effectiveness of mobile apps in the area of religious orientation, especially among middle-aged adults. For instance, before the Covid-19 outbreak, middle-aged Muslims in Malaysia preferred to attend religious courses and trainings to improve their spiritual and religious orientation [ 96 ] based on face-to-face with teachers in a classroom. Therefore, it is critical to determine whether middle-aged adults intend and consent to religious and spiritual learning, such as learning the Quran to be conducted via mobile devices. It is hoped that the use of mobile learning will enable adults' lifelong learning to be improved and done continuously under any situation in the future. This study suggests further studies on middle-aged in the field of mobile learning as follows:

Skills and Knowledge Development

The use of mobile learning among middle-aged adults begins with an awareness and intention to use mobile devices. Generally, middle-aged adults who own smartphones, they already have skills to download apps from the Google Store or App Store and set security preferences. Hence, they must intend to use mobile learning to develop their skills and knowledge. This is because between the ages of 40 and 60, they are usually busy with their work while facing problems such as increasing concerns about health, death of a friend or relative, changes in wages/salaries, and concerns about changes in physical appearance. Therefore, middle-aged adults need to seek knowledge that will make them be satisfied and enable them to lead a better and healthier lifestyle. For example, middle-aged Muslims can learn to understand the Quran through mobile learning to achieve a better quality of life because the Quran is the final revelation and book from Allah s.w.t to humankind as guidance and direction to the right path.

Mobile Learning Application with Multimedia

Mobile learning Application with multimedia plays a great role in motivating learners in learning via digital devices such as smartphones. It is crucial to design and develop mobile learning apps with appropriate multimedia elements such as texts, images, icons, and animations that meet the needs of middle-aged adult learners. In addition, middle-aged adults need to be helped to increase their motivation to learn and improve their memory performance in vocabulary memorization. Therefore, for future work, mobile app development needs to be carefully developed based on user needs especially for the multimedia elements such as the text, graphic, video and animation.

Mobile Learning Application and Quick Assessment

Assessment is a critical component of learning since it demonstrates progress. Because most of the learning occurs online and involves many students, a teacher develops easy assessment tools and procedures that enable them to rapidly assess their students’ learning progress. Numerous game-based apps have aided in the facilitation of teaching and may be used to measure a student learning progress. Additionally, to make mobile learning more interesting, game-based elements have been used to improve the students’ engagement and enjoyment in learning. For instance, Kahoot is a game-based technological platform that can be accessed using, for instance, a smart device or a laptop. The game-based learning application (app) can benefit working adults who are adult learners with diverse learning abilities. Chunking method was used to break down complex concepts into smaller parts in the form of multiple-choice questions. The students’ learning process is tested and corrected, in real time, through the statistics which are generated from this chunking process. Kahoot creates a safe environment for students to make mistakes through multiple choice questions, and yet relearn it without being judged by their peers. However, the drawback of Kahoot is, it does not adequately support the learning experience of adult learners Seah [ 74 ]. Therefore, in the future, the development of mobile learning apps for middle-aged adults might include a gamification aspect that allows easy assessment for self-monitoring of learning progress.

Research Methodology

The finding of this study shows that questionnaire is a popular instrument used across quantitative and mixed research approaches for data collection. The questionnaire developed by the researcher will be validated by the experts and tested before it was distributed accordingly to the targeted participants. However, based on the research context and methodologies found in the literature, the study on middle-aged adults was not getting the enough intention among researchers. Furthermore, as Covid-19 pandemic has impacted people’s life, many are reluctant to participate in answering questionnaires as they may be unmotivated due to job loss, adaptation to new norms or due to the death of their family members. Therefore, in the future, it is hereby recommended that a contribution back to society such as given some tokens to the participants [ 66 , 90 ] can be practiced in the research methodology. Besides that, a researcher also can conduct a free intensive course of related field to a group of respondents to upgrade the lifestyle and well-being among respondents. Hence, this can increase public participation in research, especially when involving busy and elderly respondents and at the same time the respondents can learn new knowledge while also contributing to the research study.

Adedoyin OB, Soykan E (2020) Covid-19 pandemic and online learning: the challenges and opportunities. Interact Learn Environ:1–13

Al Masarweh M (2019) Evaluating M-Learning System Adoption by Faculty Members in Saudi Arabia Using Concern Based Adoption Model (CBAM) Stages of Concern. Int J Emerg Technol Learn 14(5)

Al-Adwan AS, Al-Madadha A, Zvirzdinaite Z (2018) Modeling students’ readiness to adopt mobile learning in higher education: An empirical study. Int Rev Res Open Dist Learn 19(1)

Alkhezzi F, Ahmed MS (2020) A Review of Mobile Learning Technology Integration: Models, Frameworks, and Theories. Coll Stud J 54(4):491–504, Winter2020

Google Scholar  

Al-Sabbagh KW, Bradley L, Bartram L (2019) Mobile language learning applications for Arabic speaking migrants–a usability perspective. Lang Learn Higher Educ 9(1):71–95

Article   Google Scholar  

Amarya S, Singh K, Sabharwal M (2018) Ageing process and physiological changes. In: Gerontology. IntechOpen

Ansari JAN, Khan NA (2020) Exploring the role of social media in collaborative learning the new domain of learning. Smart Learn Environ 7(1):1–16

Anshari M, Almunawar MN, Shahrill M, Wicaksono DK, Huda M (2017) Smartphones usage in the classrooms: learning aid or interference? Educ Inf Technol 22(6):3063–3079

Bensalem E (2018) The Impact of WhatsApp on EFL students' Vocabulary Learning. Arab World Eng J 9(1):23–03

Bere A, Rambe P (2019) Understanding Mobile learning using a social embeddedness approach: a case of instant messaging. Int J Educ Dev Using Inf Commun Technol 15(2):132–153

Cao Y, Qin X, Li J, Long Q, Hu B (2020) Exploring seniors’ continuance intention to use mobile social network sites in China: a cognitive-affective-conative model. Univ Access Inf Soc 21:1–22

Cheng C-H, Chen Y-S, Sangaiah AK, Su Y-H (2018) Evidence-based personal applications of medical computing models in risk factors of cardiovascular disease for the middle-aged and elderly. Pers Ubiquit Comput 22(5):921–936

Chittaro L (2016) Designing serious games for safety education:“learn to brace” Versus traditional pictorials for aircraft passengers. IEEE Trans Vis Comput Graph 22(5):1527–1539

Article   MathSciNet   Google Scholar  

Christensen R, Knezek G (2017) Readiness for integrating mobile learning in the classroom: challenges, preferences and possibilities. Comput Hum Behav 76:112–121. https://doi.org/10.1016/j.chb.2017.07.014

Cid A, Sotelo R, Leguisamo M, Ramírez-Michelena M (2020) Tablets for deeply disadvantaged older adults: challenges in long-term care facilities. Int J Human-Comput Stud 144:102504

Daungcharone K, Panjaburee P, Thongkoo K (2019) A mobile game-based C programming language learning: results of university students' achievement and motivations. Int J Mobile Learn Org 13(2):171–192

de Lara SMA, de Mattos Fortes RP, Russo CM, Freire AP (2016) A study on the acceptance of website interaction aids by older adults. Univ Access Inf Soc 15(3):445–460

Devshikha Bose KP (2020) Lana Grover, "a Mobile learning Community in a Living Learning Community : perceived impact on digital fluency and communication," the online journal of new horizons in education

Dhanapal S, Salman NW, Raman Kutty GK, Ansari KMS, Shah SA (2019) 'XYZ' Application as a Tool for Teaching and Learning in Institutions of Higher Learning: An Exploratory Study. Aplikasi 'XYZ' Sebagai Alat untuk Pengajaran dan Pembelajaran di Institusi Pengajian Tinggi: Satu Kajian Tinjauan (55):1–01

Martino FD, Delmastro F, Dolciotti C (2021) Malnutrition risk assessment in frail older adults using m-health and machine learning. ICC 2021 - IEEE International Conference on Communications, pp 1–6

Ejaz A, Rahim M, Khoja SA (2019) "The Effect of Cognitive Load on Gesture Acceptability of Older Adults in Mobile Application," in 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON): IEEE, pp. 0979–0986.

Farivar S, Abouzahra M, Ghasemaghaei M (2020) Wearable device adoption among older adults: a mixed-methods study. Int J Inf Manag 55:102209

Galić S, Lušić Z, Stanivuk T (2020) E-LEARNING IN MARITIME AFFAIRS. J Nav Archit Mar Eng 17(1):39, 01//

Gan CL, Balakrishnan V (2018) Mobile Technology in the Classroom: what drives student-lecturer interactions? Int Human-Comput Interact 34(7):666–679. https://doi.org/10.1080/10447318.2017.1380970

García Botero G, Questier F, Zhu C (2019) Self-directed language learning in a mobile-assisted, out-of-class context: do students walk the talk? Comput Assist Lang Learn 32(1–2):71–97

Garg R (2021) "Its changes so often": Parental non-/use of mobile devices while caring for infants and toddlers at home. Proc. ACM Hum.-Comput. Interact., 5(CSCW2), Article 369. https://doi.org/10.1145/3479513

Gong C, Saha K, Chancellor S (2021) "The Smartest Decision for My Future": Social Media Reveals Challenges and Stress During Post-College Life Transition. Proc ACM Human-Comput Interact 5(CSCW2):1–29

Gordon NP, Crouch E (2019) Digital information technology use and patient preferences for internet-based health education modalities: cross-sectional survey study of middle-aged and older adults with chronic health conditions. JMIR aging 2(1):e12243

Grant MM (2019) Difficulties in defining mobile learning: analysis, design characteristics, and implications. Educ Technol Res Dev 67(2):361–388

Guinibert M (2020) Learn from your environment: A visual literacy learning model. Australas J Educ Technol 36(4):173–188. https://doi.org/10.14742/ajet.5200

Hall GS (1922) Senescence, the last half of life. Appleton, New York

Book   Google Scholar  

Huang Y-M, Lou S-J, Huang T-C, Jeng Y-L (2019) Middle-aged adults’ attitudes toward health app usage: a comparison with the cognitive-affective-conative model. Univ Access Inf Soc 18(4):927–938

Huizenga J, Admiraal W, Ten Dam G, Voogt J (2019) Mobile game-based learning in secondary education: students’ immersion, game activities, team performance and learning outcomes. Comput Hum Behav 99:137–143

Intarasirisawat J, Ang CS, Efstratiou C, Dickens L, Sriburapar N, Sharma D, Asawathaweeboon B (2020) An automated Mobile game-based screening tool for patients with alcohol dependence. Proc ACM Interact, Mobile, Wearab Ubiquitous Technol 4(3):1–23

Introduction to psychology: lifespan development. https://courses.lumenlearning.com/wmopen-psychology/chapter/reading-adulthood/ . Accessed 16 Feb 2021

Iqbal S (2017) Mobile phone usage and students’ perception towards m-learning: A case of undergraduate students in Pakistan. J Distance Educ/Revue de l'ducation Dist 32(1)

Islam MN, Ahmed MA, Islam AN (2020) Chakuri-bazaar: a mobile application for illiterate and semi-literate people for searching employment. Int Mobile Human Comput Interact (IJMHCI) 12(2):22–39

Jeno LM, Dettweiler U, Grytnes J-A (2020) The effects of a goal-framing and need-supportive app on undergraduates' intentions, effort, and achievement in mobile science learning. Comput Educ 159, 12/01/December 2020. https://doi.org/10.1016/j.compedu.2020.104022

Jones SL, Hue W, Kelly RM, Barnett R, Henderson V, Sengupta R (2021) Determinants of longitudinal adherence in smartphone-based self-tracking for chronic health conditions: evidence from axial Spondyloarthritis. Proc ACM Interact, Mobile, Wearab Ubiquitous Technol 5(1):1–24

Jones J, Yuan Y, Yarosh S (2021) Be consistent, work the program, be present every day: exploring Technologies for Self-Tracking in early recovery. Proc ACM Interact, Mobile, Wearab Ubiquitous Technol 5(4):1–26

Justo R et al (2020) Analysis of the interaction between elderly people and a simulated virtual coach. J Ambient Intell Humaniz Comput:1–16

Kadir MA, Arifin S, Latipun, Fuad AN (2016) Adult Learners' understanding in learning Islam using andragogy approach: a study in Kampung Siglap mosque and Al-Zuhri higher learning institute. J Educ Pract 7(32):1–10

Kadirire J, Guy R (2009) Mobile learning demystified. Evol Mobile Teach learn:15–56

Kang H, Lin X (2019) Lifelong learning on the go: English language Mobile learning in China. New Direct Adult Contin Educ 2019(162):49–60

Keegan D (2005)The incorporation of mobile learning into mainstream education and training. In: World Conference on Mobile Learning, Cape Town, vol 11, pp 1–17

Keskin NO, Metcalf D (2011) The current perspectives, theories and practices of mobile learning. Turkish Online J Educ Technol-TOJET 10(2):202–208

Kiconco RI, Rooks G, Snijders C (2020) Learning mobile money in social networks: comparing a rural and urban region in Uganda. Comput Hum Behav 103:214–225

Kitchenham B (2004) Procedures for performing systematic reviews. Keele, UK, Keele University 33(2004):1–26

Klimova B (2020) Benefits of the use of Mobile applications for learning a foreign language by elderly population. Procedia Comput Sci 176:2184–2191

Knowles MS (1975) Self-directed learning: A guide for learners and teachers.

Lippitt GL, Knowles MS, Knowles MS (1984) Andragogy in action: applying modern principles of adult learning

Kuciapski M (2017) A model of mobile technologies acceptance for knowledge transfer by employees. J Knowl Manag 21(5):1053–1076. https://doi.org/10.1108/JKM-03-2016-0136

Lachman ME, Lewkowicz C, Marcus A, Peng Y (1994) Images of midlife development among young, middle-aged, and older adults. J Adult Dev 1(4):201–211

Lachman ME, Teshale S, Agrigoroaei S (2015) Midlife as a pivotal period in the life course: balancing growth and decline at the crossroads of youth and old age. Int J Behav Dev 39(1):20–31

Lazar IM, Panisoara G, Panisoara IO (2020) Digital technology adoption scale in the blended learning context in higher education: Development, validation and testing of a specific tool. PLoS One 15(7):e0235957. https://doi.org/10.1371/journal.pone.0235957

Lee K, Choi H, Cho YH (2019) Becoming a competent self: a developmental process of adult distance learning. Internet High Educ 41:25–33

Lewis R, Taylor-Poleskey M (2021) Hidden town in 3D: teaching and reinterpreting slavery virtually at a living history museum. J Comput Cultural Heritage (JOCCH) 14(2):1–14

Li Q, Luximon Y (2020) Older adults’ use of mobile device: usability challenges while navigating various interfaces. Behav Inform Technol 39(8):837–861

Lin CJ, Ho S-H (2020) The development of a mobile user interface ability evaluation system for the elderly. Appl Ergon 89:103215

Lin X, Su S (2020) Chinese college Students' attitude and intention of adopting Mobile learning. Int J Educ Dev Using Inf Commun Technol 16(2):6–21

Manca M et al (2021) The impact of serious games with humanoid robots on mild cognitive impairment older adults. Int J Hum Comput Stud 145. https://doi.org/10.1016/j.ijhcs.2020.102509

Masrom M, Nadzari AS, Mahmood NHN, Zakaria WNW, Ali NRM (2016) Mobile learning in Malaysia education institutions. Issues in Information Systems 17(4):152-157

Mather CA, Gale F, Cummings EA (2017) Governing mobile technology use for continuing professional development in the Australian nursing profession. BMC Nurs 16(1):1–11

Mendel T, Toch E (2019) My mom was getting this popup: understanding motivations and processes in helping older relatives with mobile security and privacy. Proc ACM Interact, Mobile, Wearab Ubiquitous Technol 3(4):1–20

Mican D (2019) Examining Adoption of Mobile Devices and Applications among College Students in Romania. J App Comput Sci Math 13(28)

Michelson R, DeWitt A, Nagar R, Hiniker A, Yip J, Munson SA, Kientz JA (2021) Parenting in a pandemic: juggling multiple roles and managing technology use in family life during COVID-19 in the United States. In: Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), pp 1–39

Moore M, DeBruhl B (2019) Investigating university student desires and use of smartphone privacy settings. J Comput Sci Colleges 34(4):134–141

Naismith L, Lonsdale P, Vavoula GN, Sharples M (2004) Literature review in mobile technologies and learning. Future Lab Report, 11

Nordin N, Embi MA, Yunus MM (2010) Mobile learning framework for lifelong learning. Procedia Soc Behav Sci 7:130–138

Ozdamli F, Cavus N (2011) Basic elements and characteristics of mobile learning. Procedia Soc Behav Sci 28:937–942

Palacio RR, Acosta CO, Cortez J, Morán AL (2017) Usability perception of different video game devices in elderly users. Univ Access Inf Soc 16(1):103–113

Palalas A, Wark N (2020) The relationship between mobile learning and self-regulated learning: A systematic review. Australas J Educ Technol 36(4):151–172. https://doi.org/10.14742/ajet.5650

Sarrab M, Baghdadi Y, Al-Shihi H, Bourdoucen H (2016) A model for mobile learning non-functional requirement elicitation. Int J Mobile Learn Org 10(3):129–158

Seah D (2020) Using Kahoot in law school: differentiated instruction for working adults with diverse learning abilities. Int J Mobile Learn Org 14(1):36–48

Sevkli AZ, Motiwalla L, Abdulkarem HF (2017) The design and implementation of a context-aware mobile hadith learning system. Int J Mobile Learn Org 11(4):295–313

Sharma SK, Sarrab M, Al-Shihi H (2017) Development and validation of mobile learning acceptance measure. Interact Learn Environ 25(7):847–858

Shuib L, Ghani NA, Elaish MM, Yadegaridehkordi E (2019) Mobile English language learning (MELL): a literature review. Educ Rev 71(2):257–276. https://doi.org/10.1080/00131911.2017.1382445

Shukla S (2020) M-learning adoption of management students’: a case of India. Educ Inf Technol 26:1–32

Sin F, Berger S, Kim I-J, Yoon D (2021) Digital social interaction in older adults during the COVID-19 pandemic. Proc ACM Human-Comput Interact 5(CSCW2):1–20

Stewart AJ, Ostrove JM, Helson R (2001) Middle aging in women: patterns of personality change from the 30s to the 50s. J Adult Dev 8(1):23–37

Sundgren M (2017) Blurring time and place in higher education with bring your own device applications: a literature review. Educ Inf Technol 22(6):3081–3119

Swanson JA (2020) Assessing the Effectiveness of the Use of Mobile Technology in a Collegiate Course: A Case Study in M-learning. Technol Knowl Learn 25(2):389, 06//

Tan E, Law R (2016) mLearning as a softer visitor management approach for sustainable tourism. J Sustain Tour 24(1):132, 01//

Thongsri N, Shen L, Bao Y, Alharbi IM (2018) "Integrating UTAUT and UGT to explain behavioural intention to use M-learning: a developing country’s perspective," Journal of Systems and Information Technology

Traxler J (2009) Current state of mobile learning. Mobile Learning: Trans del Educ Train 1:9–24

Tu Y, Zou D, Zhang R (2020) A comprehensive framework for designing and evaluating vocabulary learning apps from multiple perspectives. Int J Mobile Learn Org 14(3):370–397

Vacher M, Aman F, Rossato S, Portet F, Lecouteux B (2019) Making emergency calls more accessible to older adults through a hands-free speech interface in the house. ACM Transact Acces Comput (TACCESS) 12(2):1–25

Venkataraman JB, Ramasamy S (2018) Factors influencing mobile learning: a literature review of selected journal papers. Int J Mob Learn Org 12:99–112

Vicente P, Lopes I (2016) Attitudes of older mobile phone users towards mobile phones. Communications 41(1):71–86

Wan Z, Bao L, Gao D, Toch E, Xia X, Mendel T, Lo D (2019) Appmod: helping older adults manage mobile security with online social help. Proc ACM Interact, Mobile, Wearab Ubiquitous Technol 3(4):1–22

Wang Y, Matz-Costa C, Miller J, Carr DC, Kohlbacher F (2018) Uses and gratifications sought from mobile phones and loneliness among japanese midlife and older Adults: a Mediation Analysis. Innov Aging 2(3)

Wardaszko M, Podgórski B (2017) Mobile learning game effectiveness in cognitive learning by adults: a comparative study. Simul Gaming 48(4):435–454

Wong CY, Ibrahim R, Hamid TA, Mansor EI (2018) Usability and design issues of smartphone user interface and mobile apps for older adults. In: International Conference on User Science and Engineering. Springer, pp 93–104

Chapter   Google Scholar  

Yaffe MJ, Stewart MA (1984) The problems and concerns of middle age. Can Fam Physician 30:1089–1093

Yang H-L, Lin S-L (2019) The reasons why elderly mobile users adopt ubiquitous mobile social service. Comput Hum Behav 93:62–75

Zainal A, Ahmad NA, Razak FHA, Nordin A (2017) Older adults’ requirements of Islamic Mobile applications: a multimethod exploration. Adv Sci Lett 23(5):4236–4240. https://doi.org/10.1166/asl.2017.8335

Zhang R, Zou D (2020) Influential factors of working adults' perceptions of mobile-assisted vocabulary learning with multimedia annotations. Int J Mobile Learn Org 14(4):533–548

Download references

Acknowledgments

We would like to thank you to Universiti Teknikal Malaysia Melaka (UTeM) for sponsoring the corresponding author in this research numbered UTeM.02.13.04/500-4/12/16/1/2(91)

Conflict of interest

The authors declare that there is no conflict of interest regarding the publication of this paper.

Availability of data and materials

The datasets analyzed during the current study are available from the corresponding author on reasonable request.

The corresponding author is sponsored by Universiti Teknikal Malaysia Melaka (UTeM) numbered UTeM.02.13.04/500-4/12/16/1/2(91) to pursue her PhD degree in these studies.

Author information

Authors and affiliations.

Department of Software Engineering, Faculty of Computer Science and Information Technology, Universiti Malaya, 50603, Kuala Lumpur, Malaysia

Syahida Mohtar, Nazean Jomhari & Mumtaz Begum Mustafa

Department of Al-Quran and Al-Hadith, Academy of Islamic Studies, Universiti Malaya, 50603, Kuala Lumpur, Malaysia

Zulkifli Mohd Yusoff

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Syahida Mohtar .

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Mohtar, S., Jomhari, N., Mustafa, M.B. et al. Mobile learning: research context, methodologies and future works towards middle-aged adults – a systematic literature review. Multimed Tools Appl (2022). https://doi.org/10.1007/s11042-022-13698-y

Download citation

Received : 02 August 2021

Revised : 17 June 2022

Accepted : 15 August 2022

Published : 20 August 2022

DOI : https://doi.org/10.1007/s11042-022-13698-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mobile application
• Self-directed learning
• Middle-aged adults
• Methodology
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 07 May 2013

Usability of mobile applications: literature review and rationale for a new usability model

• Rachel Harrison 1 ,
• Derek Flood 1 &
• David Duce 1  

Journal of Interaction Science volume  1 , Article number:  1 ( 2013 ) Cite this article

192k Accesses

347 Citations

3 Altmetric

Metrics details

The usefulness of mobile devices has increased greatly in recent years allowing users to perform more tasks in a mobile context. This increase in usefulness has come at the expense of the usability of these devices in some contexts. We conducted a small review of mobile usability models and found that usability is usually measured in terms of three attributes; effectiveness, efficiency and satisfaction. Other attributes, such as cognitive load, tend to be overlooked in the usability models that are most prominent despite their likely impact on the success or failure of an application. To remedy this we introduces the PACMAD (People At the Centre of Mobile Application Development) usability model which was designed to address the limitations of existing usability models when applied to mobile devices. PACMAD brings together significant attributes from different usability models in order to create a more comprehensive model. None of the attributes that it includes are new, but the existing prominent usability models ignore one or more of them. This could lead to an incomplete usability evaluation. We performed a literature search to compile a collection of studies that evaluate mobile applications and then evaluated the studies using our model.

Introduction

Advances in mobile technology have enabled a wide range of applications to be developed that can be used by people on the move. Developers sometimes overlook the fact that users will want to interact with such devices while on the move. Small screen sizes, limited connectivity, high power consumption rates and limited input modalities are just some of the issues that arise when designing for small, portable devices. One of the biggest issues is the context in which they are used. As these devices are designed to enable users to use them while mobile, the impact that the use of these devices has on the mobility of the user is a critical factor to the success or failure of the application.

Current research has demonstrated that cognitive overload can be an important aspect of usability [ 1 , 2 ]. It seems likely that mobile devices may be particularly sensitive to the effects of cognitive overload, due to their likely deployment in multiple task settings and limitations of size. This aspect of usability is often overlooked in existing usability models, which are outlined in the next section, as these models are designed for applications which are seldom used in a mobile context. Our PACMAD usability model for mobile applications, which we then introduce, incorporates cognitive load as this attribute directly impacts and may be impacted by the usability of an application.

A literature review, outlined in the following section, was conducted as validation of the PACMAD model. This literature review examined which attributes of usability, as defined in the PACMAD usability model, were used during the evaluation of mobile applications presented in a range of papers published between 2008 and 2010. Previous work by Kjeldskov & Graham [ 3 ] has looked at the research methods used in mobile HCI, but did not examine the particular attributes of usability incorporated in the PACMAD model. We also present the results of the literature review.

The impact of this work on future usability studies and what lessons other researchers should consider when performing usability evaluations on mobile applications are also discussed.

Background and literature review

Existing models of usability.

Nielsen [ 4 ] identified five attributes of usability:

  Efficiency : Resources expended in relation to the accuracy and completeness with which users achieve goals;

  Satisfaction : Freedom from discomfort, and positive attitudes towards the use of the product.

  Learnability : The system should be easy to learn so that the user can rapidly start getting work done with the system;

  Memorability : The system should be easy to remember so that the casual user is able to return to the system after some period of not having used it without having to learn everything all over again;

  Errors : The system should have a low error rate, so that users make few errors during the use of the system and that if they do make errors they can easily recover from them. Further, catastrophic errors must not occur.

In addition to this Nielsen defines Utility as the ability of a system to meet the needs of the user. He does not consider this to be part of usability but a separate attribute of a system. If a product fails to provide utility then it does not offer the features and functions required; the usability of the product becomes superfluous as it will not allow the user to achieve their goals. Likewise, the International Organization for Standardization (ISO) defined usability as the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [ 5 ]. This definition identifies 3 factors that should be considered when evaluating usability.

  User : Person who interacts with the product;

  Goal : Intended outcome;

  Context of use : Users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a product is used.

Each of the above factors may have an impact on the overall design of the product and in particular will affect how the user will interact with the system. In order to measure how usable a system is, the ISO standard outlines three measurable attributes:

  Effectiveness : Accuracy and completeness with which users achieve specified goals;

Unlike Nielsen’s model of usability, the ISO standard does not consider Learnability, Memorability and Errors to be attributes of a product’s usability although it could be argued that they are included implicitly within the definitions of Effectiveness, Efficiency and Satisfaction. For example, error rates can be argued to have a direct effect on efficiency.

Limitations for mobile applications

The models presented above were largely derived from traditional desktop applications. For example, Nielsen’s work was largely based on the design of telecoms systems, rather than computer software. The advent of mobile devices has presented new usability challenges that are difficult to model using traditional models of usability. Zhang and Adipat [ 6 ] highlighted a number of issues that have been introduced by the advent of mobile devices:

  Mobile Context : When using mobile applications the user is not tied to a single location. They may also be interacting with nearby people, objects and environmental elements which may distract their attention.

  Connectivity : Connectivity is often slow and unreliable on mobile devices. This will impact the performance of mobile applications that utilize these features.

  Small Screen Size : In order to provide portability mobile devices contain very limited screen size and so the amount of information that can be displayed is limited.

  Different Display Resolution : The resolution of mobile devices is reduced from that of desktop computers resulting in lower quality images.

  Limited Processing Capability and Power : In order to provide portability, mobile devices often contain less processing capability and power. This will limit the type of applications that are suitable for mobile devices.

  Data Entry Methods : The input methods available for mobile devices are different from those for desktop computers and require a certain level of proficiency. This problem increases the likelihood of erroneous input and decreases the rate of data entry.

From our review it is apparent that many existing models for usability do not consider mobility and its consequences, such as additional cognitive load. This complicates the job of the usability practitioner, who must consequently define their task model to explicitly include mobility. One might argue that the lack of reference to a particular context could be a strength of a usability model provided that the usability practitioner has the initiative and knows how to modify the model for a particular context.

The PACMAD usability model aims to address some of the shortcomings of existing usability models when applied to mobile applications. This model builds on existing theories of usability but is tailored specifically for applications that can be used on mobile devices. The PACMAD usability model is depicted in Figure  1 side by side with Nielsen’s and the ISO’s definition of usability. The PACMAD usability model incorporates the attributes of both the ISO standard and Nielsen’s model and also introduces the attribute of cognitive load which is of particular importance to mobile applications. The following section introduces the PACMAD usability model and describes in detail each of the attributes of usability mentioned below as well as the three usability factors that are part of this model: user, task and context.

Comparison of usability models.

The PACMAD usability model for mobile applications identifies three factors (User, Task and Context of use) that should be considered when designing mobile applications that are usable. Each of these factors will impact the final design of the interface for the mobile application. In addition to this the model also identifies seven attributes that can be used to define metrics to measure the usability of an application. The following section outlines each of these factors and attributes in more detail.

Factors of usability

The PACMAD usability model identifies three factors which can affect the overall usability of a mobile application: User , Task and Context of use . Existing usability models such as those proposed by the ISO [ 5 ] and Nielsen [ 4 ] also recognise these factors as being critical to the successful usability of an application. For mobile applications Context of use plays a critical role as an application may be used in multiple, very different contexts.

User It is important to consider the end user of an application during the development process. As mobile applications are usually designed to be small, the traditional input methods, such as a keyboard and mouse, are no longer practical. It is therefore necessary for application designers to look at alternative input methods. Some users may find it difficult to use some of these methods due to physical limitations. For example it has been shown [ 7 ] that some Tetraplegic users who have limited mobility in their upper extremities tend to have high error rates when using touch screens and this may cause unacceptable difficulties with certain (usually small) size targets.

Another factor that should be considered is the user’s previous experience. If a user is an expert at the chosen task then they are likely to favour shortcut keys to accomplish this task. On the other hand novice users may prefer an interface that is intuitive and easy to navigate and which allows them to discover what they need. This trade-off must be considered during the design of the application.

Task The word task refers here to the goal the user is trying to accomplish with the mobile application. During the development of applications, additional features can be added to an application in order to allow the user to accomplish more with the software. This extra functionality comes at the expense of usability as these additional features increase the complexity of the software and therefore the user’s original goal can become difficult to accomplish.

For example, consider a digital camera. If a user wants to take a photograph, they must first select between different modes (e.g. video, stills, action, playback, etc.) and then begin to line up the shot. This problem is further compounded if the user needs to take a photograph at night and needs to search through a number of menu items to locate and turn on a flashlight.

Context of use The word context refers here to the environment in which the user will use the application. We want to be able to view context separately from both the user and the task. Context not only refers to a physical location but also includes other features such as the user’s interaction with other people or objects (e.g. a motor vehicle) and other tasks the user may be trying to accomplish. Research has shown that using mobile applications while walking can slow down the walker’s average walking speed [ 8 ]. As mobile applications can be used while performing other tasks it is important to consider the impact of using the mobile application in the appropriate context.

Attributes of usability

The PACMAD usability model identifies 7 attributes which reflect the usability of an application: Effectiveness , Efficiency , Satisfaction , Learnability , Memorability , Errors and Cognitive load . Each of these attributes has an impact on the overall usability of the application and as such can be used to help assess the usability of the application.

Effectiveness Effectiveness is the ability of a user to complete a task in a specified context. Typically effectiveness is measured by evaluating whether or not participants can complete a set of specified tasks.

Efficiency Efficiency is the ability of the user to complete their task with speed and accuracy. This attribute reflects the productivity of a user while using the application. Efficiency can be measured in a number of ways, such as the time to complete a given task, or the number of keystrokes required to complete a given task.

Satisfaction Satisfaction is the perceived level of comfort and pleasantness afforded to the user through the use of the software. This is reflected in the attitudes of the user towards the software. This is usually measured subjectively and varies between individual users. Questionnaires and other qualitative techniques are typically used to measure a user’s attitudes towards a software application.

Learnability A recent survey of mobile application users [ 9 ] found that users will spend on average 5 minutes or less learning to use a mobile application. There are a large number of applications available on mobile platforms and so if users are unable to use an application they may simply select a different one. For this reason the PACMAD model includes the attribute Learnability as suggested by Nielsen.

Learnability is the ease with which a user can gain proficiency with an application. It typically reflects how long it takes a person to be able to use the application effectively. In order to measure Learnability, researchers may look at the performance of participants during a series of tasks, and measure how long it takes these participants to reach a pre-specified level of proficiency.

Memorability The survey also found that mobile applications are used on an infrequent basis and that participants used almost 50% of the applications only once a month [ 9 ]. Thus there may be a large period of inactivity between uses and so participants may not easily recall how to use the application. Consequently the PACMAD usability model includes the attribute of Memorability as also suggested by Nielsen.

Memorability is the ability of a user to retain how to use an application effectively. Software might not be used on a regular basis and sometimes may only be used sporadically. It is therefore necessary for users to remember how to use the software without the need to relearn it after a period of inactivity. Memorability can be measured by asking participants to perform a series of tasks after having become proficient with the use of the software and then asking them to perform similar tasks after a period of inactivity. A comparison can then be made between the two sets of results to determine how memorable the application was.

Errors The PACMAD usability model extends the description of Errors, first proposed by Nielsen, to include an evaluation of the errors that are made by participants while using mobile apps. This allows developers to identify the most troublesome areas for users and to improve these areas in subsequent iterations of development. This attribute is used to reflect how well the user can complete the desired tasks without errors. Nielsen [ 4 ] states that users should make few errors during the use of a system and that if they do make errors they should be able to easily recover from them. The error rate of users may be used to infer the simplicity of a system. The PACMAD usability model considers the nature of errors as well as the frequency with which they occur. By understanding the nature of these errors it is possible to prevent these errors from occurring in future versions of the application.

Cognitive load The main contribution of the PACMAD model is its inclusion of Cognitive Load as an attribute of usability. Unlike traditional desktop applications, users of mobile applications may be performing additional tasks, such as walking, while using the mobile device. For this reason it is important to consider the impact that using the mobile device will have on the performance of the user of these additional tasks. For example a user may wish to send a text message while walking. In this case the user’s walking speed will be reduced as they are concentrating on sending the message which is distracting them from walking.

Cognitive load refers to the amount of cognitive processing required by the user to use the application. In traditional usability studies a common assumption is that the user is performing only a single task and can therefore concentrate completely on that task. In a mobile context users will often be performing a second action in addition to using the mobile application [ 8 , 10 ]. For example a user may be using a stereo while simultaneously driving a car. In this scenario it is important that the cognitive load required by the mobile application, in this case the stereo, does not adversely impact the primary task.

While the user is using the application in a mobile context it will impact both the user’s ability to move and to operate the mobile application. Therefore it is important to consider both dimensions when studying the usability of mobile applications. One way this can be measured is through the NASA Task Load Index (TLX) [ 11 ]. This is a subjective workload assessment tool for measuring the cognitive workload placed on a user by the use of a system. In this paper we adopt a relatively simple view of cognitive load. For a more accurate assessment it may be preferable to adopt a more powerful multi-factorial approach [ 1 , 12 ] but this is beyond the scope of this paper.

Literature review

In order to evaluate the appropriateness and timeliness of the PACMAD usability model for mobile applications, a literature review was conducted to review current approaches and to determine the need for a comprehensive model that includes cognitive load. We focused on papers published between 2008 and 2010 which included an evaluation of the usability of a mobile application.

Performing the literature review

The first step in the literature review was to collect all of the publications from the identified sources. These sources were identified by searching the ACM digital library, IEEE digital library and Google Scholar. The search strings used during these searches were “ Mobile Application Evaluations ”, “ Usability of mobile applications ” and “ Mobile application usability evaluations ”. The following conferences and journals were identified as being the most relevant sources: the Mobile HCI conference (MobileHCI), the International Journal of Mobile Human Computer Interaction (IJMHCI), the ACM Transactions on Computer-Human Interaction (TOCHI), the International Journal of Human Computer Studies (IJHCS), the Personal and Ubiquitous Computing journal (PUC), and the International Journal of Human-Computer Interaction (IJHCI). We also considered the ACM Conference on Human Factors in Computing Systems (CHI) and the IEEE Transactions on Mobile Computing (IEEE TOMC). These sources were later discarded as very few papers (less than 5% of the total) were relevant.

The literature review was limited to the publications between the years 2008 and 2010 due to the emergence of smart phones during this time. Table  1 shows the number of publications that were examined from each source.

The sources presented above included a number of different types of publications (Full papers, short papers, doctoral consortium, editorials, etc.). We focused the study only on full or short research papers from peer reviewed sources. This approach was also adopted by Budgen et al. [ 13 ]. Table  2 shows the number of remaining publications by source.

The abstract of each of the remaining papers was examined to determine if the paper:

Conducted an evaluation of a mobile application/device;

Contained some software component with which the users interact;

Conducted an evaluation which was focused on the interaction with the application or device;

Publications which did not meet the above criteria were removed.

The following exclusion criteria were u sed to exclude papers:

Focused only on application development methodologies and techniques;

Contained only physical interaction without a software component;

Examined only social aspects of using mobile applications;

Did not consider mobile applications.

Each abstract was reviewed by the first two authors to determine if it should be included within the literature review. When a disagreement arose between the reviewers it was discussed until mutual agreement was reached. A small number of relevant publications were unavailable to the authors. Table  3 shows the number of papers included within the literature review by source.

Each of the remaining papers was examined by one reviewer (either the first or second author of this paper). The reviewer examined each paper in detail and identified for each one:

 The attribute of usability that could be measured through the collected metrics;

 The focus of the research presented.

 The type of study conducted;

To ensure the quality of the data extraction performed the first and second author independently reviewed a 10% sample and compared these results. When a disagreement arose it was discussed until an agreement was reached.

Twenty papers that were identified as being relevant did not contain any formal evaluations of the proposed technologies. The results presented below exclude these 20 papers. In addition to this some papers presented multiple studies. In these cases each study was considered independently and so the results based on the number of studies within the evaluated papers rather than the number of papers.

Limitations

This literature review is limited for a number of reasons. Firstly a small number of papers were unavailable to the researchers (8 out of 139 papers considered relevant). This unavailability of less than 6% of the papers probably does not have a large impact on the results presented. By omitting certain sources from the study a bias may have been introduced. We felt that the range of sources considered was a fair representation of the field of usability of mobile applications although some outlying studies may have been omitted due to limited resources. Our reviews of these sources led us to believe that the omitted papers were of borderline significance. Ethical approval for this research was given by Oxford Brookes University Research Ethics Committee.

Research questions

To evaluate the PACMAD usability model three Research Questions (RQ1 to RQ3) were established to determine how important each of the factors and attributes of usability are in the context of mobile applications.

RQ1: What attributes are used when considering the usability of mobile applications?

This research question was established to discover what attributes are typically used to analyse mobile applications and which metrics are associated with them. The answers to this question provide evidence and data for the PACMAD usability model.

RQ2: To what extent are the factors of usability considered in existing research?

In order to determine how research in mobile applications is evolving, RQ2 was established to examine the current research trends into mobile applications, with a particular focus on the factors that affect usability.

In addition to this we wanted to establish which research methods are most commonly used when evaluating mobile applications. For this reason, a third research question was established.

RQ3: What research methodologies are used to evaluate the usability of mobile applications?

There are many ways in which mobile applications can be evaluated including controlled studies, field studies, ethnography, experiments, case-studies, surveys, etc. This research question aims to identify the most common research methodologies used to evaluate mobile apps. The answers to this question will throw light on the maturity of the mobile app engineering field.

The above research questions were answered by examining the literature on mobile applications. The range of literature on the topic of mobile applications is so broad it was important to limit the literature review to the most relevant and recent publications and to limit the publication interval to papers published between 2008 and 2010.

Table  4 shows the percentage of studies that include metrics, such as time to complete a given task, which either directly or indirectly assesses the attributes of usability included within the PACMAD usability model. In some cases the studies evaluated multiple attributes of usability and therefore the results above present both the percentage and the number of studies in which each attribute was considered. These studies often do not explicitly cite usability or any usability related criteria, and so the metrics used for the papers’ analyses were used to discover the usability attributes considered. This lack of precision is probably due to a lack of agreement as to what constitutes usability and the fact that the attributes are not orthogonal. The three most common attributes, Effectiveness, Efficiency and Satisfaction, correspond to the attributes identified by the ISO’s standard for usability.

One of the reasons these attributes are so widely considered is their direct relationship to the technical capabilities of the system. Both Effectiveness and Efficiency are related to the design and implementation of the system and so are usually tested thoroughly. These attributes are also relatively easy to measure. In most cases the Effectiveness of the system is evaluated by monitoring whether a user can accomplish a pre-specified task. Efficiency can be measured by finding the time taken by the participant to complete this task. Questionnaires and structured interviews can be used to determine the Satisfaction of users towards the system. Approximately 22% of the papers reviewed evaluated all three of these attributes.

The focus on these attributes of usability implies that Learnability, Memorability, Errors, and Cognitive load, are considered to be of less importance than Effectiveness, Efficiency and Satisfaction. Learnability, Memorability, Errors, and Cognitive load are not easy to evaluate and this may be why their assessment is often overlooked. As technology matures designers have begun to consider usability earlier in the design process. This is reflected to a certain extent by technological changes away from command line towards GUI based interfaces.

The aspects of usability that were considered least often in the papers reviewed are Learnability and Memorability. There are numerous reasons for this. The nature of these attributes demands that they are evaluated over periods of time. To effectively measure Learnability, users’ progress needs to be checked at regular intervals or tracked over many completions of a task. In the papers reviewed, Learnability was usually measured indirectly by the changes in effectiveness or efficiency over many completions of a specified task.

Memorability was only measured subjectively in the papers reviewed. One way to objectively measure Memorability is to examine participants’ use of the system after a period of inactivity with the system. The practical problem of recruiting participants who are willing to return multiple times to participate in an evaluation is probably one of the reasons why this attribute is not often measured objectively.

What differentiates mobile applications from more traditional applications is the ability of the user to use the application while moving. In this context, the users’ attention is divided between the act of moving and using the application. About 26% of the studies considered cognitive load. Some of these studies used the change in performance of the user performing the primary task (which was usually walking or driving) as an indication of the cognitive load. Other studies used the NASA TLX [ 11 ] to subjectively measure cognitive load.

Table  5 shows the current research trends within mobile application research. It can be seen that the majority of work is focused on a task approximately 47% of the papers reviewed focus on allowing users to complete a specific task. The range of tasks considered is too broad to provide a detailed description and so we present here only some of the most dominant trends seen within the literature review.

The integration of cameras into mobile devices has enabled the emergence of a new class of application for mobile devices known as augmented reality. For example Bruns and Bimber [ 14 ] have developed an augmented reality application which allows users to take a photograph of an exhibit at an art gallery which allows the system to find additional information about the work of art. Similar systems have also been developed for Points of Interest (POIs) for tourists [ 15 ].

While using maps is a traditional way of navigating to a destination, mobile devices incorporating GPS (Global Positioning Satellite) technology have enabled researchers to investigate new ways of helping users to navigate. A number of systems [ 16 , 17 ] have proposed the use of tactile feedback to help guide users. Through the use of different vibration techniques the system informs users whether they should turn left, right or keep going straight. Another alternative to this is the use of sound. By altering the spatial balance and volume of a user’s music, Jones et al. [ 18 ] have developed a system for helping guide users to their destination.

One of the biggest limitations to mobile devices is the limited input modalities. Developers of apps do not have a large amount of space for physical buttons and therefore researchers are investigating other methods of interaction. This type of research accounts for approximately 29% of the studies reviewed.

The small screen size found on mobile applications has meant that only a small fraction of a document can be seen in detail. When mobile devices are used navigating between locations, this restriction can cause difficulty for users. In an effort to address this issue Burigat et al. [ 19 ] have developed a Zoomable User Interface with Overview (ZUIO). This interface allows a user to zoom into small sections of a document, such as a map, while displaying a small scale overview of the entire document so that the user can see where on the overall document they are. This type of system can also be used with large documents, such as web pages and images.

Audio interfaces [ 20 ] are a type of interface that is being investigated to assist drivers to use in-car systems. Traditional interfaces present information to users by visual means, but for drivers this distraction has safety critical implications. To address this issue audio inputs are common for in-vehicle systems. The low quality of voice recognition technology can limit its effectiveness within this context. Weinberg et al. [ 21 ] have shown that multiple push-to-talk buttons can improve the performance of users of such systems. Other types of interaction paradigms in these papers include touch screens [ 22 ], pressure based input [ 23 ], spatial awareness [ 24 ] and gestures [ 25 ]. As well as using these new input modalities a number of researchers are also looking at alternative output modes such as sound [ 26 ] and tactile feedback [ 27 ].

In addition to considering the specific tasks and input modalities, a small number of researchers are investigating ways to assist specific types of users, such as those suffering from physical or psychological disabilities, to complete common tasks. This type of research accounts for approximately 9% of the evaluated papers. Approximately 8% of the papers evaluated have focused on the context in which mobile applications are being used. The remaining 6% of studies are concerned with new development and evaluation methodologies for mobile applications. These include rapid prototyping tools for in-car systems, the effectiveness of expert evaluations and the use of heuristics for evaluating mobile haptic interfaces.

RQ3 was posed to investigate how usability evaluations are currently conducted. The literature review revealed that 7 of the papers evaluated did not contain any usability evaluations. Some of the remaining papers included multiple studies to evaluate different aspects of a technology or were conducted at different times during the development process. Table  6 shows the percentage of studies that were conducted using each research methodology.

By far the most dominant research methodology used in the examined studies was controlled experiments, accounting for approximately 59% of the studies. In a controlled experiment, all variables are held constant except the independent variable, which is manipulated by the experimenter. The dependant variable is the metric which is measured by the experimenter. In this way a cause and effect relationship may be investigated between the dependant and independent variables. Causality can be inferred from the covariation of the independent and dependent variables, temporal precedence of the cause as the manipulation of the independent variable and the elimination of confounding factors though control and internal validity tests.

Although the most common approach is the use of controlled experiments, other research methodologies were also used. A number of studies evaluated the use of new technologies through field studies. Field studies are conducted in a real world context, enabling evaluators to determine how users would use a technology outside of a controlled setting. These studies often revealed issues that would not be seen in a controlled setting.

For example a system designed by Kristoffersen and Bratteberg [ 28 ] to help travellers get to and from an airport by train without the use of paper tickets was deployed. This system used a credit card as a form of ticket for a journey to or from the airport. During the field study a number of usability issues were experienced by travellers. One user wanted to use a card to buy a ticket for himself and a companion; the system did not include this functionality as the developers of the system had assumed each user would have their own credit card and therefore designed the system to issue each ticket on a different credit card.

The evaluation also revealed issues relating to how the developers had implemented the different journey types, i.e. to and from the airport. When travelling to the airport users are required to swipe their credit card at the beginning and end of each journey, whereas when returning from the airport the user only needs to swipe their card when leaving the airport. One user found this out after he had swiped his card to terminate a journey from the airport, but was instead charged for a second ticket to the airport.

Although controlled experiments and field studies account for almost 90% of the studies, other strategies are also used. Surveys were used to better understand how the public reacted to mobile systems. Some of these studies were specific to a new technology or paradigm, [ 29 ] while others considered uses such as working while on the move [ 30 ]. In two cases (1% of the studies) archival research was used to investigate a particular phenomena relating to mobile technologies. A study conducted by Fehnert and Kosagowsky [ 31 ] used archival research to investigate the relationship between expert evaluations of user experience quality of mobile phones and subsequent usage figures. Lacroix et al. [ 32 ] used archival research to investigate the relationship between goal difficulty and performance within the context of an on-going activity intervention program.

In some cases it was found that no formal evaluation was conducted but instead the new technology presented in the paper was evaluated informally with colleagues of the developers. These evaluations typically contained a small number of participants and provide anecdotal evidence of a system’s usability.

The results obtained during the literature review reinforced the importance of cognitive load as an attribute of usability. It was found that almost 23% of the studies measured the cognitive load of the application under evaluation. These results show that current researchers in the area of mobile applications are beginning to recognise the importance of cognitive load in this domain and as such there is sufficient evidence for including it within the PACMAD model of usability.

The results also show that Memorability is not considered an important aspect of usability by many researchers. Only 2% of the studies evaluated Memorability. If an application is easy to learn then users may be willing to relearn how to use the application and therefore Memorability may indeed not be significant. On the other hand, some applications have a high learning curve and as such require a significant amount of time to learn. For these applications Memorability is an important attribute.

The trade-off between Learnability and Memorability is a consideration for application developers. Factors such as the task to be accomplished and the characteristics of the user should be considered when making this decision. The PACMAD model recommends that both factors should be considered although it also recognises that it may be adequate to evaluate only one of these factors depending on the application under evaluation. The literature review has also shown that the remaining attributes of usability are considered extensively by current research. Effectiveness, Efficiency and Satisfaction were included in over 50% of the studies. It was also found the Errors were evaluated in over 30% of these studies.

When considering the factors that can affect usability, it was found that the task is the most dominant factor being researched. Over 45% of the papers examined focused primarily on allowing a user to accomplish a task. When the interaction with an application is itself considered as a task this figure rises to approximately 75%. Context of use and the User were considered in less than 10% of the papers. Context of use can vary enormously and so should be considered an important factor of usability [ 5 , 33 ]. Our results indicate that context is not extensively researched and this suggests a gap in the literature.

It was revealing that some components of the PACMAD model occur only infrequently in the literature. As mentioned above Learnability and Memorability are rarely investigated, perhaps suggesting that researchers expected users to be able to learn to use apps without much difficulty., This finding could also be due to the difficulty of finding suitable subjects willing to undergo experiments on these attributes or the lack of standard research methods for these attributes. Effectiveness, Efficiency, Satisfaction and Errors were investigated more frequently, possibly because these attributes are widely recognised as important, and also possibly because research methods for investigating these attributes are well understood and documented. Almost a quarter of the studies investigated discussed Cognitive Load. It is surprising that this figure is not higher although this could again be due to the lack of a well-defined research methodology for investigating this attribute.

Conclusions

The range and availability of mobile applications is expanding rapidly. With the increased processing power available on portable devices, developers are increasing the range of services that they provide. The small size of mobile devices has limited the ways in which users can interact with them. Issues such as the small screen size, poor connectivity and limited input modalities have an effect on the usability of mobile applications.

The prominent models of usability do not adequately capture the complexities of interacting with applications on a mobile platform. For this reason, this paper presents our PACMAD usability model which augments existing usability models within the context of mobile applications.

To prove the concept of this model a literature review has been conducted. This review has highlighted the extent to which the attributes of the PACMAD model are considered within the mobile application domain. It was found that each attribute was considered in at least 20% of studies, with the exception of Memorability. It is believed one reason for this may be the difficulty associated with evaluating Memorability.

The literature review has also revealed a number of novel interaction methods that are being researched at present, such as spatial awareness and pressure based input. These techniques are in their infancy but with time and more research they may eventually be adopted.

Appendix A: Papers used in the literature review

Apitz, G., F. Guimbretière, and S. Zhai, Foundations for designing and evaluating user interfaces based on the crossing paradigm. ACM Trans. Comput.-Hum. Interact., 2008. 17(2): p. 1–42.

Arning, K. and M. Ziefle, Ask and You Will Receive: Training Novice Adults to use a PDA in an Active Learning Environment. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(1): p. 21–47.

Arvanitis, T.N., et al., Human factors and qualitative pedagogical evaluation of a mobile augmented reality system for science education used by learners with physical disabilities. Personal Ubiquitous Comput., 2009. 13(3): p. 243–250.

Axtell, C., D. Hislop, and S. Whittaker, Mobile technologies in mobile spaces: Findings from the context of train travel. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 902–915.

Baber, C., et al., Mobile technology for crime scene examination. Int. J. Hum.-Comput. Stud., 2009. 67(5): p. 464–474.

Bardram, J.E., Activity-based computing for medical work in hospitals. ACM Trans. Comput.-Hum. Interact., 2009. 16(2): p. 1–36.

Bergman, J., J. Kauko, and J. Keränen, Hands on music: physical approach to interaction with digital music, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Bergman, J. and J. Vainio, Interacting with the flow, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Bertini, E., et al., Appropriating Heuristic Evaluation for Mobile Computing International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 20–41.

Böhmer, M. and G. Bauer, Exploiting the icon arrangement on mobile devices as information source for context-awareness, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Bostr, F., et al., Capricorn - an intelligent user interface for mobile widgets, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Brewster, S.A. and M. Hughes, Pressure-based text entry for mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Bruns, E. and O. Bimber, Adaptive training of video sets for image recognition on mobile phones. Personal Ubiquitous Comput., 2009. 13(2): p. 165–178.

Brush, A.J.B., et al., User experiences with activity-based navigation on mobile devices, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Burigat, S., L. Chittaro, and S. Gabrielli, Navigation techniques for small-screen devices: An evaluation on maps and web pages. Int. J. Hum.-Comput. Stud., 2008. 66(2): p. 78–97.

Büring, T., J. Gerken, and H. Reiterer, Zoom interaction design for pen-operated portable devices. Int. J. Hum.-Comput. Stud., 2008. 66(8): p. 605–627.

Buttussi, F., et al., Using mobile devices to support communication between emergency medical responders and deaf people, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Chen, N.Y., F. Guimbretière, and C.E. Löckenhoff, Relative role of merging and two-handed operation on command selection speed. Int. J. Hum.-Comput. Stud., 2008. 66(10): p. 729–740.

Chen, T., Y. Yesilada, and S. Harper, What input errors do you experience? Typing and pointing errors of mobile Web users. Int. J. Hum.-Comput. Stud., 2010. 68(3): p. 138–157.

Cherubini, M., et al., Text versus speech: a comparison of tagging input modalities for camera phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Chittaro, L. and A. Marassi, Supporting blind users in selecting from very long lists of items on mobile phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Chittaro, L. and D. Nadalutti, Presenting evacuation instructions on mobile devices by means of location-aware 3D virtual environments, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Clawson, J., et al., Mobiphos: a collocated-synchronous mobile photo sharing application, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Cockburn, A. and C. Gutwin, A model of novice and expert navigation performance in constrained-input interfaces. ACM Trans. Comput.-Hum. Interact., 2010. 17(3): p. 1–38.

Cox, A.L., et al., Tlk or txt? Using voice input for SMS composition. Personal Ubiquitous Comput., 2008. 12(8): p. 567–588.

Crossan, A., et al., Instrumented Usability Analysis for Mobile Devices International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 1–19.

Cui, Y., et al., Linked internet UI: a mobile user interface optimized for social networking, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Cummings, M.L., et al., Supporting intelligent and trustworthy maritime path planning decisions. Int. J. Hum.-Comput. Stud., 2010. 68(10): p. 616–626.

Dahl, Y. and D. Svan, A comparison of location and token-based interaction techniques for point-of-care access to medical information. Personal Ubiquitous Comput., 2008. 12(6): p. 459–478.

Dai, L., A. Sears, and R. Goldman, Shifting the focus from accuracy to recallability: A study of informal note-taking on mobile information technologies. ACM Trans. Comput.-Hum. Interact., 2009. 16(1): p. 1–46.

Decle, F. and M. Hachet, A study of direct versus planned 3D camera manipulation on touch-based mobile phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Duh, H.B.-L., V.H.H. Chen, and C.B. Tan, Playing different games on different phones: an empirical study on mobile gaming, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Dunlop, M.D. and M.M. Masters, Investigating five key predictive text entry with combined distance and keystroke modelling. Personal Ubiquitous Comput., 2008. 12(8): p. 589–598.

Ecker, R., et al., pieTouch: a direct touch gesture interface for interacting with in-vehicle information systems, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Eslambolchilar, P. and R. Murray-Smith, Control centric approach in designing scrolling and zooming user interfaces. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 838–856.

Fehnert, B. and A. Kosagowsky, Measuring user experience: complementing qualitative and quantitative assessment, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Fickas, S., M. Sohlberg, and P.-F. Hung, Route-following assistance for travelers with cognitive impairments: A comparison of four prompt modes. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 876–888.

Froehlich, P., et al., Exploring the design space of Smart Horizons, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Gellersen, H., et al., Supporting device discovery and spontaneous interaction with spatial references. Personal Ubiquitous Comput., 2009. 13(4): p. 255–264.

Ghiani, G., B. Leporini, and F. Patern, Vibrotactile feedback as an orientation aid for blind users of mobile guides, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Gostner, R., E. Rukzio, and H. Gellersen, Usage of spatial information for selection of co-located devices, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Goussevskaia, O., M. Kuhn, and R. Wattenhofer, Exploring music collections on mobile devices, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Greaves, A. and E. Rukzio, Evaluation of picture browsing using a projector phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hachet, M., et al., Navidget for 3D interaction: Camera positioning and further uses. Int. J. Hum.-Comput. Stud., 2009. 67(3): p. 225–236.

Hall, M., E. Hoggan, and S. Brewster, T-Bars: towards tactile user interfaces for mobile touchscreens, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hang, A., E. Rukzio, and A. Greaves, Projector phone: a study of using mobile phones with integrated projector for interaction with maps, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hardy, R., et al., Mobile interaction with static and dynamic NFC-based displays, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Heikkinen, J., T. Olsson, and K. Väänänen-Vainio-Mattila, Expectations for user experience in haptic communication with mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Henze, N. and S. Boll, Evaluation of an off-screen visualization for magic lens and dynamic peephole interfaces, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Herbst, I., et al., TimeWarp: interactive time travel with a mobile mixed reality game, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Hinze, A.M., C. Chang, and D.M. Nichols, Contextual queries express mobile information needs, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Hutter, H.-P., T. Müggler, and U. Jung, Augmented mobile tagging, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Jones, M., et al., ONTRACK: Dynamically adapting music playback to support navigation. Personal Ubiquitous Comput., 2008. 12(7): p. 513–525.

Joshi, A., et al., Rangoli: a visual phonebook for low-literate users, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Jumisko-Pyykk, S. and M.M. Hannuksela, Does context matter in quality evaluation of mobile television?, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kaasinen, E., User Acceptance of Mobile Services. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(1): p. 79–97 pp.

Kaasinen, E., et al., User Experience of Mobile Internet: Analysis and Recommendations. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 4–23.

Kane, S.K., J.O. Wobbrock, and I.E. Smith, Getting off the treadmill: evaluating walking user interfaces for mobile devices in public spaces, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kang, N.E. and W.C. Yoon, Age- and experience-related user behavior differences in the use of complicated electronic devices. Int. J. Hum.-Comput. Stud., 2008. 66(6): p. 425–437.

Kanjo, E., et al., MobGeoSen: facilitating personal geosensor data collection and visualization using mobile phones. Personal Ubiquitous Comput., 2008. 12(8): p. 599–607.

Kawsar, F., E. Rukzio, and G. Kortuem, An explorative comparison of magic lens and personal projection for interacting with smart objects, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Keijzers, J., E.d. Ouden, and Y. Lu, Usability benchmark study of commercially available smart phones: cell phone type platform, PDA type platform and PC type platform, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Kenteris, M., D. Gavalas, and D. Economou, An innovative mobile electronic tourist guide application. Personal Ubiquitous Comput., 2009. 13(2): p. 103–118.

Komninos, A. and M.D. Dunlop, A calendar based Internet content pre-caching agent for small computing devices. Personal Ubiquitous Comput., 2008. 12(7): p. 495–512.

Kratz, S., I. Brodien, and M. Rohs, Semi-automatic zooming for mobile map navigation, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Kray, C., et al., Bridging the gap between the Kodak and the Flickr generations: A novel interaction technique for collocated photo sharing. Int. J. Hum.-Comput. Stud., 2009. 67(12): p. 1060–1072.

Kristoffersen, S. and I. Bratteberg, Design ideas for IT in public spaces. Personal Ubiquitous Comput., 2010. 14(3): p. 271–286.

Lacroix, J., P. Saini, and R. Holmes, The relationship between goal difficulty and performance in the context of a physical activity intervention program, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Lavie, T. and J. Meyer, Benefits and costs of adaptive user interfaces. Int. J. Hum.-Comput. Stud., 2010. 68(8): p. 508–524.

Lee, J., J. Forlizzi, and S.E. Hudson, Iterative design of MOVE: A situationally appropriate vehicle navigation system. Int. J. Hum.-Comput. Stud., 2008. 66(3): p. 198–215.

Liao, C., et al., Papiercraft: A gesture-based command system for interactive paper. ACM Trans. Comput.-Hum. Interact., 2008. 14(4): p. 1–27.

Lin, P.-C. and L.-W. Chien, The effects of gender differences on operational performance and satisfaction with car navigation systems. Int. J. Hum.-Comput. Stud., 2010. 68(10): p. 777–787.

Lindley, S.E., et al., Fixed in time and “time in motion”: mobility of vision through a SenseCam lens, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Liu, K. and R.A. Reimer, Social playlist: enabling touch points and enriching ongoing relationships through collaborative mobile music listening, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Liu, N., Y. Liu, and X. Wang, Data logging plus e-diary: towards an online evaluation approach of mobile service field trial, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Liu, Y. and K.-J. Räihä, RotaTxt: Chinese pinyin input with a rotator, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Lucero, A., J. Keränen, and K. Hannu, Collaborative use of mobile phones for brainstorming, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Luff, P., et al., Swiping paper: the second hand, mundane artifacts, gesture and collaboration. Personal Ubiquitous Comput., 2010. 14(3): p. 287–299.

Mallat, N., et al., An empirical investigation of mobile ticketing service adoption in public transportation. Personal Ubiquitous Comput., 2008. 12(1): p. 57–65.

McAdam, C., C. Pinkerton, and S.A. Brewster, Novel interfaces for digital cameras and camera phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

McDonald, D.W., et al., Proactive displays: Supporting awareness in fluid social environments. ACM Trans. Comput.-Hum. Interact., 2008. 14(4): p. 1–31.

McKnight, L. and B. Cassidy, Children’s Interaction with Mobile Touch-Screen Devices: Experiences and Guidelines for Design. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(2): p. 1–18.

Melto, A., et al., Evaluation of predictive text and speech inputs in a multimodal mobile route guidance application, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Miyaki, T. and J. Rekimoto, GraspZoom: zooming and scrolling control model for single-handed mobile interaction, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Moustakas, K., et al., 3D content-based search using sketches. Personal Ubiquitous Comput., 2009. 13(1): p. 59–67.

Oakley, I. and J. Park, Motion marking menus: An eyes-free approach to motion input for handheld devices. Int. J. Hum.-Comput. Stud., 2009. 67(6): p. 515–532.

Oulasvirta, A., Designing mobile awareness cues, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Oulasvirta, A., S. Estlander, and A. Nurminen, Embodied interaction with a 3D versus 2D mobile map. Personal Ubiquitous Comput., 2009. 13(4): p. 303–320.

Ozok, A.A., et al., A Comparative Study Between Tablet and Laptop PCs: User Satisfaction and Preferences. International Journal of Human-Computer Interaction, 2008. 24(3): p. 329–352.

Park, Y.S., et al., Touch key design for target selection on a mobile phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Peevers, G., G. Douglas, and M.A. Jack, A usability comparison of three alternative message formats for an SMS banking service. Int. J. Hum.-Comput. Stud., 2008. 66(2): p. 113–123.

Preuveneers, D. and Y. Berbers, Mobile phones assisting with health self-care: a diabetes case study, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Puikkonen, A., et al., Practices in creating videos with mobile phones, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Reischach, F.v., et al., An evaluation of product review modalities for mobile phones, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Reitmaier, T., N.J. Bidwell, and G. Marsden, Field testing mobile digital storytelling software in rural Kenya, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Robinson, S., P. Eslambolchilar, and M. Jones, Exploring casual point-and-tilt interactions for mobile geo-blogging. Personal and Ubiquitous Computing, 2010. 14(4): p. 363–379.

Rogers, Y., et al., Enhancing learning: a study of how mobile devices can facilitate sensemaking. Personal Ubiquitous Comput., 2010. 14(2): p. 111–124.

Rohs, M., et al., Impact of item density on the utility of visual context in magic lens interactions. Personal Ubiquitous Comput., 2009. 13(8): p. 633–646.

Sá, M.d. and L. Carriço, Lessons from early stages design of mobile applications, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Sadeh, N., et al., Understanding and capturing people’s privacy policies in a mobile social networking application. Personal Ubiquitous Comput., 2009. 13(6): p. 401–412.

Salvucci, D.D., Rapid prototyping and evaluation of in-vehicle interfaces. ACM Trans. Comput.-Hum. Interact., 2009. 16(2): p. 1–33.

Salzmann, C., D. Gillet, and P. Mullhaupt, End-to-end adaptation scheme for ubiquitous remote experimentation. Personal Ubiquitous Comput., 2009. 13(3): p. 181–196.

Schildbach, B. and E. Rukzio, Investigating selection and reading performance on a mobile phone while walking, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Schmid, F., et al., Situated local and global orientation in mobile you-are-here maps, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Schröder, S. and M. Ziefle, Making a completely icon-based menu in mobile devices to become true: a user-centered design approach for its development, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Scott, J., et al., RearType: text entry using keys on the back of a device, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Seongil, L., Mobile Internet Services from Consumers’ Perspectives. International Journal of Human-Computer Interaction, 2009. 25(5): p. 390–413.

Sharlin, E., et al., A tangible user interface for assessing cognitive mapping ability. Int. J. Hum.-Comput. Stud., 2009. 67(3): p. 269–278.

Sintoris, C., et al., MuseumScrabble: Design of a Mobile Game for Children’s Interaction with a Digitally Augmented Cultural Space. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(2): p. 53–71.

Smets, N.J.J.M., et al., Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Sodnik, J., et al., A user study of auditory versus visual interfaces for use while driving. Int. J. Hum.-Comput. Stud., 2008. 66(5): p. 318–332.

Sørensen, C. and A. Al-Taitoon, Organisational usability of mobile computing-Volatility and control in mobile foreign exchange trading. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 916–929.

Stapel, J.C., Y.A.W.d. Kort, and W.A. IJsselsteijn, Sharing places: testing psychological effects of location cueing frequency and explicit vs. inferred closeness, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Streefkerk, J.W., M.P.v. Esch-Bussemakers, and M.A. Neerincx, Field evaluation of a mobile location-based notification system for police officers, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Takayama, L. and C. Nass, Driver safety and information from afar: An experimental driving simulator study of wireless vs. in-car information services. Int. J. Hum.-Comput. Stud., 2008. 66(3): p. 173–184.

Takeuchi, Y. and M. Sugimoto, A user-adaptive city guide system with an unobtrusive navigation interface. Personal Ubiquitous Comput., 2009. 13(2): p. 119–132.

Tan, F.B. and J.P.C. Chou, The Relationship Between Mobile Service Quality, Perceived Technology Compatibility, and Users’ Perceived Playfulness in the Context of Mobile Information and Entertainment Services. International Journal of Human-Computer Interaction, 2008. 24(7): p. 649–671.

Taylor, C.A., N. Samuels, and J.A. Ramey, Always On: A Framework for Understanding Personal Mobile Web Motivations, Behaviors, and Contexts of Use. International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 24–41.

Turunen, M., et al., User expectations and user experience with different modalities in a mobile phone controlled home entertainment system, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Vartiainen, E., Improving the User Experience of a Mobile Photo Gallery by Supporting Social Interaction International Journal of Mobile Human Computer Interaction (IJMHCI), 2009. 1(4): p. 42–57.

Vuolle, M., et al., Developing a questionnaire for measuring mobile business service experience, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Weinberg, G., et al., Contextual push-to-talk: shortening voice dialogs to improve driving performance, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Wilson, G., C. Stewart, and S.A. Brewster, Pressure-based menu selection for mobile devices, in Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. 2010, ACM: Lisbon, Portugal.

Wobbrock, J.O., B.A. Myers, and H.H. Aung, The performance of hand postures in front- and back-of-device interaction for mobile computing. Int. J. Hum.-Comput. Stud., 2008. 66(12): p. 857–875.

Xiangshi, R. and Z. Xiaolei, The Optimal Size of Handwriting Character Input Boxes on PDAs. International Journal of Human-Computer Interaction, 2009. 25(8): p. 762–784.

Xu, S., et al., Development of a Dual-Modal Presentation of Texts for Small Screens. International Journal of Human-Computer Interaction, 2008. 24(8): p. 776–793.

Yong, G.J. and J.B. Suk, Development of the Conceptual Prototype for Haptic Interface on the Telematics System. International Journal of Human-Computer Interaction, 2010. 26(1): p. 22–52.

Yoo, J.-W., et al., Cocktail: Exploiting Bartenders’ Gestures for Mobile Interaction. International Journal of Mobile Human Computer Interaction (IJMHCI), 2010. 2(3): p. 44–57.

Yoon, Y., et al., Context-aware photo selection for promoting photo consumption on a mobile phone, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

You, Y., et al., Deploying and evaluating a mixed reality mobile treasure hunt: Snap2Play, in Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. 2008, ACM: Amsterdam, The Netherlands.

Yu, K., F. Tian, and K. Wang, Coupa: operation with pen linking on mobile devices, in Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. 2009, ACM: Bonn, Germany.

Authors’ note

This research is supported by Oxford Brookes University through the central research fund and in part by Lero - the Irish Software Engineering Research Centre ( http://www.lero.ie ) grant 10/CE/I1855.

Adams R: Decision and stress: cognition and e-accessibility in the information workplace. Springer Universal Access in the Information Society 2007, 5 (4):363–379. 10.1007/s10209-006-0061-9

Article   Google Scholar  

Adams R: Applying advanced concepts of cognitive overload and augmentation in practice; the future of overload. In Foundations of augmented cognition . 2nd edition. Edited by: Schmorrow D, Stanney KM, Reeves LM. Arlington, VA: Springer Berlin Heidelberg; 2006:223–229.

Google Scholar  

Kjeldskov J, Graham C: A review of mobile HCI research methods . Udine, Italy: 5th International Symposium, Mobile HCI 2003; 2003. September 8–11, 2003, Proceedings

Book   Google Scholar  

Nielsen J: Usability engineering. Morgan Kaufmann Pub 1994.

ISO 9241: Ergonomics Requirements for Office Work with Visual Display Terminals (VDTs) International Standards Organisation, Geneva 1997.

Zhang D, Adipat B: Challenges, methodologies, and issues in the usability testing of mobile applications. International Journal of Human-Computer Interaction 2005, 18 (3):293–308. 10.1207/s15327590ijhc1803_3

Guerreiro TJV, Nicolau H, Jorge J, Gonçalves D Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. In Assessing mobile touch interfaces for tetraplegics . Lisbon, Portugal: ACM; 2010. 2010

Chapter   Google Scholar  

Schildbach B, Rukzio E Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Investigating selection and reading performance on a mobile phone while walking . Lisbon, Portugal: ACM; 2010. 2010

Flood D, Harrison R, Duce D, Iacob C: Evaluating Mobile Applications: A Spreadsheet Case Study. International Journal of Mobile Human Computer Interaction (IJMHCI) 2013, 4 (4):37–65. 10.4018/jmhci.2012100103

Salvucci DD: Predicting the effects of in-car interface use on driver performance: an integrated model approach. International Journal of Human-Computer Studies 2001, 55 (1):85–107. 10.1006/ijhc.2001.0472

Hart SG, Staveland LE: Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. Human mental workload 1988, 1 (3):139–183.

Flood D, Germanakos P, Harrison R, Mc Caffery F: Estimating cognitive overload in mobile applications for decision support within the medical domain . Wroclaw, Poland: 14th International conference on Enterprise Information Systems (ICEIS 2012); 2012.

Budgen D, Burn AJ, Brereton OP, Kitchenham BA, Pretorius R: (2010) Empirical evidence about the UML: a systematic literature review . Software: Practice and Experience; 2010.

Bruns E, Bimber O: Adaptive training of video sets for image recognition on mobile phones. Personal Ubiquitous Comput 2009, 13 (2):165–178. 10.1007/s00779-008-0194-3

Schinke T, Henze N, Boll S Proceedings of the 12th international conference on human computer interaction with mobile devices and services, September 07–10, 2010. In Visualization of off-screen objects in mobile augmented reality . Portugal: Lisbon; 2010.

Smets NJJM, Brake GM, Neerincx MA, Lindenberg J Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Effects of mobile map orientation and tactile feedback on navigation speed and situation awareness . Amsterdam, The Netherlands: ACM; 2008.

Ghiani G, Leporini B, Patern F Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Vibrotactile feedback as an orientation aid for blind users of mobile guides . Amsterdam, The Netherlands: ACM; 2008.

Jones M, Jones S, Bradley G, Warren N, Bainbridge D, Holmes G: ONTRACK: Dynamically adapting music playback to support navigation. Personal Ubiquitous Computing 2008, 12 (7):513–525. 10.1007/s00779-007-0155-2

Burigat, S, Chittaro, L, Parlato, E, ACM In proceedings of the 10th international conference on Human computer interaction with mobile devices and services (pp. 147–156). Map, diagram, and web page navigation on mobile devices: the effectiveness of zoomable user interfaces with overviews 2008. September

Sodnik J, Dicke C, Tomaic S, Billinghurst M: A user study of auditory versus visual interfaces for use while driving. Int. J. Hum.-Comput. Stud 2008, 66 (5):318–332. 10.1016/j.ijhcs.2007.11.001

Weinberg G, Harsham B, Forlines C, Medenica Z Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Contextual push-to-talk: shortening voice dialogs to improve driving performance . Lisbon, Portugal: ACM; 2010. 2010

Park YS, Han SH, Park J, Cho Y Proceedings of the 10th international conference on human computer interaction with mobile devices and services. In Touch key design for target selection on a mobile phone . Amsterdam, The Netherlands: ACM; 2008.

Brewster SA, Hughes M Proceedings of the 11th international conference on human-computer interaction with mobile devices and services. In Pressure-based text entry for mobile devices . Bonn, Germany: ACM; 2009.

Oakley I, Park J: Motion marking menus: an eyes-free approach to motion input for handheld devices. Int J Hum.-Comput. Stud 2009, 67 (6):515–532. 10.1016/j.ijhcs.2009.02.002

Hall M, Hoggan E, Brewster S Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In T-Bars: towards tactile user interfaces for mobile touchscreens . Amsterdam, The Netherlands: ACM; 2008. 2008

McAdam C, Pinkerton C, Brewster SA Proceedings of the 12th international conference on human computer interaction with mobile devices and services. In Novel interfaces for digital cameras and camera phones . Lisbon, Portugal: ACM; 2010. 2010

Heikkinen J, Olsson T, Väänänen-Vainio-Mattila K Proceedings of the 11th international conference on human-computer interaction with mobile devices and services. In Expectations for user experience in haptic communication with mobile devices . Bonn, Germany: ACM; 2009.

Kristoffersen S, Bratteberg I: Design ideas for IT in public spaces. Personal Ubiquitous Comput 2010, 14 (3):271–286. 10.1007/s00779-009-0255-2

Mallat N, Rossi M, Tuunainen VK, Oörni A: An empirical investigation of mobile ticketing service adoption in public transportation. Personal Ubiquitous Comput 2008, 12 (1):57–65.

Axtell C, Hislop D, Whittaker S: Mobile technologies in mobile spaces: findings from the context of train travel. International Journal of Human Computer Studies 2008, 66 (12):902–915. 10.1016/j.ijhcs.2008.07.001

Fehnert B, Kosagowsky A Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In Measuring user experience: complementing qualitative and quantitative assessment . Amsterdam, The Netherlands: ACM; 2008.

Lacroix J, Saini P, Holmes R Proceedings of the 10th international conference on Human computer interaction with mobile devices and services. In The relationship between goal difficulty and performance in the context of a physical activity intervention program, . Amsterdam, The Netherlands: ACM; 2008.

Maguire M: Context of use within usability activities. International Journal of Human-Computer Studies 2001, 55 (4):453–483. 2001 10.1006/ijhc.2001.0486

Download references

Author information

Authors and affiliations.

Oxford Brookes University, Oxford, UK

Rachel Harrison, Derek Flood & David Duce

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Rachel Harrison .

Additional information

Competing interests.

The authors declare that they have no competing interests.

Authors’ contributions

DF performed the literature review, helped to propose the PACMAD model and drafted the manuscript. RH assisted the literature review, proposed the PACMAD model and drafted the limitations section. DAD helped to refine the conceptual framework and direct the research. All authors read and approved the final manuscript.

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2, rights and permissions.

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article.

Harrison, R., Flood, D. & Duce, D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 1 , 1 (2013). https://doi.org/10.1186/2194-0827-1-1

Download citation

Received : 10 March 2013

Accepted : 10 March 2013

Published : 07 May 2013

DOI : https://doi.org/10.1186/2194-0827-1-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Mobile Phone
• Mobile Device
• Cognitive Load
• Augmented Reality
• Usability Model

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Gesture Recognition Using Visible Light on Mobile Devices

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, index terms.

Communication hardware, interfaces and storage

Human-centered computing

Human computer interaction (HCI)

Interaction devices

Pointing devices

Touch screens

Interaction techniques

Gestural input

Ubiquitous and mobile computing

Social and professional topics

Computing / technology policy

Government technology policy

Governmental regulations

Recommendations

Multi-scenario gesture recognition using kinect.

Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...

Eye gesture recognition on portable devices

Hand-held portable devices have received only little attention as a platform in the eye tracking community so far. This is mainly due to their -- until recently -- limited sensing capabilities and processing power. In this work-in-progress paper we ...

Reconstructing Hand Poses Using Visible Light

Free-hand gestural input is essential for emerging user interactions. We present Aili, a table lamp reconstructing a 3D hand skeleton in real time, requiring neither cameras nor on-body sensing devices. Aili consists of an LED panel in a lampshade and a ...

Information

Published in.

Publication History

• Research-article

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View Options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

View options.

View or Download as a PDF file.

View online with eReader .

Share this Publication link

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PeerJ Comput Sci

A systematic literature review on the usability of mobile applications for visually impaired users

Muna al-razgan.

1 King Saud University, Riyadh, Saudi Arabia

Sarah Almoaiqel

Nuha alrajhi.

2 Imam Muhammad Ibn Saud University, Riyadh, Saudi Arabia

Alyah Alhumegani

Abeer alshehri, bashayr alnefaie, raghad alkhamiss, shahad rushdi, associated data.

The following information was supplied regarding data availability:

This is a systematic literature review; there is no raw data.

Interacting with mobile applications can often be challenging for people with visual impairments due to the poor usability of some mobile applications. The goal of this paper is to provide an overview of the developments on usability of mobile applications for people with visual impairments based on recent advances in research and application development. This overview is important to guide decision-making for researchers and provide a synthesis of available evidence and indicate in which direction it is worthwhile to prompt further research. We performed a systematic literature review on the usability of mobile applications for people with visual impairments. A deep analysis following the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) guidelines was performed to produce a set of relevant papers in the field. We first identified 932 papers published within the last six years. After screening the papers and employing a snowballing technique, we identified 60 studies that were then classified into seven themes: accessibility, daily activities, assistive devices, navigation, screen division layout, and audio guidance. The studies were then analyzed to answer the proposed research questions in order to illustrate the different trends, themes, and evaluation results of various mobile applications developed in the last six years. Using this overview as a foundation, future directions for research in the field of usability for the visually impaired (UVI) are highlighted.

Introduction

The era of mobile devices and applications has begun. With the widespread use of mobile applications, designers and developers need to consider all types of users and develop applications for their different needs. One notable group of users is people with visual impairments. According to the World Health Organization, there are approximately 285 million people with visual impairments worldwide ( World Health Organization, 2020 ). This is a huge number to keep in mind while developing new mobile applications.

People with visual impairments have urged more attention from the tech community to provide them with the assistive technologies they need ( Khan & Khusro, 2021 ). Small tasks that we do daily, such as picking out outfits or even moving from one room to another, could be challenging for such individuals. Thus, leveraging technology to assist with such tasks can be life changing. Besides, increasing the usability of applications and developing dedicated ones tailored to their needs is essential. The usability of an application refers to its efficiency in terms of the time and effort required to perform a task, its effectiveness in performing said tasks, and its users’ satisfaction ( Ferreira et al., 2020 ). Researchers have been studying this field intensively and proposing different solutions to improve the usability of applications for people with visual impairments.

This paper provides a systematic literature review (SLR) on the usability of mobile applications for people with visual impairments. The study aims to find discussions of usability issues related to people with visual impairments in recent studies and how they were solved using mobile applications. By reviewing published works from the last six years, this SLR aims to update readers on the newest trends, limitations of current research, and future directions in the research field of usability for the visually impaired (UVI).

This SLR can be of great benefit to researchers aiming to become involved in UVI research and could provide the basis for new work to be developed, consequently improving the quality of life for the visually impaired. This review differs from previous review studies ( i.e.,   Khan & Khusro, 2021 ) because we classified the studies into themes in order to better evaluate and synthesize the studies and provide clear directions for future work. The following themes were chosen based on the issues addressed in the reviewed papers: “Assistive Devices,” “Navigation,” “Accessibility,” “Daily Activities,” “Audio Guidance,” and “Gestures.” Figure 1 illustrates the percentage of papers classified in each theme.

The remainder of this paper is organized as follows: the next section specifies the methodology, following this, the results section illustrates the results of the data collection, the discussion section consists of the research questions with their answers and the limitations and potential directions for future work, and the final section summarizes this paper’s main findings and contribution.

Survey Methodology

This systematic literature review used the Meta-Analyses (PRISMA, 2009) guidelines to produce a set of relevant papers in the field. This SLR was undertaken to address the research questions described below. A deep analysis was performed based on a group of studies; the most relevant studies were documented, and the research questions were addressed.

A. Research questions

The research questions addressed by this study are presented in Table 1 with descriptions and the motivations behind them.

B. Search strategy

This review analysed and synthesised studies on usability for the visually impaired from a user perspective following a systematic approach. As proposed by Tanfield, Denyer & Smart (2003) , the study followed a three-stage approach to ensure that the findings were both reliable and valid. These stages were planning the review, conducting the review by analysing papers, and reporting emerging themes and recommendations. These stages will be discussed further in the following section.

1. Planning stage

The planning stage of this review included defining data sources and the search string protocol as well as inclusion and exclusion criteria.

Data sources.

We aimed to use two types of data sources: digital libraries and search engines. The search process was manually conducted by searching through databases. The selected databases and digital libraries are as follows:

• • ACM Library
• • IEEE Xplore
• • ScienceDirect
• • SpringerLink
• • ISI Web of Knowledge
• • Scopus.

The selected search engines were as follows:

• • DBLP (Computer Science Bibliography Website)
• • Google Scholar
• • Microsoft Academic

Search string.

The above databases were initially searched using the following keyword protocol: (“Usability” AND (”visual impaired” OR ”visually impaired” OR “blind” OR “impairment”) AND “mobile”). However, in order to generate a more powerful search string, the Network Analysis Interface for Literature Studies (NAILS) project was used. NAILS is an automated tool for literature analysis. Its main function is to perform statistical and social network analysis (SNA) on citation data ( Knutas et al., 2015 ). In this study, it was used to check the most important work in the relevant fields as shown in Fig. 2 .

NAILS produced a report displaying the most important authors, publications, and keywords and listed the references cited most often in the analysed papers ( Knutas et al., 2015 ) . The new search string was generated after using the NAILS project as follows: (“Usability” OR “usability model” OR “usability dimension” OR “Usability evaluation model” OR “Usability evaluation dimension”) AND (“mobile” OR “Smartphone”) AND (“Visually impaired” OR “Visual impairment” OR “Blind” OR “Low vision” OR “Blindness”).

Inclusion and exclusion criteria.

To be included in this systematic review, each study had to meet the following screening criteria:

• • The study must have been published between 2015 and 2020.
• • The study must be relevant to the main topic (Usability of Mobile Applications for Visually Impaired Users).
• • The study must be a full-length paper.
• • The study must be written in English because any to consider any other languages, the research team will need to use the keywords of this language in this topic and deal with search engines using that language to extract all studies related to our topic to form an SLR with a comprehensive view of the selected languages. Therefore, the research team preferred to focus on studies in English to narrow the scope of this SLR.

A research study was excluded if it did not meet one or more items of the criteria.

2. Conducting stage

The conducting stage of the review involved a systematic search based on relevant search terms. This consisted of three substages: exporting citations, importing citations into Mendeley, and importing citations into Rayyan.

Exporting citations.

First, in exporting the citations and conducting the search through the mentioned databases, a total of 932 studies were found. The numbers are illustrated in Fig. 3 below. The highest number of papers was found in Google Scholar, followed by Scopus, ISI Web of Knowledge, ScienceDirect, IEEE Xplore, Microsoft Academic, and DBLP and ACM Library with two studies each. Finally, SpringerLink did not have any studies that met the inclusion criteria.

The chance of encountering duplicate studies was determined to be high. Therefore, importing citations into Mendeley was necessary in order to eliminate the duplicates.

Importing citations into mendeley.

Mendeley is an open-source reference and citation manager. It can highlight paragraphs and sentences, and it can also list automatic references on the end page. Introducing the use of Mendeley is also expected to avoid duplicates in academic writing, especially for systematic literature reviews ( Basri & Patak, 2015 ). Hence, in the next step, the 932 studies were imported into Mendeley, and each study’s title and abstract were screened independently for eligibility. A total of 187 duplicate studies were excluded. 745 total studies remained after the first elimination process. The search stages are shown in Fig. 4 below.

Importing citations into rayyan.

Rayyan QCRI is a free web and mobile application that helps expedite the initial screening of both abstracts and titles through a semi-automated process while incorporating a high level of usability. Its main benefit is to speed up the most tedious part of the systematic literature review process: selecting studies for inclusion in the review ( Ouzzani et al., 2016 ). Therefore, for the last step, another import was done using Rayyan to check for duplications a final time. Using Rayyan, a total of 124 duplicate studies were found, resulting in a total of 621 studies. Using Rayyan, a two-step filtration was conducted to guarantee that the papers have met the inclusion criteria of this SLR. After filtering based on the abstracts, 564 papers did not meet the inclusion criteria. At this stage, 57 studies remained. The second step of filtration eliminated 11 more studies by reading the full papers; two studies were not written in the English language, and nine were inaccessible.

Snowballing.

Snowballing is an emerging technique used to conduct systematic literature reviews that are considered both efficient and reliable using simple procedures. The procedure for snowballing consisted of three phases in each cycle. The first phase is refining the start set, the second phase is backward snowballing, and the third is forward snowballing. The first step, forming the start set, is basically identifying relevant papers that can have a high potential of satisfying the criteria and research question. Backward snowballing was conducted using the reference list to identify new papers to include. It shall start by going through the reference list and excluding papers that do not fulfill the basic criteria; the rest that fulfil criteria shall be added to the SLR. Forward snowballing refers to identifying new papers based on those papers that cited the paper being examined ( Juneja & Kaur, 2019 ). Hence, in order to be sure that we concluded all related studies after we got the 46 papers, a snowballing step was essential. Forward and backward snowballing were conducted. Each of the 46 studies was examined by checking their references to take a look at any possible addition of sources and examining all papers that cited this study. The snowballing activity added some 38 studies, but after full reading, it became 33 that matched the inclusion criteria. A total of 79 studies were identified through this process.

Quality assessment.

A systematic literature review’s quality is determined by the content of the papers included in the review. As a result, it is important to evaluate the papers carefully ( Zhou et al., 2015 ). Many influential scales exist in the software engineering field for evaluating the validity of individual primary studies and grading the overall intensity of the body of proof. Hence, we adapted the comprehensive guidelines specified by Kitchenhand and Charters ( Keele, 2007 ), and the quasi-gold standard (QGS) ( Keele, 2007 ) was used to establish the quest technique, where a robust search strategy for enhancing the validity and reliability of a SLR’s search process is devised using the QGS. By applying this technique, our quality assessment questions were focused and aligned with the research questions mentioned earlier.

In our last step, we had to verify the papers’ eligibility; we conducted a quality check for each of the 79 studies. For quality assessment, we considered whether the paper answered the following questions:

QA1: Is the research aim clearly stated in the research?

QA2: Does the research contain a usability dimension or techniques for mobile applications for people with visual impairments?

QA3: Is there an existing issue with mobile applications for people with visual impairments that the author is trying to solve?

QA4: Is the research focused on mobile application solutions?

After discussing the quality assessment questions and attempting to find an answer in each paper, we agreed to score each study per question. If the study answers a question, it will be given 2 points; if it only partially answers a question, it will be given 1 point; and if there is no answer for a given question in the study, it will have 0 points.

The next step was to calculate the weight of each study. If the total weight was higher or equal to four points, the paper was accepted in the SLR; if not, the paper was discarded since it did not reach the desired quality level. Figure 5 below illustrates the quality assessment process. After applying the quality assessment, 39 papers were rejected since they received less than four points, which resulted in a final tally of 60 papers.

To summarize, this review was conducted according to the Preferred Reporting Items for SLRs and Meta-Analyses (PRISMA) ( Liberati et al., 2009 ). The PRISMA diagram shown in Fig. 6 illustrates all systematic literature processes used in this study.

3. Analysing stage

All researchers involved in this SLR collected the data. The papers were distributed equally between them, and each researcher read each paper completely to determine its topic, extract the paper’s limitations and future work, write a quick summary about it, and record this information in an Excel spreadsheet.

All researchers worked intensively on this systematic literature review. After completing the previously mentioned steps, the papers were divided among all the researchers. Then, each researcher read their assigned papers completely and then classified them into themes according to the topic they covered. The researchers held several meetings to discuss and specify those themes. The themes were identified by the researchers based on the issues addressed in the reviewed papers. In the end, the researchers resulted in seven themes, as shown in Fig. 7 below. The references selected for each theme can be found in the Table A1 . Afterwards, each researcher was assigned one theme to summarize its studies and report the results. In this section, we review the results.

A. Accessibility

Of a total of 60 studies, 10 focused on issues of accessibility. Accessibility is concerned with whether all users are able to have equivalent user experiences, regardless of abilities. Six studies, Darvishy, Hutter & Frei (2019) , Morris et al. (2016) , Qureshi & Hooi-Ten Wong (2020) , Khan, Khusro & Alam (2018) , Paiva et al. (2020) , and Pereda, Murillo & Paz (2020) , gave suggestions for increasing accessibility, ( Darvishy, Hutter & Frei, 2019 ; Morris et al., 2016 ), gave some suggestions for making mobile map applications and Twitter accessible to visually impaired users, and ( Qureshi & Hooi-Ten Wong, 2020 ; Khan, Khusro & Alam, 2018 ) focused on user interfaces and provided accessibility suggestions suitable for blind people. Paiva et al. (2020) and Pereda, Murillo & Paz (2020) proposed a set of heuristics to evaluate the accessibility of mobile applications. Two studies, Khowaja et al. (2019) and Carvalho et al. (2018) , focused on evaluating usability and accessibility issues on some mobile applications, comparing them, and identifying the number and types of problems that visually impaired users faced. Aqle, Khowaja & Al-Thani (2020) proposed a new web search interface designed for visually impaired users. One study, McKay (2017) , focused on accessibility challenges by applying usability tests on a hybrid mobile app with some visually impaired university students.

B. Assistive devices

People with visual impairments have an essential need for assistive technology since they face many challenges when performing activities in daily life. Out of the 60 studies reviewed, 13 were related to assistive technology. The studies Smaradottir, Martinez & Håland (2017) , Skulimowski et al. (2019) , Barbosa, Hayes & Wang, (2016) , Rosner & Perlman (2018) , Csapó et al. (2015) , Khan & Khusro (2020) , Sonth & Kallimani (2017) , Kim et al. (2016) , Vashistha et al. (2015) ; Kameswaran et al. (2020) , Griffin-Shirley et al. (2017) , and Rahman, Anam & Yeasin (2017) were related to screen readers (voiceovers). On the other hand, Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) were related to proposing an assistant device for the visually impaired. Of the studies related to screening readers, Sonth & Kallimani, (2017) , Vashistha et al. (2015) , Khan & Khusro (2020) Lewis et al. (2016) cited challenges faced by visually impaired users. Barbosa, Hayes & Wang (2016) , Kim et al. (2016) , Rahman, Anam & Yeasin (2017) suggested new applications, while Smaradottir, Martinez & Håland (2017) , Rosner & Perlman (2018) , Csapó et al. (2015) and Griffin-Shirley et al. (2017) evaluated current existing work. The studies Bharatia, Ambawane & Rane (2019) , Lewis et al. (2016) proposed using wearable devices to improve the quality of life for people with visual impairments.

C. Daily activities

In recent years, people with visual impairments have used mobile applications to increase their independence in their daily activities and learning, especially those based on the braille method. We divide the daily activity section into braille-based applications and applications designed to enhance the independence of the visually impaired. Four studies, Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , Araújo et al. (2016) and Gokhale et al. (2017) , implemented and evaluated the usability of mobile phone applications that use braille to help visually impaired people in their daily lives. Seven studies, Vitiello et al. (2018) , Kunaratana-Angkul, Wu & Shin-Renn (2020) , Ghidini et al. (2016) , Madrigal-Cadavid et al. (2019) , Marques, Carriço & Guerreiro (2015) , Oliveira et al. (2018) and Rodrigues et al. (2015) , focused on building applications that enhance the independence and autonomy of people with visual impairments in their daily life activities.

D. Screen division layout

People with visual impairments encounter various challenges in identifying and locating non-visual items on touch screen interfaces like phones and tablets. Incidents of accidentally touching a screen element and frequently following an incorrect pattern in attempting to access objects and screen artifacts hinder blind people from performing typical activities on smartphones ( Khusro et al., 2019 ). In this review, 9 out of 60 studies discuss screen division layout: ( Khusro et al., 2019 ; Khan & Khusro, 2019 ; Grussenmeyer & Folmer, 2017 ; Palani et al., 2018 ; Leporini & Palmucci, 2018 ) discuss touch screen (smartwatch tablets, mobile phones, and tablet) usability among people with visual impairments, while ( Cho & Kim, 2017 ; Alnfiai & Sampalli, 2016 ; Niazi et al., 2016 ; Alnfiai & Sampalli, 2019 ) concern text entry methods that increase the usability of apps among visually impaired people. Khusro et al. (2019) provides a novel contribution to the literature regarding considerations that can be used as guidelines for designing a user-friendly and semantically enriched user interface for blind people. An experiment in Cho & Kim (2017) was conducted comparing the two-button mobile interface usability with the one-finger method and voiceover. Leporini & Palmucci (2018) gathered information on the interaction challenges faced by visually impaired people when answering questions on a mobile touch-screen device, investigated possible solutions to overcome the accessibility and usability challenges.

E. Gestures

In total, 3 of 60 studies discuss gestures in usability. Alnfiai & Sampalli (2017) compared the performance of BrailleEnter, a gesture based input method to the Swift Braille keyboard, a method that requires finding the location of six buttons representing braille dot, while Buzzi et al. (2017) and Smaradottir, Martinez & Haland (2017) provide an analysis of gesture performance on touch screens among visually impaired people.

F. Audio guidance

People with visual impairment primarily depend on audio guidance forms in their daily lives; accordingly, audio feedback helps guide them in their interaction with mobile applications.

Four studies discussed the use of audio guidance in different contexts: one in navigation ( Gintner et al., 2017 ), one in games ( Ara’ujo et al., 2017 ), one in reading ( Sabab & Ashmafee, 2016 ), and one in videos ( Façanha et al., 2016 ). These studies were developed and evaluated based on usability and accessibility of the audio guidance for people with visual impairments and aimed to utilize mobile applications to increase the enjoyment and independence of such individuals.

G. Navigation

Navigation is a common issue that visually impaired people face. Indoor navigation is widely discussed in the literature. Nair et al. (2020) , Al-Khalifa & Al-Razgan (2016) and De Borba Campos et al. (2015) discuss how we can develop indoor navigation applications for visually impaired people. Outdoor navigation is also common in the literature, as seen in Darvishy et al. (2020) , Hossain, Qaiduzzaman & Rahman (2020) , Long et al. (2016) , Prerana et al. (2019) and Bandukda et al. (2020) . For example, in Darvishy et al. (2020) , Touch Explorer, an accessible digital map application, was presented to alleviate many of the problems faced by people with visual impairments while using highly visually oriented digital maps. Primarily, it focused on using non-visual output modalities like voice output, everyday sound, and vibration feedback. Issues with navigation applications were also presented in Maly et al. (2015) . Kameswaran et al. (2020) discussed commonly used technologies in navigation applications for blind people and highlighted the importance of using complementary technologies to convey information through different modalities to enhance the navigation experience. Interactive sonification of images for navigation has also been shown in Skulimowski et al. (2019) .

In this section, the research questions are addressed in detail to clearly achieve the research objective. Also, a detailed overview of each theme will be mentioned below.

Answers to the research questions

This section will answer the research question proposed:

RQ1: What existing UVI issues did authors try to solve with mobile devices?

Mobile applications can help people with visual impairments in their daily activities, such as navigation and writing. Additionally, mobile devices may be used for entertainment purposes. However, people with visual impairments face various difficulties while performing text entry operations, text selection, and text manipulation on mobile applications ( Niazi et al., 2016 ). Thus, the authors of the studies tried to increase touch screens’ usability by producing prototypes or simple systems and doing usability testing to understand the UX of people with visual impairments.

RQ2: What is the role of mobile devices in solving those issues?

Mobile phones are widely used in modern society, especially among users with visual impairments; they are considered the most helpful tool for blind users to communicate with people worldwide ( Smaradottir, Martinez & Håland, 2017 ). In addition, the technology of touch screen assistive technology enables speech interaction between blind people and mobile devices and permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities perform actions or interact with systems ( Niazi et al., 2016 ).

RQ3: What are the publication trends on the usability of mobile applications among the visually impaired?

As shown in Fig. 8 below, research into mobile applications’ usability for the visually impaired has increased in the last five years, with a slight dip in 2018. Looking at the most frequent themes, we find that “Assistive Devices” peaked in 2017, while “Navigation” and “Accessibility” increased significantly in 2020. On the other hand, we see that the prevalence of “Daily Activities” stayed stable throughout the research years. The term “Audio Guidance” appeared in 2016 and 2017 and has not appeared in the last three years. “Gestures” also appeared only in 2017. “Screen Layout Division” was present in the literature in the last five years and increased in 2019 but did not appear in 2020.

RQ4: What are the current research limitations and future research directions regarding usability among the visually impaired?

We divide the answer to this question into two sections: first, we will discuss limitations; then, we will discuss future work for each proposed theme.

A. Limitations

Studies on the usability of mobile applications for visually impaired users in the literature have various limitations, and most of them were common among the studies. These limitations were divided into two groups. The first group concerns proposed applications; for example, Rahman, Anam & Yeasin (2017) , Oliveira et al. (2018) and Madrigal-Cadavid et al. (2019) faced issues regarding camera applications in mobile devices due to the considerable effort needed for its usage and being heavily dependent on the availability of the internet. The other group of studies, Rodrigues et al. (2015) , Leporini & Palmucci (2018) , Alnfiai & Sampalli (2016) , and Ara’ujo et al. (2017) , have shown limitations in visually impaired users’ inability to comprehend a graphical user interface. Alnfiai & Sampalli (2017) and Alnfiai & Sampalli (2019) evaluated new braille input methods and found that the traditional braille keyboard, where knowing the exact position of letters QWERTY is required, is limited in terms of usability compared to the new input methods. Most studies faced difficulties regarding the sample size and the fact that many of the participants were not actually blind or visually impaired but only blindfolded. This likely led to less accurate results, as blind or visually impaired people can provide more useful feedback as they experience different issues on a daily basis and are more ideal for this type of study. So, the need for a good sample of participants who actually have this disability is clear to allow for better evaluation results and more feedback and recommendations for future research.

B. Future work

A commonly discussed future work in the chosen literature is to increase the sample sizes of people with visual impairment and focus on various ages and geographical areas to generalize the studies. Table 2 summarizes suggestions for future work according to each theme. Those future directions could inspire new research in the field.

RQ5: What is the focus of research on usability for visually impaired people, and what are the research outcomes in the studies reviewed?

There are a total of 60 outcomes in this research. Of these, 40 involve suggestions to improve usability of mobile applications; four of them address problems that are faced by visually impaired people that reduce usability. Additionally, 16 of the outcomes are assessments of the usability of the prototype or model. Two of the results are recommendations to improve usability. Finally, the last two outcomes are hardware solutions that may help the visually impaired perform their daily activities. Figure 9 illustrates these numbers.

Overview of the reviewed studies

In the following subsections, we summarize all the selected studies based on the classified theme: accessibility, assistive devices, daily activities, screen division layout, gestures, audio guidance, and navigation. The essence of the studies will be determined, and their significance in the field will be explored.

For designers dealing with mobile applications, it is critical to determine and fix accessibility issues in the application before it is delivered to the users ( Khowaja et al., 2019 ). Accessibility refers to giving the users the same user experience regardless of ability. In Khowaja et al. (2019) and Carvalho et al. (2018) , the researchers focused on comparing the levels of accessibility and usability in different applications. They had a group of visually impaired users and a group of sighted users test out the applications to compare the number and type of problems they faced and determine which applications contained the most violations. Because people with visual impairments cannot be ignored in the development of mobile applications, many researchers have sought solutions for guaranteeing accessibility. For example, in Qureshi & Hooi-Ten Wong (2020) , the study contributed to producing a new, effective design for mobile applications based on the suggestions of people with visual impairments and with the help of two expert mobile application developers. In Khan, Khusro & Alam (2018) , an adaptive user interface model for visually impaired people was proposed and evaluated in an empirical study with 63 visually impaired people. In Aqle, Khowaja & Al-Thani (2020) , the researchers proposed a new web search interface for users with visual impairments that is based on discovering concepts through formal concept analysis (FCA). Users interact with the interface to collect concepts, which are then used as keywords to narrow the search results and target the web pages containing the desired information with minimal effort and time. The usability of the proposed search interface (InteractSE) was evaluated by experts in the field of HCI and accessibility, with a set of heuristics by Nielsen and a set of WCAG 2.0 guidelines.

In Darvishy, Hutter & Frei (2019) , the researchers proposed a solution for making mobile map applications accessible for people with blindness or visual impairment. They suggested replacing forests in the map with green color and birds’ sound, replacing water with blue color and water sounds, replacing streets with grey color and vibration, and replacing buildings with yellow color and pronouncing the name of the building. The prototype showed that it was possible to explore a simple map through vibrations, sounds, and speech.

In Morris et al. (2016) the researchers utilized a multi-faceted technique to investigate how and why visually impaired individuals use Twitter and the difficulties they face in doing so. They noted that Twitter had become more image-heavy over time and that picture-based tweets are largely inaccessible to people with visual impairments. The researchers then made several suggestions for how Twitter could be amended to continue to be usable for people with visual impairments.

The researchers in Paiva et al. (2020) focused on how to evaluate proposed methods for ensuring the accessibility and usability of mobile applications. Their checklist, Acc-MobileCheck, contains 47 items that correspond to issues related to comprehension (C), operation (O), perception (P), and adaptation (A) in mobile interface interaction. To validate Acc-MobileCheck, it was reviewed by five experts and three developers and determined to be effective. In Pereda, Murillo & Paz (2020) , the authors also suggest a set of heuristics to evaluate the accessibility of mobile e-commerce applications for visually impaired people. Finally, McKay (2017) conducted an accessibility test for hybrid mobile apps and found that students with blindness faced many barriers to access based on how they used hybrid mobile applications. While hybrid apps can allow for increased time for marketing, this comes at the cost of app accessibility for people with disabilities.

A significant number of people with visual impairments use state-of-the-art software to perform tasks in their daily lives. These technologies are made up of electronic devices equipped with sensors and processors that can make intelligent decisions.

One of the most important and challenging tasks in developing such technologies is to create a user interface that is appropriate for the sensorimotor capabilities of users with blindness ( Csapó et al., 2015 ). Several new hardware tools have proposed to improve the quality of life for people with visual impairments. Three tools were presented in this SLR: a smart stick that can notify the user of any obstacle, helping them to perform tasks easily and efficiently ( Bharatia, Ambawane & Rane, 2019 ), and an eye that can allow users to detect colors (medical evaluation is still required) ( Lewis et al., 2016 ).

The purpose of the study in Griffin-Shirley et al. (2017) was to understand how people with blindness use smartphone applications as assistive technology and how they perceive them in terms of accessibility and usability. An online survey with 259 participants was conducted, and most of the participants rated the applications as useful and accessible and were satisfied with them.

The researchers in Rahman, Anam & Yeasin (2017) designed and implemented EmoAssist, which is a smartphone application that assists with natural dyadic conversations and aims to promote user satisfaction by providing options for accessing non-verbal communication that predicts behavioural expressions and contains interactive dimensions to provide valid feedback. The usability of this application was evaluated in a study with ten people with blindness where several tools were applied in the application. The study participants found that the usability of EmoAssist was good, and it was an effective assistive solution.

This theme contains two main categories: braille-based application studies and applications to enhance the independence of VIU. Both are summarized below.

1- Braille-based applications

Braille is still the most popular method for assisting people with visual impairments in reading and studying, and most educational mobile phone applications are limited to sighted people. Recently, however, some researchers have developed assistive education applications for students with visual impairments, especially those in developing countries. For example, in India, the number of children with visual impairments is around 15 million, and only 5% receive an education ( Gokhale et al., 2017 ). Three of the braille studies focused on education: ( Nahar, Sulaiman & Jaafar, 2020 ; Nahar, Jaafar & Sulaiman, 2019 , and Araújo et al., 2016 ). These studies all used smartphone touchscreens and action gestures to gain input from the student, and then output was provided in the form of audio feedback. In Nahar, Sulaiman & Jaafar (2020) , vibrational feedback was added to guide the users. The participants in Nahar, Sulaiman & Jaafar (2020) ; Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) included students with blindness of visual impairment and their teachers. The authors in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) evaluated the usability of their applications following the same criteria (efficiency, learnability, memorability, errors, and satisfaction). The results showed that in Nahar, Sulaiman & Jaafar (2020) , Nahar, Jaafar & Sulaiman (2019) , and Araújo et al. (2016) , the applications met the required usability criteria. The authors in Gokhale et al. (2017) presented a braille-based solution to help people with visual impairments call and save contacts. A braille keypad on the smartphone touchscreen was used to gain input from the user, which was then converted into haptic and auditory feedback to let the user know what action was taken. The usability of this application was considered before it was designed. The participants’ responses were positive because this kind of user-centric design simplifies navigation and learning processes.

2- Applications to enhance the independence of people with visual impairments

The authors in the studies explored in this section focused on building applications that enhance independence and autonomy in daily life activities for users with visual impairments.

In Vitiello et al. (2018) , the authors presented their mobile application, an assistive solution for visually impaired users called “Crania”, which uses machine learning techniques to help users with visual impairments get dressed by recognizing the colour and texture of their clothing and suggesting suitable combinations. The system provides feedback through voice synthesis. The participants in the study were adults and elderly people, some of whom were completely blind and the rest of whom had partial sight. After testing for usability, all the participants with blindness agreed that using the application was better than their original method, and half of the participants with partial sight said the same thing. At the end of the study, the application was determined to be accessible and easy to use.

In Kunaratana-Angkul, Wu & Shin-Renn (2020) , an application which allows elderly people to measure low vision status at home through their smartphones instead of visiting hospitals was tested, and most of the participants considered it to be untrustworthy because the medical information was insufficient. Even when participants were able to learn how to use the application, most of them were still confused while using it and needed further instruction.

In Ghidini et al. (2016) , the authors studied the habits of people with visual impairments when using their smartphones in order to develop an electronic calendar with different interaction formats, such as voice commands, touch, and vibration interaction. The authors presented the lessons learned and categorized them based on usability heuristics such as feedback, design, user freedom and control, and recognition instead of remembering.

In Madrigal-Cadavid et al. (2019) , the authors developed a drug information application for people with visual impairments to help them access the labels of medications. The application was developed based on a user-centered design process. By conducting a usability test, the authors recognized some usability issues for people with visual impairments, such as difficulty in locating the bar code. Given this, a new version will include a search function that is based on pictures. The application is searched by capturing the bar code or text or giving voice commands that allow the user to access medication information. The participants were people with visual impairments, and most of them required assistance using medications before using the application. This application will enhance independence for people with visual impairments in terms of using medications.

In Marques, Carriço & Guerreiro (2015) , an authentication method is proposed for users with visual impairments that allows them to protect their passwords. It is not secure when blind or visually impaired users spell out their passwords or enter the numbers in front of others, and the proposed solution allows the users to enter their password with one hand by tapping the screen. The blind participants in this study demonstrated that this authentication method is usable and supports their security needs.

In Oliveira et al. (2018) , the author noted that people with visual impairments face challenges in reading, thus he proposed an application called LeR otulos. This application was developed and evaluated for the Android operating system and recognizes text from photos taken by the mobile camera and converts them into an audio description. The prototype was designed to follow the guidelines and recommendations of usability and accessibility. The requirements of the application are defined based on the following usability goals: the steps are easy for the user to remember; the application is efficient, safe, useful, and accessible; and user satisfaction is achieved.

Interacting with talkback audio devices is still difficult for people with blindness, and it is unclear how much benefit they provide to people with visual impairments in their daily activities. The author in Rodrigues et al. (2015) investigates the smartphone adoption process of blind users by conducting experiments, observations, and weekly interviews. An eight-week study was conducted with five visually impaired participants using Samsung and an enabled talkback 2 screen reader. Focusing on understanding the experiences of people with visual impairments when using touchscreen smartphones revealed accessibility and usability issues. The results showed that the participants have difficulties using smartphones because they fear that they cannot use them properly, and that impacts their ability to communicate with family. However, they appreciate the benefits of using smartphones in their daily activities, and they have the ability to use them.

People with visual impairments encounter various challenges identifying and locating non-visual items on touch screen interfaces, such as phones and tablets. Various specifications for developing a user interface for people with visual impairments must be met, such as having touch screen division to enable people with blindness to easily and comfortably locate objects and items that are non-visual on the screen ( Khusro et al., 2019 ). Article ( Khusro et al., 2019 ) highlighted the importance of aspects of the usability analysis, such as screen partitioning, to meet specific usability requirements, including orientation, consistency, operation, time consumption, and navigation complexity when users want to locate objects on their touchscreen. The authors of Khan & Khusro (2019) describe the improvements that people with blindness have experienced in using the smartphone while performing their daily tasks. This information was determined through an empirical study with 41 people with blindness who explained their user and interaction experiences operating a smartphone.

The authors in Palani et al. (2018) provide design guidelines governing the accurate display of haptically perceived graphical materials. Determining the usability parameters and the various cognitive abilities required for optimum and accurate use of device interfaces is crucial. Also the authors of Grussenmeyer & Folmer (2017) highlight the importance of usability and accessibility of smartphones and touch screens for people with visual impairments. The primary focus in Leporini & Palmucci (2018) is on interactive tasks used to finish exercises and to answer questionnaires or quizzes. These tools are used for evaluation tests or in games. When using gestures and screen readers to interact on a mobile device, difficulties may arise ( Leporini & Palmucci, 2018 ), The study has various objectives, including gathering information on the difficulties encountered by people with blindness during interactions with mobile touch screen devices to answer questions and investigating practicable solutions to solve the detected accessibility and usability issues. A mobile app with an educational game was used to apply the proposed approach. Moreover, in Alnfiai & Sampalli (2016) and Niazi et al. (2016) , an analysis of the single-tap braille keyboard created to help people with no or low vision while using touch screen smartphones was conducted. The technology used in Alnfiai & Sampalli (2016) was the talkback service, which provides the user with verbal feedback from the application, allowing users with blindness to key in characters according to braille patterns. To evaluate single tap braille, it was compared to the commonly used QWERTY keyboard. In Niazi et al. (2016) , it was found that participants adapted quickly to single-tap Braille and were able to type on the touch screen within 15 to 20 min of being introduced to this system. The main advantage of single tap braille is that it allows users with blindness to enter letters based on braille coding, which they are already familiar with. The average error rate is lower using single-tap Braille than it is on the QWERTY keyboard. The authors of Niazi et al. (2016) found that minimal typing errors were made using the proposed keypad, which made it an easier option for people with blindness ( Niazi et al., 2016 ). In Cho & Kim (2017) , the authors describe new text entry methods for the braille system including a left touch and a double touch scheme that form a two-button interface for braille input so that people with visual impairments are able to type textual characters without having to move their fingers to locate the target buttons.

One of the main problems affecting the visually impaired is limited mobility for some gestures. We need to know what gestures are usable by people with visual impairments. Moreover, the technology of assistive touchscreen-enabled speech interaction between blind people and mobile devices permits the use of gestures to interact with a touch user interface. Assistive technology is vital in helping people living with disabilities to perform actions or interact with systems. Smaradottir, Martinez & Haland (2017) analyses a voiceover screen reader used in Apple Inc.’s products. An assessment of this assistive technology was conducted with six visually impaired test participants. The main objectives were to pinpoint the difficulties related to the performance of gestures applicable in screen interactions and to analyze the system’s response to the gestures. In this study, a user evaluation was completed in three phases. The first phase entailed training users regarding different hand gestures, the second phase was carried out in a usability laboratory where participants were familiarized with technological devices, and the third phase required participants to solve different tasks. In Knutas et al. (2015) , the vital feature of the system is that it enables the user to interactively select a 3D scene region for sonification by merely touching the phone screen. It uses three different modes to increase usability. Alnfiai & Sampalli (2017) explained a study done to compare the use of two data input methods to evaluate their efficiency with completely blind participants who had prior knowledge of braille. The comparison was made between the braille enter input method that uses gestures and the swift braille keyboard, which necessitates finding six buttons representing braille dots. Blind people typically prefer rounded shapes to angular ones when performing complex gestures, as they experience difficulties performing straight gestures with right angles. Participants highlighted that they experienced difficulties particularly with gestures that have steep or right angles. In Buzzi et al. (2017) , 36 visually impaired participants were selected and split into two groups of low-vision and blind people. They examined their touch-based gesture preferences in terms of the number of strokes, multitouch, and shape angles. For this reason, a wireless system was created to record sample gestures from various participants simultaneously while monitoring the capture process.

People with visual impairment typically cannot travel without guidance due to the inaccuracy of current navigation systems in describing roads and especially sidewalks. Thus, the author of Gintner et al. (2017) aims to design a system to guide people with visual impairments based on geographical features and addresses them through a user interface that converts text to audio using a built-in voiceover engine (Apple iOS). The system was evaluated positively in terms of accessibility and usability as tested in a qualitative study involving six participants with visual impairment.

Based on challenges faced by visually impaired game developers, Ara’ujo et al. (2017) provides guidance for developers to provide accessibility in digital games by using audio guidance for players with visual impairments. The interactions of the player can be conveyed through audio and other basic mobile device components with criteria focused on the game level and speed adjustments, high contrast interfaces, accessible menus, and friendly design. Without braille, people with visual impairments cannot read, but braille is expensive and takes effort, and so it is important to propose technology to facilitate reading for them. In Sabab & Ashmafee (2016) , the author proposed developing a mobile application called “Blind Reader” that reads an audio document and allows the user to interact with the application to gain knowledge. This application was evaluated with 11 participants, and the participants were satisfied with the application. Videos are an important form of digital media, and unfortunately people with visual impairment cannot access these videos. Therefore, Façanha et al. (2016) aims to discover sound synthesis techniques to maximize and accelerate the production of audio descriptions with low-cost phonetic description tools. This tool has been evaluated based on usability with eight people and resulted in a high acceptance rate among users.

1- Indoor navigation

Visually impaired people face critical problems when navigating from one place to another. Whether indoors or outdoors, they tend to stay in one place to avoid the risk of injury or seek the help of a sighted person before moving ( Al-Khalifa & Al-Razgan, 2016 ). Thus, aid in navigation is essential for those individuals. In Nair et al. (2020) , Nair developed an application called ASSIST, which leverages Bluetooth low energy (BLE) beacons and augmented reality (AR) to help visually impaired people move around cluttered indoor places ( e.g. , subways) and provide the needed safe guidance, just like having a sighted person lead the way. In the subway example, the beacons will be distributed across the halls of the subway and the application will detect them. Sensors and cameras attached to the individual will detect their exact location and send the data to the application. The application will then give a sequence of audio feedback explaining how to move around the place to reach a specific point ( e.g. , “in 50 ft turn right”, “now turn left”, “you will reach the destination in 20 steps”). The application also has an interface for sighted and low-vision users that shows the next steps and instructions. A usability study was conducted to test different aspects of the proposed solution. The majority of the participants agreed that they could easily reach a specified location using the application without the help of a sighted person. A survey conducted to give suggestions from the participants for future improvements showed that most participants wanted to attach their phones to their bodies and for the application to consider the different walking speeds of users. They were happy with the audio and vibration feedback that was given before each step or turn they had to take.

In Al-Khalifa & Al-Razgan (2016) , the main purpose of the study was to provide an Arabic-language application for guidance inside buildings using Google Glass and an associated mobile application. First, the building plan must be set by a sighted person who configures the different locations needed. Ebsar will ask the map builder to mark each interesting location with a QR code and generate a room number, and the required steps and turns are tracked using the mobile device’s built-in compass and accelerometer features. All of these are recorded in the application for the use of a visually impaired individual, and at the end, a full map is generated for the building. After setting the building map, a user can navigate inside the building with the help of Ebsar, paired with Google Glass, for input and output purposes. The efficiency, effectiveness, and levels of user satisfaction with this solution were evaluated. The results showed that the errors made were few, indicating that Ebsar is highly effective. The time consumed in performing tasks ranged from medium to low depending on the task; this can be improved later. Interviews with participants indicated the application’s ease of use. De Borba Campos et al. (2015) shows an application simulating a museum map for people with visual impairments. It discusses whether mental maps and interactive games can be used by people with visual impairments to recognize the space around them. After multiple usability evaluation sessions, the mobile application showed high efficiency among participants in understanding the museum’s map without repeating the visitation. The authors make a few suggestions based on feedback from the participants regarding enhancing usability, including using audio cues, adding contextual help to realise the activities carried around in a space, and focusing on audio feedback instead of graphics.

2- Outdoor navigation

Outdoor navigation is also commonly discussed in the literature. In Darvishy et al. (2020) , Touch Explorer was presented to alleviate many of the problems faced by visually impaired people in navigation by developing a non-visual mobile digital map. The application relies on three major methods of communication with the user: voice output, vibration feedback, and everyday sounds. The prototype was developed using simple abstract visuals and mostly relies on voice for explanation of the content. Usability tests show the great impact the prototype had on the understanding of the elements of the map. Few suggestions were given by the participants to increase usability, including GPS localization to locate the user on the map, a scale element for measuring the distance between two map elements, and an address search function.

In Hossain, Qaiduzzaman & Rahman (2020) , a navigation application called Sightless Helper was developed to provide a safe navigation method for people with visual impairments. It relies on footstep counting and GPS location to provide the needed guidance. It can also ensure safe navigation by detect objects and unsafe areas and can detect unusual shaking of the user and alert an emergency contact about the problem. The user interaction categories are voice recognition, touchpad, buttons, and shaking sensors. After multiple evaluations, the application was found to be useful in different scenarios and was considered usable by people with visual impairments. The authors in Long et al. (2016) propose an application that uses both updates from users and information about the real world to help visually impaired people navigate outdoor settings. After interviews with participants, some design goals were set, including the ability to tag an obstacle on the map, check the weather, and provide an emergency service. The application was evaluated and was found to be of great benefit; users made few errors and found it easy to use. In Prerana et al. (2019) , a mobile application called STAVI was presented to help visually impaired people navigate from a source to a destination safely and avoid issues of re-routing. The application depends on voice commands and voice output. The application also has additional features, such as calling, messages, and emergency help. The authors in Bandukda et al. (2020) helped people with visual impairments explore parks and natural spaces using a framework called PLACES. Different interviews and surveys were conducted to identify the issues visually impaired people face when they want to do any leisure activity. These were considered in the development of the framework, and some design directions were presented, such as the use of audio to share an experience.

3- General issues

The authors in Maly et al. (2015) discuss implementing an evaluation model to assess the usability of a navigation application and to understand the issues of communication with mobile applications that people with visual impairments face. The evaluation tool was designed using a client–server architecture and was applied to test the usability of an existing navigation application. The tool was successful in capturing many issues related to navigation and user behavior, especially the issue of different timing between the actual voice instruction and the position of the user. The authors in Kameswaran et al. (2020) conducted a study to find out which navigation technologies blind people can use and to understand the complementarity between navigation technologies and their impact on navigation for visually impaired users. The results of the study show that visually impaired people use both assistive technologies and those designed for non-visually impaired users. Improving voice agents in navigation applications was discussed as a design implication for the visually impaired. In Skulimowski et al. (2019) , the authors show how interactive sonification can be used in simple travel aids for the blind. It uses depth images and a histogram called U-depth, which is simple auditory representations for blind users. The vital feature of this system is that it enables the user to interactively select a 3D scene region for sonification by touching the phone screen. This sonic representation of 3D scenes allows users to identify the environment’s general appearance and determine objects’ distance. The prototype structure was tested by three blind individuals who successfully performed the indoor task. Among the test scenes used included walking along an empty corridor, walking along a corridor with obstacles, and locating an opening between obstacles. However, the results showed that it took a long time for the testers to locate narrow spaces between obstacles.

RQ6: What evaluation methods were used in the studies on usability for visually impaired people that were reviewed?

The most prevalent methods to evaluate the usability of applications were surveys and interviews. These were used to determine the usability of the proposed solutions and obtain feedback and suggestions regarding additional features needed to enhance the usability from the participants’ points of view. Focus groups were also used extensively in the literature. Many of the participants selected were blindfolded and were not actually blind or visually impaired. Moreover, the samples selected for the evaluation methods mentioned above considered the age factor depending on the study’s needs.

Limitation and future work

The limitations of this paper are mainly related to the methodology followed. Focusing on just eight online databases and restricting the search with the previously specified keywords and string may have limited the number of search results. Additionally, a large number of papers were excluded because they were written in other languages. Access limitations were also faced due to some libraries asking for fees to access the papers. Therefore, for future works, a study to expand on the SLR results and reveal the current usability models of mobile applications for the visually impaired to verify the SLR results is needed so that this work contributes positively to assessing difficulties and expanding the field of usability of mobile applications for users with visual impairments.

Conclusions

In recent years, the number of applications focused on people with visual impairments has grown, which has led to positive enhancements in those people’s lives, especially if they do not have people around to assist them. In this paper, the research papers focusing on usability for visually impaired users were analyzed and classified into seven themes: accessibility, daily activities, assistive devices, gestures, navigation, screen division layout, and audio guidance. We found that various research studies focus on accessibility of mobile applications to ensure that the same user experience is available to all users, regardless of their abilities. We found many studies that focus on how the design of the applications can assist in performing daily life activities like braille-based application studies and applications to enhance the independence of VI users. We also found papers that discuss the role of assistive devices like screen readers and wearable devices in solving challenges faced by VI users and thus improving their quality of life. We also found that some research papers discuss limited mobility of some gestures for VI users and investigated ways in which we can know what gestures are usable by people with visual impairments. We found many research papers that focus on improving navigation for VI users by incorporating different output modalities like sound and vibration. We also found various studies focusing on screen division layout. By dividing the screen and focusing on visual impairment-related issues while developing user interfaces, visually impaired users can easily locate the objects and items on the screens. Finally, we found papers that focus on audio guidance to improve usability. The proposed applications use voice-over and speech interactions to guide visually impaired users in performing different activities through their mobiles. Most of the researchers focused on usability in different applications and evaluated the usability issues of these applications with visually impaired participants. Some of the studies included sighted participants to compare the number and type of problems they faced. The usability evaluation was generally based on the following criteria: accessibility, efficiency, learnability, memorability, errors, safety, and satisfaction. Many of the studied applications show a good indication of these applications’ usability and follow the participants’ comments to ensure additional enhancements in usability. This paper aims to provide an overview of the developments on usability of mobile applications for people with visual impairments and use this overview to highlight potential future directions.

References selected for each theme.

Funding Statement

This research project was supported by a grant from the Research Center of the Female Scientific and Medical Colleges, Deanship of Scientific Research, King Saud University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional Information and Declarations

The authors declare there are no competing interests.

Muna Al-Razgan , Sarah Almoaiqel , Nuha Alrajhi , Alyah Alhumegani , Abeer Alshehri , Bashayr Alnefaie , Raghad AlKhamiss and Shahad Rushdi conceived and designed the experiments, performed the experiments, analyzed the data, performed the computation work, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Ohio State nav bar

The Ohio State University

• BuckeyeLink
• Find People
• Search Ohio State

A new ventilator-on-a-chip model to study lung damage

Device enables real-time detection of injury at cellular level.

For the first time, scientists are able to directly compare the different kinds of injury that mechanical ventilation causes to cells in the lungs.

In a new study, using a ventilator-on-a-chip model developed at The Ohio State University, researchers found that shear stress from the collapse and reopening of the air sacs is the most injurious type of damage.

This miniature “organ-on-a-chip” model simulates not only lung injury during mechanical ventilation, but also repair and recovery, in human-derived cells in real time, said co-lead author Samir Ghadiali , PhD, professor and chair of biomedical engineering at Ohio State .

“The initial damage is purely physical, but the processes after that are biological in nature – and what we’re doing with this device is coupling the two,” Ghadiali said.

The team hopes the device will also help in the hunt for therapies to address ventilator-induced lung injury.

“This is an important advance in the field that will hopefully allow for a better understanding of how lung injury develops in mechanically ventilated patients and identification of therapeutic targets so that we can give drugs to prevent that kind of injury or treat it when it happens,” said co-lead author Joshua Englert , MD, associate professor of pulmonary, critical care and sleep medicine at The Ohio State University Wexner Medical Center .

The research was published recently in the journal Lab on a Chip .

Ventilators save the lives of patients with severe respiratory problems related to disease or trauma, but it has been known for a long time that the mechanical forces exerted on the lungs also cause injury. The damage at the cellular level can make the barrier between tiny air sacs and capillaries carrying blood become leaky, leading to fluid buildup that interferes with oxygen getting to the lungs.

Of particular value is the ventilator-on-a chip’s measurement of real-time changes to cells that affect the integrity of that barrier, enabled by an innovative approach: growing human lung cells on a synthetic nanofiber membrane mimicking the complex lung matrix. It’s closer to the authentic ventilated lung microenvironment than any similar lung chip systems to date, the researchers say.

The device measures the effects of three types of mechanical stress on the integrity of the barrier: lung cell stretch from overinflation, increased pressure on lung cells, and cyclical collapse and reopening of air sacs.

Experiments showed that overinflation with a high volume of air and cyclic collapse and reopening of air sacs both led the barrier to become leaky, but the cells could recover more quickly from overinflation than from the repetitive opening and closing of air sacs.

Englert said the collapse and reopening may be more problematic because it makes fluid in the lungs move, exposing cells to high amounts of shear stress.

“There really hasn’t been a lot of data that could allow for the comparison of those two injurious forces in the same system,” he said. “But now for the first time, we can use the same device with the same cells and induce both types of injury and see what happens. Our data suggests neither one of them is good, they’re both injurious, but that the collapse and reopening seems to be more severe and makes recovery harder.”

This finding was a demonstration of the model’s sophistication, Ghadiali said.

“We knew for a long time that collapse and reopening is a pretty injurious force, but we never could measure it in real time,” he said. “Now that we know that collapse and reopening injury happens much quicker and takes a long time to recover, we plan to use the ventilator on a chip to figure out how to prevent this injury and/or enhance the repair.”

Next steps involve modeling diseases such as pneumonia and traumatic injuries experienced by ICU patients in combination with mechanical action.

“We’re in the early stages of developing some of those models, diving a little bit deeper into the complexity of lung injury in ICU patients,” Englert said. “This model is a platform we can build upon.”

Ghadiali and Englert, also investigators in Ohio State’s Davis Heart and Lung Research Institute , credited first author Basia Gabela-Zuniga, who recently received her PhD in biomedical engineering, with getting the project over the finish line, and acknowledged the contributions of College of Engineering co-authors Heather Powell and Natalia Higuita-Castro. Additional co-authors were Vasudha Shukla and Christopher Bobba of Ohio State.

This work was supported by the National Institutes of Health and the U.S. Department of Defense.

More Ohio State News

Buckeye current team captures world records.

There’s fast, and then there’s Buckeye fast. For the Buckeye Current electric motorcycle team, there is no one faster.

Ohio State’s Farm Science Review connects university to agricultural community

The Ohio State University’s 62nd Farm Science Review is underway this week. The three-day event highlights the deep connections between the university and the state’s agriculture community.

Hemp shows high promise as potential natural insecticide

As part of the race to combat global insecticide resistance, new research shows that the same CBD people use to treat a variety of ailments is also extremely effective at killing mosquito larvae.

Contact: Admissions | Webmaster | Page maintained by University Communications

Request an alternate format of this page | Web Services Status | Nondiscrimination notice

• Samsung expands mobile phone production beyond Asian countries, says DIGITIMES Research

Credit: DIGITIMES

Samsung's strategic expansion of its global manufacturing footprint in 2023 underscores its commitment to diversifying production, mitigating risks, and bolstering its regional market presence. By increasing production diversification, Samsung aims to enhance flexibility and cost-efficiency, with India, Brazil, Argentina, and Pakistan emerging as key hubs in this strategy, according to DIGITIMES Research .

DIGITIMES Research analyst Yen Chou reports that in 2023, Samsung expanded its global manufacturing footprint. In addition to traditional production bases like India and Vietnam, Samsung has been developing manufacturing ecosystems in Brazil, Argentina, and Pakistan.

These efforts are expected to diversify Samsung's production base, minimize risks associated with supply chain disruptions, and strengthen its position in regional markets. This diversification of manufacturing operations allows Samsung to reduce its dependency on any single location while increasing flexibility and cost-efficiency in production.

India, along with Vietnam, continues to be a major hub in Samsung's mobile phone manufacturing strategy, thanks to the manufacturing facility located in Greater Noida, Uttar Pradesh. The company is ramping up production across a broad spectrum of devices, from entry-level to premium models.

Chou noted that India has become an export hub for high-end models like the Galaxy S24 Plus, which are now being shipped to European markets. This shift aligns with Samsung's broader goal of leveraging local production to quickly meet regional demands while cutting down on costs related to international logistics.

Samsung's Brazilian operations have experienced substantial growth, with the local supply chain now supporting the production of mid-range to high-end devices, including the Galaxy Z Flip5 5G. Furthermore, Chou reported that Brazil has begun local manufacturing of accessories such as charging cables and manuals for the Galaxy S23 FE, further deepening the country's integration into Samsung's global supply chain.

In Pakistan, Samsung has been outsourcing smartphone assembly to Lucky Motor, a local electronics manufacturing services (EMS) provider, since 2021, including the assembly of the Galaxy S23 FE 5G. However, despite these advancements, Chou noted that Pakistan's local supply chain remains underdeveloped, with components such as charging cables and power adapters still being imported from Vietnam.

Phone Arena reports that in 2023, around 21 million mobile phones were produced domestically in Pakistan, with local and Chinese brands leading the charge, while 1.7 million units were imported, according to the Pakistan Telecommunication Authority. The country offers incentives to mobile phone manufacturers to encourage local production.

Similarly, Samsung's manufacturing operations in Argentina have expanded with the help of a local EMS provider. Early models like the Galaxy A22 4G and Galaxy A53 5G were among the first to be assembled locally. The company has since increased its capabilities to assemble devices such as the Galaxy Z Flip5 5G. However, like Pakistan, Chou indicated that Argentina's supply chain is still evolving, relying on imported accessories from Vietnam to complement the assembly process.

Samsung's diversification of its global manufacturing network serves as a strategic response to geopolitical uncertainties and the growing demand for region-specific products. By expanding in emerging markets, Samsung is better positioned to cater to local preferences while ensuring a more resilient and adaptive supply chain across its key regions.

Samung mobile phone production update

Full range of low, mid, and high-end models, with increasing production capacity

Maturing local manufacturing supply chain, increased production. Expanded product line to include mid-range and high-end models, such as Galaxy Z Flip5 5G

Outsourced assembly to local EMS providers, primarily for mid-range and low-end models. Relies on imported accessories, supply chain still incomplete

Lucky Motor handles assembly, increased production capacity. Primarily produces mid-range models, relies on imported accessories

Source: DIGITIMES Research, September 2024

About the analyst

Yen Chou received a master's degree from the Graduate Program for Political Economy at National Cheng Kung University. He worked as an assistant researcher at the Institute for Physical Planning and Information and as an analyst at DIGITIMES Research . His research focuses on Samsung's global investment and operation, the electronics industry in Vietnam, and the display industry in South Korea.

Credit: DIGITIMES Research

• TOP STORIES
• 7 DAYS NEWS
• How far is China from 28nm process after domestic lithography breakthrough?
• Tata plans to build two more fabs in India
• Huawei's triple-fold Mate XT reaches 5 million pre-orders, but converting to actual orders will be challenging
• Debunking China's first indigenous DUV lithography system
• Huawei encounters difficulties with Kirin SoC performance breakthroughs
• Glass substrate technology gains traction for advanced packaging by 2026
• Chinese smartphones employ 3nm chips, exacerbating Qualcomm and MediaTek competition
• TSMC solves mass production issues in US; advanced process self-sufficiency may be realized
• Will Luxshare gain from Huawei-shunning Chinese carmakers?

Members only

• Please login to read more
• New users, please register first

Notification

New method identifies biomarkers in wastewater using origami-paper sensors

• Download PDF Copy

Researchers at Cranfield University have developed an innovative new method for identifying biomarkers in wastewater using origami-paper sensors, enabling the tracking of infectious diseases using the camera in a mobile phone. The new test device is low-cost and fast and could dramatically change how public health measures are directed in any future pandemics.

Wastewater a key way to track infections

Testing wastewater is one of the primary ways to assess the prevalence of infectious diseases in populations. Researchers take samples from various water treatment plants around the country and use the results to understand which areas currently have the highest infection rates. The method was used in the COVID-19 pandemic to track community infection rates and direct public health measures.

Zhugen Yang, Professor of Biosensing and Environmental Health at Cranfield University, led the development of the sentinel sensors. It builds on research he conducted in 2020 to develop a test to detect SARS-CoV-2 (commonly known as COVID-19), Influenza A and Influenza B in wastewater using a paper-based platform and UV torch or mobile phone camera.

Until now, the most accurate ways of testing wastewater samples have been methods like the polymerase chain reaction (PCR) test which must be conducted in centralised laboratories by well-trained personnel. That means samples are collected, stored and transported in a cold chain to the lab location before being processed, which can take multiple days and is comparatively expensive.

'Origami' folded paper-based test is read by mobile phone

The new test method is rapid, user-friendly and portable. Wastewater samples are placed onto a wax-printed paper sheet which is folded in an 'origami' style. The paper contains chemicals that react to certain disease markers, triggering a fluorescent colour to emerge. Using a mobile phone camera, the results can be read and data collected rapidly

Professor Yang developed the new method as part of the national COVID-19 wastewater surveillance programme. In 2021, at the height of the pandemic, he performed field tests using the test at four quarantine hotels around Heathrow Airport. The whole sample-to-answer process took under 90 mins compared to around four hours for a PCR test, with the tests conducted in the basement of one of the hotels using minimal equipment. Results showed that this new device gives results at least as accurate as the PCR test but at a much lower cost and can provide an early warning of disease in the community. The device is particularly useful for areas with limited resource because of its ease of use, low cost and fast results.

During COVID-19 we proved that fast community sewage analysis is a really effective way to track infectious diseases and help manage public health. The simple test we have developed costs just £1 and uses the commonly available camera function in a mobile phone, making it readily accessible. This could be a real game-changer when it comes to predicting disease rates and improving public health in the face of future pandemics." Zhugen Yang, Professor of Biosensing and Environmental Health, Cranfield University

Related Stories

• Researchers find no evidence of monkeypox infection in pets, despite presence of viral DNA
• Chinese farmed fur animals found to harbor dozens of novel viruses
• NIH funds new research to combat emerging viruses

The device has been featured at the London Science Museum recognising its contribution to the National Wastewater Surveillance Programme during the COVID-19 pandemic.

Further development of the test is being sponsored by the Leverhulme Trust Research Leadership Scheme and a grant from the Biotechnology and Biological Sciences Research Council. In future, it could potentially be used to track new variants and help to establish whether the variant is still spreading in the community, as well as monitoring antimicrobial resistance from one health perspective.

Paper microfluidic sentinel sensor enables rapid and on-site wastewater surveillance in community settings is published in the journal Cell Reports Physical Science, and also involved scientists from the University of Glasgow and Zhejiang University in China.

Cranfield University

Pan, Y., et al . (2024). Paper microfluidic sentinel sensors enable rapid and on-site wastewater surveillance in community settings.  Cell Reports Physical Science . doi.org/10.1016/j.xcrp.2024.102154 .

Posted in: Device / Technology News | Disease/Infection News | Healthcare News

Tags: Antimicrobial Resistance , Biotechnology , Cell , Chemicals , Cold , Cold chain , covid-19 , Education , Infectious Diseases , Influenza , Pandemic , Polymerase , Polymerase Chain Reaction , Public Health , Research , SARS , SARS-CoV-2 , Technology

Suggested Reading

Cancel reply to comment

• Trending Stories
• Latest Interviews
• Top Health Articles

How can microdialysis benefit drug development

Ilona Vuist

In this interview, discover how Charles River uses the power of microdialysis for drug development as well as CNS therapeutics.

Global and Local Efforts to Take Action Against Hepatitis

Lindsey Hiebert and James Amugsi

In this interview, we explore global and local efforts to combat viral hepatitis with Lindsey Hiebert, Deputy Director of the Coalition for Global Hepatitis Elimination (CGHE), and James Amugsi, a Mandela Washington Fellow and Physician Assistant at Sandema Hospital in Ghana. Together, they provide valuable insights into the challenges, successes, and the importance of partnerships in the fight against hepatitis.

Addressing Important Cardiac Biology Questions with Shotgun Top-Down Proteomics

In this interview conducted at Pittcon 2024, we spoke to Professor John Yates about capturing cardiomyocyte cell-to-cell heterogeneity via shotgun top-down proteomics.

Latest News

Newsletters you may be interested in

Your AI Powered Scientific Assistant

Hi, I'm Azthena, you can trust me to find commercial scientific answers from News-Medical.net.

A few things you need to know before we start. Please read and accept to continue.

• Use of “Azthena” is subject to the terms and conditions of use as set out by OpenAI .
• Content provided on any AZoNetwork sites are subject to the site Terms & Conditions and Privacy Policy .
• Large Language Models can make mistakes. Consider checking important information.

Great. Ask your question.

Azthena may occasionally provide inaccurate responses. Read the full terms .

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions .

Provide Feedback

• Best Phones
• Best Android Phones
• Best iPhone
• Best Phones for Kids
• Best Tablets
• Best Android Tablets
• Best Smartwatch
• Best Fitness Trackers
• Best Cell Phone Plans
• Best Prepaid Phone Plans
• Best USB-C Cables
• Best Smart Notebooks
• Best Smart Rings
• Best Garmin Watch

Digital Trends may earn a commission when you buy through links on our site. Why trust us?

Remarkable Paper Pro review: one digital notebook to rule them all

“The Remarkable Paper Pro is fantastic for distraction-free note-taking with color.”

• Thin and lightweight
• Distraction-free for maximum productivity
• Minimal latency
• Color e-ink screen
• Built-in light
• Easy to use
• Long battery life
• Not backward compatible with previous stylus
• Not a great e-reader

These days, most people take notes on a computer or tablet , and more often than not, those notes are typed out instead of handwritten. It’s becoming rarer to handwrite notes, but if you’re the kind of person who can get easily distracted by everything else on a computer, handwriting notes is still your best bet.

Remarkable Paper Pro: design and accessories

Remarkable paper pro: display, remarkable paper pro: software and performance, remarkable paper pro: battery and charging, remarkable paper pro: verdict.

When I was in school, I remember taking notes on paper before moving on to a laptop. But some classes, like math or science, required drawing diagrams or working out complex equations, which can be tricky with only a computer. Those were a few times when I wished I could have had a tablet reminiscent of paper and pen that would also let me type if I needed to.

Enter the Remarkable Paper Pro. This is the third generation of the Remarkable tablet, which debuted in 2016 and was followed up with the Remarkable 2 in 2020. Remarkable tablets feature an e-ink display and use a proprietary stylus to recreate the feeling of writing with pen and paper. The Paper Pro is the first Remarkable tablet with a color e-ink screen.

Though it’s been years since I’ve been in school, the Remarkable Paper Pro is exactly the kind of tool I wish I’d had back then.

The design of the Remarkable Paper Pro is as simple as it comes. The body is aluminum, and the paper-like surface is textured glass. The display is 11.8 inches, which makes it just like a sheet of A4 paper. It’s super lightweight and thin at 274.1mm by 196.6mm by 5.1 mm (10.8 inches by 7.8 inches by 0.24 inches) and just over 1 pound.

Since it’s so thin and light, it’s very comfortable to hold. The edges of the frame have two thin grooves going all around, which give it a distinctive look. The sleep/wake button is located on the top edge to the left.

The design of the Remarkable Paper Pro is as simple as it comes.

On the rear, you’ll find the Remarkable logo in the middle aligned to the left, four little rubberized “feet” in each corner, and a connector strip at the bottom for use with the Type Folio accessory. The little rubber feet help prevent the Remarkable Paper Pro from moving around on a table when you’re writing on it, and they also have magnets to attach to the Book Folio or Type Folio cases.

The front of the Paper Pro has a textured glass color e-ink display surrounded by three uniform bezels at the top and sides. However, the bottom bezel is a bit thicker than the rest, but serves no purpose. It would have been nice if a button or gesture could be used here for easier navigation (but you can swipe on the screen to go back and forth on pages).

When you purchase a Remarkable Paper Pro, it comes with the standard Marker stylus for free, but you can upgrade to the Marker Plus for $50 more. My unit came with the Marker Plus, which has a finely textured finish that’s nicer on the hand and has a built-in eraser at the top. It is similar to an Apple Pencil, and it even attaches magnetically to the side of the Paper Pro to charge. Both Marker models come with an additional six replacement tips for free.

You can also purchase the Book Folio or Type Folio cases. The Book Folio is a basic cover for the Paper Pro that keeps it safe and secure on the go. It costs $89.

The Type Folio adds a full-size keyboard to your Paper Pro, backlit keys, and a palm rest. It’s quite comfortable to type on, and the keys have good tactile feedback and travel. If you plan to use the Paper Pro for productivity, then the Type Folio is a must. It’s pretty pricey at $229, but if you want a keyboard for your Paper Pro, this is the only way to go about it, as Remarkable does not support third-party keyboards.

The Remarkable Paper Pro features an 11.8-inch color e-ink display. However, it’s not like other color e-ink screens where there’s just a color filter on the display.

Instead, Remarkable uses what it calls Canvas Color technology, which it has built on top of the e-ink color display. Canvas Color utilizes physical color particles underneath that move around the screen as you go. Remarkable claims that this technology means better saturation and depth, more natural tones, and better blending.

The Canvas Color technology is interesting because it appears black or gray as you write or draw. But once you lift the Marker from the screen, the Paper Pro recolors what you wrote or drew to the chosen color. Unfortunately, typed text can only appear in black.

The Remarkable Paper Pro also differentiates itself from its predecessors by having an integrated light. No, it’s not quite a backlight like you’re used to with a regular tablet or smartphone. It’s a front light, similar to other e-readers. This light faces the display and illuminates it, reflecting the screen back to you, allowing the Paper Pro to be used in the dark. If you’ve used something like a Kindle Paperwhite, it’s similar to that.

Even at full brightness, it’s hard to tell that the Paper Pro has a light. But this illuminated glow is useful when you want to use the Paper Pro late at night, on a nighttime flight, or in a dimly lit room for a presentation.

Remarkable’s tablets run Remarkable OS, and the Paper Pro is no different. Remarkable OS is a custom Linux-based operating system designed specifically for digital paper displays.

If you’ve used an e-ink display before, you should be familiar with how this device works. Unlike regular smartphone and tablet screens, there is always that tiny delay on the screen when navigating because the paper display needs to refresh. But for the most part, using the Remarkable Paper Pro has been a smooth experience, and the latency when using the Marker on the screen is minimal.

Using the Remarkable Paper Pro has been a smooth experience.

When creating new notebooks, you can customize each individually by choosing a paper template. The default option is blank, but you can choose lined, grid, storyboards, dots, music, piano, week planner, and more. With so many different templates, the Paper Pro is quite versatile for all kinds of use cases.

The Paper Pro can also convert your handwritten notes to text with the selection tool. However, since it uses MyScript, the conversion requires a data connection. As long as your writing is somewhat legible, it should be no issue and accurate.

The Remarkable Paper Pro can also be used to read web articles, but you’ll need to use the Read on Remarkable extension for Google Chrome or Microsoft Office. This will also require a Remarkable Cloud account, which is free to set up. But if you want unlimited storage, automatic syncing, and more features, you’ll have to subscribe to Remarkable Connect, which costs around $3 a month.

I created an account after setting up my Paper Pro, and the syncing is pretty reliable. There are Remarkable apps for macOS, Windows, iOS, and Android, so notes taken on your Remarkable device can be viewed and edited on other devices and synced.

However, since the Remarkable Paper Pro isn’t a dedicated e-reader, it’s not the best overall experience when reading articles or documents. You can, as it supports PDF and ePub files for import, but it’s not quite like having a Kindle or other e-reader. Reading is a secondary task on the Remarkable Paper Pro, as it’s primarily a digital notebook.

Like other e-ink devices, the Remarkable Paper Pro has excellent battery life. On a single charge, it should last about two weeks. I’ve had my Paper Pro for at least a week and a half, and it’s still at about 60% battery. To conserve power, the device goes to sleep after 20 minutes of inactivity.

The Remarkable Paper Pro has excellent battery life.

It charges using a USB-C cable. However, I found it to be rather slow at charging. When I first received the tablet, it took over an hour to get enough power to turn it on and set it up.

The Marker is also battery-powered since it’s an active stylus. On a single charge, it should last about two months. However, since you’re likely to have it attached to the Paper Pro anyway, which charges it, I doubt battery life with the Marker will be of any concern.

The Remarkable Paper Pro is the gadget I wish I had when I was going to school. I would have loved to have had a tablet on which I could write or take notes, draw diagrams, or make sketches when needed. This is an excellent device for students or anyone who just wants a distraction-free writing environment.

Having color on the Remarkable Paper Pro can be very useful, and having a lit display is incredibly helpful. It’s barely noticeable during the day, but it makes all the difference in the world at night or in dark environments. And if you get the Marker Plus, having the eraser at the top of the stylus truly replicates that pencil and paper feel.

This is the kind of gadget I wish I had when I was going to school.

However, this is an expensive product. The Paper Pro by itself with the standard Marker costs $579. The Marker Plus adds another $50 to that ($629), and the Type Folio makes it an excellent productivity device, but costs $229, bringing the total to $829.

If you already have a Remarkable 2, you may want to think about if you really want the color e-ink display and the light. Unfortunately, the previous Marker or Marker Plus with the Remarkable 2 can’t be used with the Paper Pro.

But if you don’t already have a Remarkable device, and you’ve been wanting a great digital notebook gadget that eliminates distractions and helps you focus, then the Remarkable Paper Pro is fantastic. Just know that it’s not a full collaborative device and is also not a substitute for an actual e-book reader despite the high price. If you want a combination device for digital note-taking and to use as an e-reader, consider the Amazon Kindle Scribe . But if distraction-free writing, notetaking, etc., is what you’re after, it doesn’t get much better than this.

It's the Wednesday before the official release of the next round of iPhones, which means the first reviews are now coming out for the iPhone 16 and iPhone 16 Pro series. Whether you've already ordered a new phone or are still on the fence, these reviews are always exciting to read in anticipation. Here are some of the highlights. iPhone 16/iPhone 16 Plus iPhone 16 Review: A More Fun Camera, But Not Perfect

This year's regular iPhone lineup comes in fantastic new colors, includes an all-new Camera Control like its iPhone Pro cousins, and features an Action Button for the first time. The iPhone 16 and 16 Plus also have a new A18 chipset with more RAM, macro photography, improved battery life, and more.

Nothing says “great outdoors” and “fitness” like a great smartwatch. Also known as wearables, these are primarily wrist-worn devices that contain a number of health and location-tracking sensors, along with numerous features that are unlocked by pairing your new watch (or fitness ring) with a compatible phone or tablet. We see a lot of smartwatch deals in our daily discount dives, but the following offer caught us by surprise. Right now, you’ll be able to order the 45mm-band version of the Garmin Venu 3 in Slate through Amazon for $400. At full price, this model sells for $450.

Why you should buy the Garmin Venu 3 There are no shortage of smartwatches to pick from, so what makes the Venu 3 so special? For starters, it has a beautiful 1.4-inch AMOLED watch face that achieves solid brightness levels, as well as great colors and contrast. And because Garmin opted to use Corning Gorilla Glass 3, you can guarantee your Venu 3 will be able to withstand the occasional drop or ding. The 45mm band size is also very comfortable and can easily be swapped out, though the Stainless Steel one that comes in the box is quite remarkable!

Apple's new iPhone 16 series is up for preorder already, and after snagging the best iPhone 16 Pro deals, you're probably thinking about how you protect your shiny, expensive investment. You may want the best rugged case, or you could chosoe to be brave and leave it fully unprotected.

Regardless of whether you pick up a case or not, you'll definitely want to buy one of the best Apple iPhone 16 Pro Max screen protectors. The question is which one should you buy? Is one type better than others? Do you go for something that provides minimalist protection for just scratches, or one of the most durable options that will keep your phone protected even if you have butterfingers?

Advertisement

Supported by

How Israel Built a Modern-Day Trojan Horse: Exploding Pagers

The Israeli government did not tamper with the Hezbollah devices that exploded, defense and intelligence officials say. It manufactured them as part of an elaborate ruse.

• Share full article

By Sheera Frenkel Ronen Bergman and Hwaida Saad

The pagers began beeping just after 3:30 in the afternoon in Lebanon on Tuesday, alerting Hezbollah operatives to a message from their leadership in a chorus of chimes, melodies, and buzzes.

But it wasn’t the militants’ leaders. The pagers had been sent by Hezbollah’s archenemy, and within seconds the alerts were followed by the sounds of explosions and cries of pain and panic in streets, shops and homes across Lebanon.

Powered by just a few ounces of an explosive compound concealed within the devices, the blasts sent grown men flying off motorcycles and slamming into walls, according to witnesses and video footage. People out shopping fell to the ground, writhing in agony, smoke snaking from their pockets.

Mohammed Awada, 52, and his son were driving by one man whose pager exploded, he said. “My son went crazy and started to scream when he saw the man’s hand flying away from him,” he said.

By the end of the day, at least a dozen people were dead and more than 2,700 were wounded, many of them maimed. And the following day, 20 more people were killed and hundreds wounded when walkie-talkies in Lebanon also began mysteriously exploding. Some of the dead and wounded were Hezbollah members, but others were not; four of the dead were children.

We are having trouble retrieving the article content.

Please enable JavaScript in your browser settings.

Thank you for your patience while we verify access. If you are in Reader mode please exit and  log into  your Times account, or  subscribe  for all of The Times.

Thank you for your patience while we verify access.

Already a subscriber?  Log in .

Want all of The Times?  Subscribe .

• Israel-Gaza War
• War in Ukraine
• US Election
• Kamala Harris
• Donald Trump
• US & Canada
• UK Politics
• N. Ireland Politics
• Scotland Politics
• Wales Politics
• Latin America
• Middle East
• In Pictures
• BBC InDepth
• Executive Lounge
• Technology of Business
• Women at the Helm
• Future of Business
• Science & Health
• Artificial Intelligence
• AI v the Mind
• Film & TV
• Art & Design
• Entertainment News
• Arts in Motion
• Destinations
• Australia and Pacific
• Caribbean & Bermuda
• Central America
• North America
• South America
• World’s Table
• Culture & Experiences
• The SpeciaList
• Natural Wonders
• Weather & Science
• Climate Solutions
• Sustainable Business
• Green Living

What we know about the Hezbollah device explosions

At least 32 people, including two children, were killed and thousands more injured, many seriously, after communication devices, some used by the armed group Hezbollah, dramatically exploded across Lebanon on Tuesday and Wednesday.

In the latest round of blasts on Wednesday, exploding walkie-talkies killed 20 and injured at least 450 people, according to Lebanon's health ministry.

The explosions occurred in the vicinity of a large crowd that had gathered for the funerals of four victims of Tuesday's simultaneous pager blasts, which killed at least 12 people and injured nearly 3,000.

BBC teams in the city reported chaotic scenes in which ambulances struggled to reach the injured, and locals became suspicious of anyone using a phone.

The explosions deepened unease in Lebanese society, coming a day after the apparently similar, and highly sophisticated attack targeting thousands of pagers used by Hezbollah members.

The militant group blamed its adversary Israel. Israeli officials have so far declined to comment.

Two firms based in Taiwan and Hungary accused in media reports of manufacturing the pagers have both denied responsibility. A Japanese company which apparently makes the walkie-talkies said it stopped producing that model 10 years ago.

Here is what we know so far.

How did the attacks unfold?

The first round of blasts began in Lebanon's capital Beirut and several other areas of the country at about 15:30 local time (13:30 BST) on Tuesday.

Witnesses reported seeing smoke coming from people's pockets, before seeing small explosions that sounded like fireworks and gunshots.

Citing US officials, the New York Times said that the pagers received messages that appeared to be coming from Hezbollah's leadership before detonating. The messages instead appeared to trigger the devices, the outlet reported.

Explosions continued for around an hour after the initial blasts, the Reuters news agency reported.

Soon after, scores of people began arriving at hospitals across Lebanon, with witnesses reporting mass confusion in emergency departments.

Similar scenes played out across the country in another round of blasts on Wednesday, at around 17:00 local time (15:00 BST).

Reports suggest it was walkie-talkies that were blown up, devices that were purchased by Hezbollah five months ago, according to a security source speaking to Reuters news agency.

At least one explosion was close to a funeral being held in Beirut for some of the victims of Tuesday's attack, creating panic among those near the procession.

Twenty people have been killed and at least 450 injured, according to Lebanon's health ministry.

• Follow live updates on this story
• Video captures Beirut skyline as devices explode across city
• From Taiwan to Hungary, a complex picture emerges of the pagers' origins
• No electronic equipment considered safe after Lebanon device attacks
• Hezbollah and its conflict with Israel explained

What do we know about the devices?

Details about the walkie-talkies detonated in Wednesday's explosions are still coming to light.

Footage shot in the aftermath showed destroyed devices bearing the brand Icom, a Japanese company. A statement from the firm describes the IC-V82 model as a handheld radio which was exported to the Middle East from 2004 to 2014 and has not been shipped since then.

Icom said production on that model stopped 10 years ago. The manufacturing of the batteries has also stopped, it says.

The company says it is not possible to confirm whether the IC-V82s that exploded in yesterday's attacks were shipped directly from Icom, or via a distributor. It said any products for overseas markets were sold only to the firm’s authorised distributors.

But the models may not even be from Icom.

Earlier, a sales executive at the US subsidiary of Icom told AP news agency that the devices which exploded in Lebanon appeared to be a knock-off product – adding it was easy to find counterfeit versions of the product online.

The pagers that exploded on Tuesday were a new brand that the group had not used before, one Hezbollah operative told AP. A Lebanese security official told Reuters that around 5,000 pagers were brought into the country about five months ago.

Labels seen on fragments of exploded pagers point to a pager model called the Rugged Pager AR-924. But its Taiwanese manufacturer Gold Apollo has denied any involvement with the explosions. When the BBC visited Gold Apollo on Wednesday local police were searching the company's offices, inspecting documents and questioning staff.

• Taiwan pager maker stunned by link to Lebanon attacks

The founder, Hsu Ching-Kuang, said his company had signed an agreement with a Hungarian-based company - BAC - to manufacture the devices and use his company's name. He added that money transfers from them had been "very strange", without elaborating.

BBC Verify has accessed BAC’s company records, which reveal it was first incorporated in 2022.

Its CEO Cristiana Bársony-Arcidiacono told NBC that she knew nothing about the explosions. “I don’t make the pagers. I am just the intermediate. I think you got it wrong," she said.

The Hungarian government said the company had "no manufacturing or operational site" in the country.

What prompted the pager attack?

Unnamed US and Israeli officials told Axios that detonating the pagers all at once was initially planned as the opening move in an "all-out" offensive against Hezbollah. But in recent days Israel became concerned Hezbollah had become aware of the plan - so they were set off early.

Israeli officials have not commented on the allegations, but most analysts agree that it seems likely it is behind the attack.

Prof Simon Mabon, chair in International Relations at Lancaster University, told the BBC: "We know that Israel has a precedent of using technology to track its target" - but he called the scale of this attack "unprecedented".

Lina Khatib, from the UK-based Chatham House, said the attack suggested that Israel has "deeply" infiltrated Hezbollah's "communications network".

In its statement accusing Israel of being behind the attacks, Hezbollah said it held the country "fully responsible for this criminal aggression that also targeted civilians".

Why does Hezbollah use pagers?

Hezbollah has relied heavily on pagers as a low-tech means of communications to try to evade location-tracking by Israel. Pagers are wireless telecommunications devices that receive and display alphanumeric or voice messages.

They are much harder to track than mobile phones, which have long since been abandoned as simply too vulnerable, as Israel’s assassination of the Hamas bomb-maker Yahya Ayyash demonstrated as long ago as 1996, when his phone exploded in his hand.

In February, Hassan Nasrallah directed Hezbollah fighters to get rid of their phones, saying they had been infiltrated by Israeli intelligence. He told his forces to break, bury or lock their phones in an iron box.

Experts now say the directive, issued during a live televised address, may have forewarned Israeli intelligence operatives that the group would be seeking a new - likely lower tech - method of communications.

What is known about the victims of Tuesday's attack?

A source close to Hezbollah told AFP news agency that two of those killed in Tuesday's attack were the sons of two Hezbollah MPs. They also said the daughter of a Hezbollah member was killed.

Among the injured was Iran's ambassador to Lebanon, Mojtaba Amani. Reports in Iranian media said his injuries were minor.

Hezbollah chief Hassan Nasrallah was not hurt in the explosions, Reuters reported quoting a source.

Lebanese Public Health Minister Firass Abiad said damage to the hands and face made up the majority of injuries.

The victims presenting to emergency rooms were a variety of ages, from the old to the very young, some wearing civilian clothes, he told the BBC's Newshour programme.

Outside of Lebanon, 14 people were injured in similar blasts in neighbouring Syria, according to UK-based campaign group the Syrian Observatory for Human Rights.

Will the Hezbollah-Israel conflict escalate?

Hezbollah is allied with Israel's arch-nemesis in the region, Iran. The group is part of Tehran's Axis of Resistance and has been engaged in a low-level war with Israel for months, frequently exchanging rocket and missile fire across Israel's northern border. Entire communities have been displaced from both sides.

The blasts came just hours after Israel's security cabinet made the safe return of residents to the north of the country an official war goal .

While visiting an Israeli airbase on Wednesday, Defence Minister Yoav Gallant said the country was "opening a new phase in the war" and and the "centre of gravity is shifting to the north through the diversion of resources and forces".

Despite the ongoing tensions, observers say that until now both sides have aimed to contain hostilities without crossing the line into full-scale war. But there are fears that the situation could spiral out of control.

Additional reporting by Frances Mao

Bowen: Tactical triumph for Israel, but Hezbollah won't be deterred

Pager explosions will devastate hezbollah's morale and manpower, what is hezbollah and why is it fighting with israel.