Mobiles for Quality Improvement (m4QI) – SHOPS Project
As low- and middle-income countries transition from paper to digital systems, family planning programs can benefit from unprecedented opportunities to improve services. Investments in digital health tools have expanded exponentially, but information on what worksāand what does notā remains limited and scattered. As investments have increased, digital applications and data fragmentation have proliferated, but stakeholders are moving towards more coordinated efforts to scale digital health solutions, support countriesā digital health infrastructure, and share evidence-based learnings.
This Digital Health Compendium enables users to explore case studies across a range of digital health technologies used to enhance family planning programs mainly in sub-Saharan Africa, but also in other regions of the world. Digital health applications in family planning programs can be broadly classified as those affecting demand generation, service delivery, supply chain management, and the policy and enabling environment. In many low- and middle-income countries, digital health innovations were adopted earlier in other health sectors, including HIV/AIDS, maternal and child health, and noncommunicable disease prevention and response. As a result, much of the impact evidence is likewise restricted to those sectors. To advance greater adoption of digital technology in family planning programs, more data and information on the challenges, opportunities, scalability, and results are needed. This compendium aims to consolidate emerging information and data on applications of digital technology in family planning programs to inform adoption and scale-up of successful approaches.
All of the case studies were submitted by the implementing organizations and include a description of the digital health intervention, program context, and, if available, important findings and lessons learned through rigorous evaluations or program data. The compendium facilitates a quick search for case studies based on the target user for digital health intervention, building block for the digital health enabling environment, family planning program classification, and country location. The case studies give policy and program decisionmakers insights on real-world applications of digital health, promising practices, challenges, and other lessons that can be applied to current and future programs.
Sarah Romorini
Senior Associate
Abt Associates
Email
Inactive
The information below appeared in the original case study.
Abt Associates Inc. leads a team of partners that includes:
Banyan Global
Jhpiego
Marie Stopes International
Monitor Group
OāHanlon Health Consulting
USAID
Pamela Riley
Senior mHealth Advisor
Strengthening Health Outcomes through the Private Sector (SHOPS)
Abt Associates
Uganda
Health Care Provider
Services and Applications
Service Delivery
This case study was originally published in the mHealth Compendium Volume 1, developed by the African Strategies for Health project, implemented by Management Sciences for Health with support from the U.S. Agency for International Development (USAID). Updates to the original case study, submitted by the implementing organization in March 2021, appear in the final section of this case study.
Many developing countries have a severe shortage of health providers, and many of the providers who are working have only limited access to up-to-date clinical protocols, or face-to-face trainings. Mobile phones offer an innovative channel through which to provide cost-effective approaches for clinical training and support for improving quality of care.
SHOPSā partners Abt Associates, Jhpiego, and Marie Stopes International (MSI), collaborated in a mobile learning and performance support pilot called Mobiles for Quality Improvement (m4QI) conducted in Uganda during the period September 2010 ā August 2011. The goal of m4QI was to demonstrate the potential for positive behavioral change in service delivery by reinforcing face- to-face induction training lessons provided to Marie Stopes staff. Research supports the theory that spaced reinforcement of training combined with testing can significantly improve long-term knowledge retention and facilitate behavioral change.
The objectives of m4QI were to develop and test a technology-supported approach to performance improvement including processes for identifying performance gaps in adherence to clinical protocols, a platform to manage and automate the delivery and receipt of text message reminders and quizzes to address the gaps, and production of actionable data to improve effectiveness of supportive supervision and follow-up. To support scalability and replicability, the pilot platform was designed for users of low-end phones, and those without Internet access.
The pilot was conducted with 34 family planning staff working in six geographically dispersed service delivery sites which included three MSU Health Centers and three MSU Outreach Teams that offer family planning services.
A Uganda software development organization, Appfrica, was selected to develop FrontlineSMS: Learn, which allows for the delivery and receipt of text messages of daily instructions, tips, and quizzes to target behaviors related to infection prevention, client care, and adherence to standards and guidelines. Adapted from the FrontlineSMS platform, this open source application is intended to work in environments with low-end phones and no access to internet. The text message software used in the pilot program is freely available from FrontlineSMS: Learn.
The m4QI platform was hosted and managed by staff of MSUās research department, who were responsible for locating a computer, acquiring a modem and SIM card to attach to the computer, downloading and installing the FrontlineSMS: Learn software, entering the participants phone numbers, adding messages and scheduling their delivery, and monitoring the software operation.
A total of 3,449 messages were sent to project participants, with an 86.5 percent success rate of receipt. Post-pilot interviews with participants suggested high acceptability of text messages for performance improvement, with generally positive comments, though some negative feedback included the lack of message clarity and frustration with technical problems.
A total of 251 incoming messages were received from participants in response to questions delivered, with a decrease in response rates observed while modem issues were being resolved at the beginning of the project as well as when participants were notified toward the end that evaluations were beginning, indicating the project would be ending. The average response rate was 19 percent, with wide variation in response rates by location and by cadre, varying from an average of 11 responses per provider at the most active site to an average of 1.3 messages per provider at the least active site.
Through structured interviews conducted at the end of the pilot, providers reported the following:
ā¢ Being motivated by reminders to adhere to hand-washing rules;
ā¢ Referring to training manuals when receiving a quiz question about treatment protocols;
ā¢ Re-learning steps in instrument sterilization they had forgotten; and
ā¢ Using tips about pain management to more closely attend to clients.
The pilot was also described as promoting team learning and further research on text questions, and increased use of training reference manuals and clinical guideline documents.
ā¢ Technology-supported interventions require dedicated human resources.
ā¢ Planning and budgeting for mobile learning initiatives should include broad internal stakeholder input.
ā¢ Adequate participant orientation is critical to ensure engagement in text message training interventions.
ā¢ A process for prepayment of airtime subsidies is needed when personal phones are used for workplace purposes.
The m4QI pilot produced a process and software tool that can be replicated globally to improve service delivery in low-resource settings. It allows trainers to manage the delivery of reinforcement and assessment messages to providers, and to make data-driven programmatic decisions for supportive supervision and follow-up training. The results of the m4QI pilot regarding self-reported behavior change support expanded applications with larger-scale populations, in various countries, across a wide range of provider training needs.
M4QI Uganda is no longer active. However, inputs from the project have informed other quality improvement apps.Ā
M4QI Uganda was a modest study in 2011, with a small sample size that tested whether text messages can contribute to changed behaviors. This was an under-studied question at the time, as limited digital solutions were available. The M4QI initiative was a beta test of a frontline SMS plug-in developed to allow for advance scheduling of messages and quizzes. In 2011, this was not an off-the-shelf function and there were limited solutions that offered these features. After the project ended, use of M4QI also ended, and the lessons learned were applied to other quality improvement initiatives. These early pilots helped lay the groundwork for the principles of digital development that now guide this space: strategic, holistic planning and governance and increased in-country ownership of solutions. The more recent SHOPS Plus Family Planning chatbot in India is a testament to this.
References
Riley, Pamela, āM4QI Uganda Projectā (presentation slides), International Conference Family Planning, November 15, 2013),Ā Ā