Mobiles for Quality Improvement (m4QI) – SHOPS Project

Read On
Contact

Sarah Romorini
Senior Associate
Abt Associates
Email

CASE STUDY SOURCE

UPDATE: March, 2021

mHealth Compendium, Volume 1
Published by:
Management Sciences for Health
Originally published:
November, 2012

Project Status

Inactive


The information below appeared in the original case study.

IMPLEMENTATION PARTNERS

Abt Associates Inc. leads a team of partners that includes:
Banyan Global
Jhpiego
Marie Stopes International
Monitor Group
O’Hanlon Health Consulting

FUNDER

USAID

CONTACT (as Listed in Original Case Study)

Pamela Riley
Senior mHealth Advisor
Strengthening Health Outcomes through the Private Sector (SHOPS)
Abt Associates

GEOGRAPHIC SCOPE

Uganda

Target Users

Health Care Provider

Enabling Environment Building Blocks

Services and Applications

Family Planning Program Classification

Service Delivery

This case study was originally published in the mHealth Compendium Volume 1, developed by the African Strategies for Health project, implemented by Management Sciences for Health with support from the U.S. Agency for International Development (USAID). Updates to the original case study, submitted by the implementing organization in March 2021, appear in the final section of this case study.

Introduction

Many developing countries have a severe shortage of health providers, and many of the providers who are working have only limited access to up-to-date clinical protocols, or face-to-face trainings. Mobile phones offer an innovative channel through which to provide cost-effective approaches for clinical training and support for improving quality of care.

SHOPS’ partners Abt Associates, Jhpiego, and Marie Stopes International (MSI), collaborated in a mobile learning and performance support pilot called Mobiles for Quality Improvement (m4QI) conducted in Uganda during the period September 2010 – August 2011. The goal of m4QI was to demonstrate the potential for positive behavioral change in service delivery by reinforcing face- to-face induction training lessons provided to Marie Stopes staff. Research supports the theory that spaced reinforcement of training combined with testing can significantly improve long-term knowledge retention and facilitate behavioral change.

The objectives of m4QI were to develop and test a technology-supported approach to performance improvement including processes for identifying performance gaps in adherence to clinical protocols, a platform to manage and automate the delivery and receipt of text message reminders and quizzes to address the gaps, and production of actionable data to improve effectiveness of supportive supervision and follow-up. To support scalability and replicability, the pilot platform was designed for users of low-end phones, and those without Internet access.

About m4QI

The pilot was conducted with 34 family planning staff working in six geographically dispersed service delivery sites which included three MSU Health Centers and three MSU Outreach Teams that offer family planning services.

A Uganda software development organization, Appfrica, was selected to develop FrontlineSMS: Learn, which allows for the delivery and receipt of text messages of daily instructions, tips, and quizzes to target behaviors related to infection prevention, client care, and adherence to standards and guidelines. Adapted from the FrontlineSMS platform, this open source application is intended to work in environments with low-end phones and no access to internet. The text message software used in the pilot program is freely available from FrontlineSMS: Learn.

The m4QI platform was hosted and managed by staff of MSU‘s research department, who were responsible for locating a computer, acquiring a modem and SIM card to attach to the computer, downloading and installing the FrontlineSMS: Learn software, entering the participants phone numbers, adding messages and scheduling their delivery, and monitoring the software operation.

Evaluation and Results

A total of 3,449 messages were sent to project participants, with an 86.5 percent success rate of receipt. Post-pilot interviews with participants suggested high acceptability of text messages for performance improvement, with generally positive comments, though some negative feedback included the lack of message clarity and frustration with technical problems.

A total of 251 incoming messages were received from participants in response to questions delivered, with a decrease in response rates observed while modem issues were being resolved at the beginning of the project as well as when participants were notified toward the end that evaluations were beginning, indicating the project would be ending. The average response rate was 19 percent, with wide variation in response rates by location and by cadre, varying from an average of 11 responses per provider at the most active site to an average of 1.3 messages per provider at the least active site.
Through structured interviews conducted at the end of the pilot, providers reported the following:

• Being motivated by reminders to adhere to hand-washing rules;
• Referring to training manuals when receiving a quiz question about treatment protocols;
• Re-learning steps in instrument sterilization they had forgotten; and
• Using tips about pain management to more closely attend to clients.

The pilot was also described as promoting team learning and further research on text questions, and increased use of training reference manuals and clinical guideline documents.

Lessons Learned

• Technology-supported interventions require dedicated human resources.
• Planning and budgeting for mobile learning initiatives should include broad internal stakeholder input.
• Adequate participant orientation is critical to ensure engagement in text message training interventions.
• A process for prepayment of airtime subsidies is needed when personal phones are used for workplace purposes.

Conclusion

The m4QI pilot produced a process and software tool that can be replicated globally to improve service delivery in low-resource settings. It allows trainers to manage the delivery of reinforcement and assessment messages to providers, and to make data-driven programmatic decisions for supportive supervision and follow-up training. The results of the m4QI pilot regarding self-reported behavior change support expanded applications with larger-scale populations, in various countries, across a wide range of provider training needs.

PROJECT UPDATES

M4QI Uganda is no longer active. However, inputs from the project have informed other quality improvement apps. 

M4QI Uganda was a modest study in 2011, with a small sample size that tested whether text messages can contribute to changed behaviors. This was an under-studied question at the time, as limited digital solutions were available. The M4QI initiative was a beta test of a frontline SMS plug-in developed to allow for advance scheduling of messages and quizzes. In 2011, this was not an off-the-shelf function and there were limited solutions that offered these features. After the project ended, use of M4QI also ended, and the lessons learned were applied to other quality improvement initiatives. These early pilots helped lay the groundwork for the principles of digital development that now guide this space: strategic, holistic planning and governance and increased in-country ownership of solutions. The more recent SHOPS Plus Family Planning chatbot in India is a testament to this.

References

Riley, Pamela, “M4QI Uganda Project” (presentation slides), International Conference Family Planning, November 15, 2013),  

https://www.shopsplusproject.org/sites/default/files/resources/SHOPS%20m4QI%20mHWG%20Panel%20ICFP%20Slides.pdf.

 - 
Arabic
 - 
ar
Bengali
 - 
bn
German
 - 
de
English
 - 
en
French
 - 
fr
Hindi
 - 
hi
Indonesian
 - 
id
Portuguese
 - 
pt
Russian
 - 
ru
Spanish
 - 
es