Kilkari, Mobile Academy & Mobile Kunji

This case study was originally published in the mHealth Compendium Special Edition 2016, developed by the African Strategies for Health project, implemented by Management Sciences for Health with support from the U.S. Agency for International Development (USAID). Updates to the original case study, submitted by the implementing organization in March 2021, appear in the final section of this case study.

Introduction

India’s maternal and infant mortality rates remain above the Sustainable Development Goal targets. The Government of India is making considerable investments in this area and various innovative approaches and initiatives are evolving to bolster these efforts. Leveraging mobile phone technology (mHealth) to change people’s health-seeking behavior is key among them.

In 2012, BBC Media Action, as a part of the Bill & Melinda Gates Foundation-funded Ananya program, designed, developed, and rolled out three mHealth services to increase the uptake of life-saving preventative maternal and child health behaviors by pregnant women, mothers of children up to two years old, and rural families in Bihar State. Ananya is a collaboration between the Gates Foundation and the Government of Bihar to dramatically reduce maternal and infant mortality in the state.

• Kilkari (a baby’s gurgle) is an outbound service that delivers weekly, time-appropriate audio messages about pregnancy, childbirth, and childcare directly to families’ mobile phones from the second trimester of pregnancy until the child is one year old.
• Mobile Academy is an interactive voice response (IVR) training course designed to refresh community health workers’ (CHWs’) knowledge of simple steps families can take to improve the health of mothers and babies, and to improve their ability to clearly communicate them.
• Mobile Kunji is a free audio-visual job aid that brings together an IVR mobile service with a printed deck of cards with important health messages. It is designed to improve engagement between CHWs and families, thus leading to healthier behavior by new and expecting mothers.

Given the success of these services, on 15 January 2016 the Government of India launched Kilkari and Mobile Academy nationally to train one million health workers and help nearly 10 million new & expecting mothers make healthier choices.

About Kilkari, Mobile Academy, and Mobile Kunji

Kilkari, Mobile Academy, and Mobile Kunji all use IVR technology that is already being used by the commercial mobile industry to make audio content available to any mobile phone across India. No new handsets, software, or technical skills are required for target groups to use the project’s services, allowing accessibility at scale from the outset. A robust, open source mobile health platform called Mobile Technology for Community Health (MOTECH) powers all three services.

At the national level, both Mobile Academy and Kilkari have been integrated with the government’s Maternal and Child Tracking System (MCTS). Currently 1.3 million pregnant women and mothers across six states who have registered mobile numbers on MCTS are automatically subscribed so that they begin receiving weekly calls. Similarly, only CHWs, who are registered in MCTS can dial into Mobile Academy and access the four-hour training course. They dial a 12-digit for access; bookmarking technology allows them to return to the exact location in the course whenever they like.

Program Design Process

When the three services were first being designed for rollout in Bihar, the goal was to create mHealth services that are scalable across the state and eventually across the country. Bihar alone has a population of 108 million people, including 27 million women of childbearing age, and over 80% of the population is rural with limited access to traditional media (TV and radio). However, 83% of women have access to mobile phones, but most of these are low-end basic phones. The aim was to use low-end technology for high-end gains, sustainably scale the solutions for millions of people, and create services that did not require smartphones, software, or technical skills to use.

The solution was found through Kilkari, Mobile Academy, and Mobile Kunji. All services are handset agnostic and employ easy-to-use familiar IVR technology. Mobile Academy empowers CHWs to access affordable standardized training content on their own terms; Kilkari delivers time-sensitive audio messages directly to the mobile phones of families; and Mobile Kunji gets around the challenge of delivering audio and visual content without distributing expensive hardware.

BBC Media Action began by researching its audiences: CHWs, pregnant women, and mothers of young children and their families. It conducted a landscaping study to understand their mobile ownership and usage patterns and began to conceptualize and design the services. This involved technological design and development, checking health content, and conducting several rounds of user-testing and content pretesting.

A user-centric design approach was employed during development involving feedback and inputs from CHWs. Mobile Kunji, Mobile Academy, and Kilkari (user interface and content) went through four rounds of rigorous user-testing and pretesting with CHWs and families to assess comprehension, usability, and perceived value. The services were modified based on the results of each round of user-testing and then tested again until satisfactory results were achieved.

BBC Media Action collaborated with the Ministry of Health and Family Welfare (MOHFW), the National Health System Resource Centre, and National Health Missions (NHMs) in the states and experts in the field of reproductive, maternal, newborn, child, and adolescent health (RMNCH+A) to compile the technical health content for all three services. Before services were launched, the technical health content went through a rigorous vetting and approval process at MOHFW. This ensured that health content was aligned with national guidelines. In addition, a plan for periodically updating content was developed to ensure that it remains accurate and relevant to the audiences.

Growth in Scale

The rollout and scale-up of the three services in individual states has involved working with the MOHFW at the national level, the NHMs at the national and state level, and an alliance of donors (Gates Foundation, USAID, Barr Foundation, and UK Department for International Development (UKAid).

In May 2012, BBC Media Action launched Mobile Academy and Mobile Kunji in Bihar as part of the Ananya program. In August 2013, BBC Media Action then launched Kilkari in Bihar under the same program. In 2014, Mobile Academy and Mobile Kunji were launched in Odisha with funding from UKAid and the state government, and in 2015, with funding from the Gates Foundation and the state government, the two services (Mobile Kunji and Mobile Academy) were launched in Uttar Pradesh.

In 2014, the MOHFW expressed interest in rolling out Mobile Academy and Kilkari nationally. At that time, BBC Media Action and the Gates Foundation began working with MOHFW to take the services to scale. The national launch of Mobile Academy and Kilkari by the MOHFW took place in January 2016. The services will be rolled-out in three phases to train 1 million community health workers and help nearly 10 million new and expecting mothers make healthier choices and lead longer, healthier lives. In phase one of the national scale-up, Kilkari was rolled out in six Empowered Action Group states (Jhakrhand, Madhya Pradesh, Odisha, Rajasthan, Uttarakhand, Uttar Pradesh) and Mobile Academy was rolled out in Jhakrhand, Madhya Pradesh, Rajas- than, and Uttarakhand. In phase two, the two services will be scaled to an additional three states (planned for 2017); in phase three, the services will be scaled across the country (planned for 2018 onwards). In 2015, the Barr Foundation and USAID joined the national scale-up effort. BBC Media Action has worked in partnership with the Grameen Foundation since 2011 to scale MOTECH to power the services. MOTECH continues to be the engine for the national deployment. BBC Media Action, in partnership with state governments, is also scaling Mobile Kunji in Bihar, Odisha, and Uttar Pradesh, and rollout is planned in Jharkhand later in 2016.

Evaluation and Results

Kilkari is being rolled out in six states currently to reach approximately one million families, with a goal to rapidly scale to reach 10 million families a year. Mobile Academy is now being rolled out to 432,000 health workers in Bihar, Jharkhand, Madhya Pradesh, Odisha, Rajasthan, Uttarakhand, and Uttar Pradesh. The MOHFW plans to scale Mobile Academy to reach nearly a million health workers over the next three years. Mobile Kunji is being rolled out on a state-by-state basis to more than 282,000 CHWs in Bihar, Odisha, and Uttar Pradesh, with plans to scale to Jharkhand soon.

The results of a usage and engagement study, commissioned by BBC Media Action and carried out by the Indian Market Research Bureau in Bihar in 2014, indicate that exposure to Mobile Kunji is positively associated with: significant increases in the knowledge, confidence, and credibility of CHWs; positive significant difference in the quality of CHWs’ interactions with new and expecting mothers; positive significant difference in levels of knowledge of life-saving preventative health behaviors among new and expecting mothers; and positive significant difference in adoption of life-saving preventative health behaviors by new and expecting mothers. A limited evaluation of Mobile Academy was also carried out as part of the usage and engagement study. It showed that Mobile Academy had a positive impact on the knowledge and confidence of CHWs, improved their standing in the community, and resulted in recognition from their peers.

Mobile usage data provides real-time information about Kilkari usage patterns by beneficiaries as well as CHWs’ progress through Mobile Academy and which Mobile Kunji content has been played to families (in what volume and with what frequency). Tracking these indicators over time provides key insights that enable on-ground project staff and government officials to make timely decisions about implementation and planning.

Indicators to monitor the usage of Mobile Academy include the uptake of the service by CHWs—this includes the number of unique users, the minutes consumed, progression and competition by CHWs in the four-hour course. This data can be aggregated for each block (smallest administrative unit in a state), district, and state. Project staff and government officials at these levels can monitor this data and use it to support and encourage CHWs in low-performing blocks to complete the course and help them if they have run into difficulties.

The indicators to assess the uptake of Kilkari include the number of users called by the service, the number of successful calls, minutes of content heard. The data generated can be aggregated, analyzed, and used by project and government staff to understand why the duration of messages listened to in one block, district, or state is lower than another.

Indicators for Mobile Kunji include the number of CHWs using the service, how many minutes heard, and specific health information accessed. This data can be used by project and government staff to understand why CHWs are not dis- cussing a particular behavior. Based on this information, field staff can provide additional support in the form of training or mentoring.

Lessons Learned

• Demonstrate sustained adoption at scale, as well as health impact. One of the biggest challenges that state and national governments face is in scaling mHealth solutions beyond pilot stage. It is necessary to plan for growth early to figure out the significant financial, technical, operational, and legal challenges involved in scaling.
• Take the path of least resistance. Modify the approach to reduce government procurement challenges. In India, as in many other countries, the government must first go through a procurement process to adopt a technology, product, or service. This can be a time consuming process. To help avert delays, BBC Media Action set up a toll free centralized long code platform, where shortcodes and integration with multiple operators’ billing systems were not required, and government only needed to contract one operator via a standard competitive process.
• To generate wider interest, demonstrate demand for services not just from users, but from state governments. The national government’s interest in BBC Media Action’s services grew after it scaled beyond Bihar. The launch in Odisha with support from UKAid and the state government in February 2014, and further program growth in other states drew attention at the national level.
• Be prepared to cope with the challenges of scaling and supporting open source software. Open source software is used to power Kilkari, Mobile Academy, and Mobile Kunji because by investing in an open source mobile health platform, the financial hurdles that the government faces in adopting the system are limited. Essentially, it reduced the total cost of government ownership. However, when deciding to use open source software, it is critical for the development community to acknowledge the risk in using open source technology. Firstly, open source software may be created by organizations that might not have the capacity required to support the software to industry standard service level agreements after launch at scale. In addition, although open source technology can be adoptable by anyone, in many cases the people who know best how to fix or further develop an application are those who built it.
• Localize content, test that it works, and keep technical information relevant and up-to-date. Understand who the target audience is and ensure that content is relevant, understandable, and engaging. It’s vital that the technical health content is reviewed periodically and updated as required. It’s best to set up a process that ensures that key stakeholders review and update content periodically.
• Content licensing protects quality. BBC Media Action is licensing its mobile health content, developed with funding from the Gates Foundation, DFID, and the Barr Foundation, to the government of India at no cost. However, the content is not open source—it cannot be downloadable by anyone under a Creative Commons license. BBC Media Action licenses its health content—usually royalty free—on a case-by-case basis to ensure that it remains factually accurate, complies with government policies and guidelines, is used in the context it was designed for, and is being used with the consent of the people featured.

Future Plans

National launch of Mobile Academy and Kilkari by the national government took place in January 2016. The services will be rolled-out in three phases, adding states in phase one and two during 2016 and 2017 dependent on feedback from users and successful adoption of the services in the phase one states. In 2018, there are plans to begin scaling the two services to the rest of the country, reaching a total of 1 million CHWs and 10 million pregnant women and mothers of children under one year of age. BBC Media Action, in partnership with state governments, is also scaling Mobile Kunji in Bihar, Odisha, and Uttar Pradesh, and rollout is planned in Jharkhand later in 2016.

The Gates Foundation’s investment has covered the cost of scaling the software for the two services (including MOTECH) so it can handle calls to and from 35 states, and to support software for a period of three years (through December 2018). Support from USAID and the Gates Foundation contributes to the national scale-up effort over the next two and a half years. Further investment is required for developing different language versions of the two services, to carry out monitoring and evaluation of the project, and to document the five-year learning journey from research and development to national scale-up. BBC Media Action is currently working to design impact-evaluation studies and a cost-benefit analysis of all three services to assess the value of delivering free health education to millions of beneficiaries and health workers.

Project Updates

Mobile Academy and Kilkari 

Mobile Academy and Kilkari continue to be run by India’s Ministry of Health and Family Welfare and are available in 13 states across the country.  

As described in the original case study above, in 2016, Mobile Academy and Kilkari were adopted by India’s Ministry of Health and Family Welfare and were scaled up to multiple states. Although the original plan was for BBC Media Action to support the ministry in setting up, running, and managing the services in nine states, by early 2018, the services had already scaled to 13 states.

By early 2019, when BBC Media Action exited the project and transitioned the services to the Ministry of Health and Family Welfare, Kilkari had reached 10 million users, and 206,000 community health workers had successfully completed Mobile Academy. The services continue to be run and managed by the ministry and its partners. 

For the national scale-up of these services, additional partners joined BBC Media Action to effectively design and deliver the services. These included implementation partners BeeHyv and IMI Mobile; technical partners Reliance Communication, Railtel, Dimagi, and Deloitte; and research partners Purple Audacity, Johns Hopkins University, and the University of Cape Town. 

Mobile Kunji 

Mobile Kunji is no longer active in any of the states in which it was introduced due to funding gaps. Between 2012 and 2015, Mobile Kunji was launched in three states: Bihar, Odisha, and Uttar Pradesh (as described in the ASH mHealth Compendium from 2016). At the end of 2015, there were significant changes made to the way the UK Department for International Development (UKAid) was funding programs in India. This meant there was no longer financial support for the roll-out of the service in Odisha, and the service was closed in December 2015, having reached just under 10,000 unique users who accessed over a million minutes of content. 

Starting in 2015, the government of Uttar Pradesh paid for most of the costs of rolling out and implementing Mobile Kunji in Uttar Pradesh. The service ran across the state until mid-2017, when, again, the service was closed because of a lack of committed funding from Uttar Pradesh’s government to keep the services running. By the time the service closed in Uttar Pradesh, it had reached over 70,000 users who had consumed over 2.4 million minutes of content. 

Mobile Kunji ran the longest in Bihar, from 2012 to 2020. The service was transitioned to the government of Bihar in May 2019, when BBC Media Action exited the program entirely. It is our understanding, however, that the project was not sustained beyond October 2020, when the committed government funding expired. By the time that BBC Media Action exited the program, transitioning it to the government of Bihar, Mobile Kunji had been used by 500,000 unique users who had accessed a cumulative 60 million minutes of audio content.

Project Evaluation

In December 2020, the Stanford University School of Medicine authored a collection of learning papers on the Ananya project in Bihar, which was published in the Journal of Global Health. One of those papers—on the impact of Mobile Kunji—is based on an analysis of three survey-based data sets, collected separately by Mathematica Policy Research, CARE India, and BBC Media Action. The analysis compared the health-related knowledge, attitudes, and behaviors among childbearing women exposed to Mobile Kunji during their visits with community health workers compared with those who were not exposed.

The Mathematica Policy Research study (2014) revealed that exposure to Mobile Kunji was associated with significantly higher odds of consuming iron-folic acid tablets as well as taking a set of three measures for delivery preparedness and appropriate infant complementary feeding. CARE India’s Community-based Household Surveys (2012–2017) demonstrated significant improvements in both early and exclusive breastfeeding in addition to birth preparedness practices. BBC Media Action’s Usage & Engagement Survey (2014) demonstrated a positive association between exposure to Mobile Kunji and exclusive breastfeeding as well as maternal respondents’ trust in their frontline worker. The paper concludes that “Significant improvements in RMNCHN-related knowledge and behaviours were observed for Bihari women who were exposed to Mobile Kunji and Dr. Anita. This analysis is unique in its rigorous evaluation across multiple data sets of mHealth interventions implemented at scale. These results can help inform global understanding of how best to use mHealth tools, for whom, and in what contexts.”

The paper can be found here.

This paper, published in the Stanford Social Innovation Review, focuses on the scale-up of mHealth services.

In 2017, the Bill & Melinda Gates Foundation commissioned Johns Hopkins University’s Bloomberg School of Public Health and the University of Cape Town to conduct a rigorous evaluation of Kilkari and Mobile Academy. Learnings around exposure to Kilkari show how frequent SIM change and the gendered dynamics of household phone use impeded our ability to consistently reach women over time. Findings suggest that if women (and their partners) receive and pick up the calls, they remain loyal listeners to the program over time. Exposure to Kilkari messages was associated with the increased use of reversible modern contraceptives, as well as other impacts, such as an increase in the immunization of children at 10 weeks of age. Detailed results have been submitted for publication and should be available later in 2021.

Early lessons from scaling mHealth services nationwide were published by the Digital Impact Alliance (DIAL) as a part of a free online guide for nongovernment organizations wanting to scale their digital solutions. Titled “Beyond Scale,” DIAL partnered with BBC Media Action, Esoko, Vital Wave, and Johns Hopkins University’s mHealth Initiative to develop this guide. 

References/Resources

Original Case Study

Yvonne MacPherson and Sara Chamberlain. “Health on the Move: Can Mobile Phone’s Save Lives?” February 2013. http://www.bbc.co.uk/mediaaction/publications-and-resources/policy/briefings/asia/india/policy_mhealth

Sara Chamberlain. “A Mobile Guide Toward Better Health.” Innovations. MIT Press. 2014. http://mitpressjournals.org/userimages/ContentEditor/1415302178306/INNOVATIONS_DIGITAL-INCLUSION.pdf

Video: Design thinking behind Mobile Kunji and Mobile Academy: http://youtu.be/IUIDhU_Zjcc

Video: Impact of Mobile Kunji and Mobile Academy: http://youtu.be/KtI-XgCuhTg

Priyanka Dutt. Six Lessons I learnt while Trying to Reach 10 Million Women in India with Life-saving Health Information. April 2016. http://blogs.worldbank.org/publicsphere/blog-post-month-six-lessons-i-learnt-while-trying-reach-10-million-women-india-life-saving-health

Bihar project website: http://www.rethink1000days.org

1. Impact of Audio-Visual Job Aid on Influencing Family Health Outcomes in Bihar: Findings from the Usage and Engagement study on Mobile Kunji. http://www.rethink1000days.org/wp-content/uploads/2016/02/Detailed-presentation_-study-on-effectiveness-of-Mobile-Kunji.pdf

2. Priyanka Dutt. “Technology saving millions of lives.” Huffington Post. January 2016. http://www.huffingtonpost.co.uk/priyanka-dutt/technologysaving-millions-of-lives_b_9058016.html

Project Updates

Ward, Victoria C., Hina Raheel, Yingjie Weng, Kala M. Mehta, Priyanka Dutt, Radharani Mitra, Padmapriya Sastry et al. “Impact of mHealth interventions for reproductive, maternal, newborn and child health and nutrition at scale: BBC Media Action and the Ananya program in Bihar, India.” Journal of Global Health 10, no. 2 (2020).

Pepper, K. T., J. Schooley, S. Chamberlain, I. Chaudhuri, S. Srikantiah, and G. L. Darmstadt. “Scaling health coverage, quality, and innovation through the public sector.” (2019).

Rep. Beyond Scale: How to Make Your Digital Development Program Sustainable. The Digital Impact Alliance, December 1, 2017. https://digitalimpactalliance.org/research/beyond-scale-how-to-make-your-digital-development-program-sustainable/.

Rethink 1000 Days (website), BBC Media Action, https://www.rethink1000days.org/home/.

The site now contains updated information, resources, dissemination materials, and reports on Kilkari and Mobile Academy.