Striata Forecast
New
Leveraging machine learning to forecast the utilization of vaccines and other pharmaceutical products.
As low- and middle-income countries transition from paper to digital systems, family planning programs can benefit from unprecedented opportunities to improve services. Investments in digital health tools have expanded exponentially, but information on what works—and what does not— remains limited and scattered. As investments have increased, digital applications and data fragmentation have proliferated, but stakeholders are moving towards more coordinated efforts to scale digital health solutions, support countries’ digital health infrastructure, and share evidence-based learnings.
This Digital Health Compendium enables users to explore case studies across a range of digital health technologies used to enhance family planning programs mainly in sub-Saharan Africa, but also in other regions of the world. Digital health applications in family planning programs can be broadly classified as those affecting demand generation, service delivery, supply chain management, and the policy and enabling environment. In many low- and middle-income countries, digital health innovations were adopted earlier in other health sectors, including HIV/AIDS, maternal and child health, and noncommunicable disease prevention and response. As a result, much of the impact evidence is likewise restricted to those sectors. To advance greater adoption of digital technology in family planning programs, more data and information on the challenges, opportunities, scalability, and results are needed. This compendium aims to consolidate emerging information and data on applications of digital technology in family planning programs to inform adoption and scale-up of successful approaches.
All of the case studies were submitted by the implementing organizations and include a description of the digital health intervention, program context, and, if available, important findings and lessons learned through rigorous evaluations or program data. The compendium facilitates a quick search for case studies based on the target user for digital health intervention, building block for the digital health enabling environment, family planning program classification, and country location. The case studies give policy and program decisionmakers insights on real-world applications of digital health, promising practices, challenges, and other lessons that can be applied to current and future programs.
New
Leveraging machine learning to forecast the utilization of vaccines and other pharmaceutical products.
Daren Trudeau
Macro-Eyes
Email
Macro-Eyes
Ongoing
PATH - in country (TZ) MOH facilitation
JSI - in country (TZ) MOH facilitation
Bill and Melinda Gates Foundation
USAID
Tanzania
Health System Manager, Data Services Provider
Legislation, Policy, and Compliance, Services and Applications, Infrastructure
Supply Chain Management
Low-income countries experience poor availability and frequent stock-outs of essential medicines at health facilities. Inaccurate forecasting, gaps in infrastructure, an inefficient supply chain, and affordability of medicines limit access to lifesaving treatments. The introduction of Logistics Management Information Systems (LMIS) to provide timely data on stock and prices has improved medicine availability measurement. However, effectively using this large amount of data in forecasting short- and long-term demand remains a challenge.
Most countries currently use basic statistical demand models that rely on a three-month average to predict the medicine and pharmaceutical commodity utilization for the coming months. However, these models often rely on basic straight-line average consumption data which leads to inaccurate estimates of demand. A more efficient, accurate, and timely way of forecasting essential medicines uses machine learning to rapidly identify hidden patterns in multidimensional data.
Unlike traditional forecasting models, Artificial Intelligence (AI) allows deep analysis of data from multiple sources and makes prediction possible in otherwise opaque places. Using similar datasets found in traditional forecasts, machine learning forecasting combines additional sets of public data and predicts information that would otherwise be unknown. When machine learning programs are supplied with a small amount of national-level utilization data, the accuracy of forecasting increases to provide a near-complete picture of future utilization.
Currently, Tanzania uses a three-month rolling average to forecast the utilization of vaccines. Historical data shows that this method is not optimal and leads to situations of overstock or stock-outs, and consequently emergency orders and/or the accumulation of expired products. [1] AI and, more specifically, machine learning has proven extremely successful at forecasting. One of the specific methods of machine learning that enables a more accurate forecast is to use cross-product training, or historical data of several products at the same time, instead of using historical data for only one product to predict utilization. It is significantly more accurate even if one of the product’s historical data has a substantial number of missing values.
MACRO-EYES STRIATA FORECAST can predict the utilization of vaccines with less than 2 percent error at the national level. This real-time, AI-powered forecasting algorithm learns over time from new data, remaining relevant to changes in health systems. The model uses and learns from publicly available data, health system data (when available), satellite imagery, atmospheric data, images, and natural language inputs (when available). More precise forecasting means less wastage, lower cost, and more lives saved with the same resources. The model has been integrated with the LMIS in several countries including:
Tanzania, with funding from the Bill and Melinda Gates Foundation. MACRO-EYES is working with the Government of Tanzania, PATH, and JSI to create a state-of-the-art, Predictive Supply Chain for Vaccines (PSCV) machine learning model using publicly available, temporally-, and regionally relevant data.
Côte d’Ivoire, where MACRO-EYES was selected as part of USAID’s intelligent forecasting competition as the principal implementation partner to introduce and scale the use of machine learning to improve contraceptive forecasting.
Sierra Leone, with funding from the Bill and Melinda Gates Foundation. MACRO-EYES is working with the Ministry of Health and Sanitation (MOHS), the National Medical Supplies Agency (NMSA), and the Directorate of Pharmaceutical Supplies (DPS) to use intelligent forecasting to improve the supply of essential medicines throughout the country. The initial deployment will be a variety of data sources including data from DHIS2.
The most mature of the three deployments of STRIATA FORECAST is in Tanzania. The forecast has shown that the use of traditional forecasting methods delivers between 43 too many and 43 too few vaccine vials for every 100 vials used. The MACRO-EYES machine learning system can cut errors from 43 to between 1 and 2 vials for every 100 vials used, a 96.4 percent reduction in misallocation.
MACRO-EYES used the Lives Saved Tool (LiST), developed by the Institute for International Programs (IIP) at Johns Hopkins Bloomberg School of Public Health and funded by the Bill & Melinda Gates Foundation, to calculate the new mortality rate with this increased coverage. Assuming a prediction accuracy of 98.5 percent across eight vaccines, we estimate an increase in coverage by 50 percent compared to the LiST tool. This would result in a 14.3 percent reduction in mortality for women aged 15-49 years old and a 0.8 percent reduction for children under five by 2027.
When forecasts are inaccurate, health workers tend to place larger orders to ensure that, if there is a peak in demand, no stockouts will happen. This approach ensures that there is always sufficient stock, but it also results in high wastage. Across an entire system, the burden of over-ordering vaccines can lead to a distorted image of the demand at higher levels of the supply chain. This large swing information distortion is referred to as the “Bullwhip Effect.” [2] Machine learning-based forecasting is more accurate and not only looks at historical order data to make predictions but uses other types of publicly available data to predict the accurate utilization at the facility level.
MACRO-EYES calculated the financial impact of a more accurate forecast for the Arusha region in Tanzania over one full year and found that there would be a 25.8 percent decrease in costs incurred by over-ordering vaccines.
Health systems have been forecasting demand for essential medicines for decades with varying degrees of accuracy. AI solutions are powerful technologies with demonstrated positive impact across healthcare and other domains. While we have seen an enormous focus on procurement in recent years, never more so than during efforts to vaccinate populations against COVID-19, governments around the world have struggled and continue to struggle to identify where to send supply. AI can provide the data insights to address these barriers to equitable distribution of essential medicines using existing data in real-time. Using AI to predict demand can improve forecasting accuracy, saving both lives and money.
1. Hariharan, R., Sundberg, J., Gallino, G., Schmidt, A., Arenth, D., Sra, S., & Fels, B. (2020). An Interpretable Predictive Model of Vaccine Utilization for Tanzania. Frontiers in artificial intelligence, 3, 559617. https://doi.org/10.3389/frai.2020.559617