Striata Forecast

New

Leveraging machine learning to forecast the utilization of vaccines and other pharmaceutical products.

Read On
CONTACT

Daren Trudeau
Macro-Eyes
Email

CASE STUDY SOURCE

Macro-Eyes

PROJECT STATUS

Ongoing

IMPLEMENTING PARTNERS

PATH - in country (TZ) MOH facilitation
JSI - in country (TZ) MOH facilitation

FUNDER

Bill and Melinda Gates Foundation
USAID

GEOGRAPHIC SCOPE

Tanzania

Target Users

Health System Manager, Data Services Provider

Enabling Environment Building Blocks

Legislation, Policy, and Compliance, Services and Applications, Infrastructure

Family Planning Program Classification

Supply Chain Management

Introduction

Low-income countries experience poor availability and frequent stock-outs of essential medicines at health facilities. Inaccurate forecasting, gaps in infrastructure, an inefficient supply chain, and affordability of medicines limit access to lifesaving treatments. The introduction of Logistics Management Information Systems (LMIS) to provide timely data on stock and prices has improved medicine availability measurement. However, effectively using this large amount of data in forecasting short- and long-term demand remains a challenge.

Most countries currently use basic statistical demand models that rely on a three-month average to predict the medicine and pharmaceutical commodity utilization for the coming months. However, these models often rely on basic straight-line average consumption data which leads to inaccurate estimates of demand. A more efficient, accurate, and timely way of forecasting essential medicines uses machine learning to rapidly identify hidden patterns in multidimensional data.

About Striata Forecast

Unlike traditional forecasting models, Artificial Intelligence (AI) allows deep analysis of data from multiple sources and makes prediction possible in otherwise opaque places. Using similar datasets found in traditional forecasts, machine learning forecasting combines additional sets of public data and predicts information that would otherwise be unknown. When machine learning programs are supplied with a small amount of national-level utilization data, the accuracy of forecasting increases to provide a near-complete picture of future utilization.

Currently, Tanzania uses a three-month rolling average to forecast the utilization of vaccines. Historical data shows that this method is not optimal and leads to situations of overstock or stock-outs, and consequently emergency orders and/or the accumulation of expired products. [1] AI and, more specifically, machine learning has proven extremely successful at forecasting. One of the specific methods of machine learning that enables a more accurate forecast is to use cross-product training, or historical data of several products at the same time, instead of using historical data for only one product to predict utilization. It is significantly more accurate even if one of the product’s historical data has a substantial number of missing values.

MACRO-EYES STRIATA FORECAST can predict the utilization of vaccines with less than 2 percent error at the national level. This real-time, AI-powered forecasting algorithm learns over time from new data, remaining relevant to changes in health systems. The model uses and learns from publicly available data, health system data (when available), satellite imagery, atmospheric data, images, and natural language inputs (when available). More precise forecasting means less wastage, lower cost, and more lives saved with the same resources. The model has been integrated with the LMIS in several countries including:

Tanzania, with funding from the Bill and Melinda Gates Foundation. MACRO-EYES is working with the Government of Tanzania, PATH, and JSI to create a state-of-the-art, Predictive Supply Chain for Vaccines (PSCV) machine learning model using publicly available, temporally-, and regionally relevant data.

Côte d’Ivoire, where MACRO-EYES was selected as part of USAID’s intelligent forecasting competition as the principal implementation partner to introduce and scale the use of machine learning to improve contraceptive forecasting.

Sierra Leone, with funding from the Bill and Melinda Gates Foundation. MACRO-EYES is working with the Ministry of Health and Sanitation (MOHS), the National Medical Supplies Agency (NMSA), and the Directorate of Pharmaceutical Supplies (DPS) to use intelligent forecasting to improve the supply of essential medicines throughout the country. The initial deployment will be a variety of data sources including data from DHIS2.

Evaluation and Results

The most mature of the three deployments of STRIATA FORECAST is in Tanzania. The forecast has shown that the use of traditional forecasting methods delivers between 43 too many and 43 too few vaccine vials for every 100 vials used. The MACRO-EYES machine learning system can cut errors from 43 to between 1 and 2 vials for every 100 vials used, a 96.4 percent reduction in misallocation.

MACRO-EYES used the Lives Saved Tool (LiST), developed by the Institute for International Programs (IIP) at Johns Hopkins Bloomberg School of Public Health and funded by the Bill & Melinda Gates Foundation, to calculate the new mortality rate with this increased coverage. Assuming a prediction accuracy of 98.5 percent across eight vaccines, we estimate an increase in coverage by 50 percent compared to the LiST tool. This would result in a 14.3 percent reduction in mortality for women aged 15-49 years old and a 0.8 percent reduction for children under five by 2027.

When forecasts are inaccurate, health workers tend to place larger orders to ensure that, if there is a peak in demand, no stockouts will happen. This approach ensures that there is always sufficient stock, but it also results in high wastage. Across an entire system, the burden of over-ordering vaccines can lead to a distorted image of the demand at higher levels of the supply chain. This large swing information distortion is referred to as the “Bullwhip Effect.” [2] Machine learning-based forecasting is more accurate and not only looks at historical order data to make predictions but uses other types of publicly available data to predict the accurate utilization at the facility level.

MACRO-EYES calculated the financial impact of a more accurate forecast for the Arusha region in Tanzania over one full year and found that there would be a 25.8 percent decrease in costs incurred by over-ordering vaccines.

Lessons Learned

  • Public and national data can be combined to gain deep insights into utilization and demand for essential medicines.
  • Machine learning uses existing data and learns even when data quality across datasets is relatively low.
  • Early engagement of Ministries of Health and other key stakeholders is key to success.
  • There is a need for donors to invest in the development and expansion of local capacity to expand the use of AI in low- and middle-income countries.

Conclusion

Health systems have been forecasting demand for essential medicines for decades with varying degrees of accuracy. AI solutions are powerful technologies with demonstrated positive impact across healthcare and other domains. While we have seen an enormous focus on procurement in recent years, never more so than during efforts to vaccinate populations against COVID-19, governments around the world have struggled and continue to struggle to identify where to send supply. AI can provide the data insights to address these barriers to equitable distribution of essential medicines using existing data in real-time. Using AI to predict demand can improve forecasting accuracy, saving both lives and money.

References/Resources

1. Hariharan, R., Sundberg, J., Gallino, G., Schmidt, A., Arenth, D., Sra, S., & Fels, B. (2020). An Interpretable Predictive Model of Vaccine Utilization for Tanzania. Frontiers in artificial intelligence, 3, 559617. https://doi.org/10.3389/frai.2020.559617