mHEALTH MATTERS:

PEOPLE, MONEY & PERFORMANCE CASE STUDIES FROM AFRICA

Recommended Citation: The PMI Africa Indoor Residual Spraying Project. October 2015. Kolyada, Lena, Elana

Fiekowsky, Beth Brennan, and Keith Mangam. mHealth Matters: People, Money & Performance Case Studies from

Africa. Bethesda, MD, Abt Associates Inc.

Contract: GHN-I-00-09-00013-00

Task Order: AID-OAA-TO-11-00035

Abt Associates Inc. 1 4550 Montgomery Avenue 1 Suite 800 North

1 Bethesda, Maryland 20814 1 T. 301.347.5000 1 F. 301.913.9061

1 www.abtassociates.com

PMI | AFRICA IRS (AIRS) PROJECT

INDOOR RESIDUAL SPRAYING (IRS 2)

TASK ORDER SIX

mHEALTH MATTERS:

PEOPLE, MONEY & PERFORMANCE Case Studies from Africa

III

CONTENTS

Acknowledgments ......................................................................................................................... v

Introduction ..................................................................................................................................vii

Case Studies ................................................................................................................................... 1

Africa: Using Mobile Payments for IRS Workers ............................................................................................... 3

Angola: Using Smartphones for Spray Data Collection .................................................................................... 6

Benin and Nigeria: Job Aids and Task Reminders via SMS ............................................................................... 8

Mali: Efficacy of Mobile Messaging to Prepare for Spray Campaign ............................................................. 10

Senegal: Smartphone-based Supervision for Real-time Management .......................................................... 13

Madagascar: SMS-based Performance Monitoring Tracking for Real-time Management ........................ 15

Way Forward ............................................................................................................................... 18

V

ACKNOWLEDGMENTS

This publication was made possible through funding from the President’s Malaria Initiative to the PMI

Africa Indoor Residual Spraying Project. The authors gratefully acknowledge the role of colleagues from

partner technology firm Dimagi, Inc., which contributed to the design and implementation of some of

the case studies described in the paper. The authors also thank Dr. Nduka Iwuchukwu, AIRS Tanzania

Chief of Party; Pamela Riley, Lead mHealth Advisor at Abt Associates; and Laura McCarty, Senior

Communications Manager for the PMI AIRS project, for their thoughtful review and feedback that

enhanced the quality of the report. Finally, the authors appreciate the time and effort that Linda Moll

provided in editing and Maria Claudia De Valdenebro for bringing this publication into its final form.

VII

INTRODUCTION

The aim of this publication is to demonstrate the potential of mobile-based technology (mHealth) to

improve the quality and efficiency of public health programs, in particular indoor residual spraying (IRS)

programs. To do so, it describes mHealth innovations implemented by the President’s Malaria Initiative

Africa Indoor Residual Spraying (PMI AIRS) Project.

IRS as A Malaria Control Strategy

IRS is the spraying of insecticide on the walls, ceilings, and other indoor resting places of mosquitoes

that transmit malaria. Since 2008, with help from PMI, many African countries have used IRS as a key

strategy to prevent malaria. The PMI AIRS Project, led by Abt Associates, manages spray operations and

logistics in 11 countries where malaria is endemic, and provides enhanced entomological monitoring in

four additional countries. Spray operations are highly complex, requiring meticulous planning at the

national, district, and village levels. This entails conducting geographical reconnaissance to identify work

sites, procuring insecticide and equipment, managing warehouses, and training large numbers of local

staff to spray homes in accordance with environmental and health guidelines. Implementation of spray

operations involves dispatching a massive amount of equipment and materials as well as workers to the

villages and towns targeted for spraying, according to set schedules. Tight spray calendars are driven by

the need to complete IRS prior to the rainy season because rains complicate the logistics of delivering

IRS and increase refusals of

beneficiaries, who must move

household items outside while

their homes are sprayed.

Teams of spray operators may

stay for a few days in the same

remote area to work in the

villages that are difficult to

access on a daily basis if travel

from the main IRS base is not

feasible. To ensure IRS

operations are on schedule,

closely monitored, and

implemented in line with

standard operating procedures,

the PMI AIRS Project

introduced several mHealth

tools.

Photo: Elana Fiekowsky

VIII

mHealth in IRS

Mobile-based technologies can intervene in public health programs at three levels: the

beneficiary/individual level, the service provider level, and the program management level. In addition,

mobile money services provide an opportunity to improve efficiency and accountability of financial flows

in health programs. The PMI AIRS Project had the opportunity to pioneer applications in all four areas.

With each PMI AIRS country program employing from 500 to 3,700 seasonal workers, orchestrating

efficient spray operations is critical to delivering high-quality, cost-effective, and on-time spray campaigns

that reduce the transmission of malaria just prior to or at the onset of the rainy season. The use of new

tools like mobile phone-based short message services (SMS) and supervision helps to do this.

IRS Beneficiary Communications

The PMI AIRS Project attempted to capitalize on the penetration of mobile phones in Mali to obviate the

high costs and difficulties of beneficiary mobilization, traditionally done through door-to-door visits. The

AIRS Mali team piloted a mobile messaging (text and voice) mobilization approach in Koulikoro District,

trying out alternative methods of mobilization to effectively prepare households for IRS, while reducing

the number of mobilizers.

mHealth for IRS Service Providers

mHealth provides a useful channel to enable operations staff to receive real-time information such as

reminders and alerts through mobile phones. Through information sent from project managers to team

leaders and then to spray operators, the Project piloted mass SMS to seasonal personnel working in

Nigeria and Benin. The purpose was to remind them of performance and environmental compliance

requirements and to encourage their continued good work. Managers also send specific SMS alerts and

reminders to various cadres of workers to address particular issues.

mHealth for IRS Program Management

At the management level, technology can be a great support in tracking the implementation, resources,

and quality of IRS program interventions. Supervision is important for maintaining safe and high-quality

spray performance. The PMI AIRS Project had a difficult time addressing quickly the findings from spray

performance supervision when relying on paper-based forms alone. Supervisory staff often suffered

“form fatigue” from the multiple reporting forms they were expected to complete and submit.

Furthermore, use of paper forms led to errors when recording issues, reporting the information on

time, and addressing in a timely way the problems revealed by the data. To improve supervision, the

Project built a mobile application that reports on IRS performance compliance throughout the spray

campaign.

IX

To monitor daily spray progress and coverage, the Project has also been relying on paper-based data

collection and reporting. Project staff spent significant time transporting paper forms from remote

villages to data centers and entering the data into an Excel spreadsheet. Alternatively, a PMI AIRS

operations manager would call each site – that had phone service – to ask for their spray progress and

then enter that data into a spreadsheet. This delayed operations manager’s access to the data and

prevented the operations manager from ordering immediate corrective action to improve the ongoing

campaign. In some cases, a spray team had left a village before it received such instructions. In 2014, the

automated system, in which team leaders send their team’s daily output data to a cloud-based database,

allowed country-level and U.S.-based management teams to analyze the data on spray progress and

order immediate corrective action if needed.

Mobile Payments in IRS

Spray personnel are also able to receive payment through their mobile phones, and use the phones to

manage their financial accounts including money deposit, transfer, and cash out. IRS campaigns typically

employ hundreds to thousands of seasonal workers including spray operators, warehouse managers, and

district coordinators. Distributing payments to so many employees is difficult anywhere, but even more

so in remote spray sites.

A functional banking system is often absent in remote spray areas and, therefore, cannot be used for

paying wages. Prior to mobile money transfers, the PMI AIRS Project had to hire a secure vehicle to

transport project staff who carried bags of cash to many remote operational sites, a costly, time-

consuming, and risky way to deliver seasonal worker payments. Mobile payments solved these problems,

offering a more cost-efficient, fast, and safe – as well as transparent – way to move funds.

The case studies presented below demonstrate how the Project has used mHealth to enhance spray

performance and results at the levels described above by improving communication, supervision, output

monitoring, and financial transactions. Currently, most of the pilots are successfully being incorporated

into the permanent spray campaigns in nine PMI AIRS country programs.

1

CASE STUDIES

3

AFRICA: USING MOBILE PAYMENTS

FOR IRS WORKERS

Background

AIRS country programs employ up to 3,700 seasonal workers for a weeks-long spray campaign each

year. It is challenging to organize bi-weekly payments for such a large number of field-based IRS workers.

Spray locations are mostly rural, often dispersed in remote villages, where the banking system is not well

established or totally absent. AIRS country programs needed cost-efficient, fast, and safe ways to deliver

payments. Initially, in most countries, finance assistants carried large amounts of cash into the field to

pay workers facing the risk of theft, fraudulent activities, or loss of significant project funds. Later on, in

some countries, the programs wired funds to the microfinance institutions based in the spray districts to

organize payments for a small fee. In other countries, the programs contracted with local banks to

disperse the payments based on worker attendance lists. More recently, making mobile payments

through telecom providers became the third - and the most efficient and transparent - option.

Implementation

Since 2013, six AIRS country programs (Ghana,

Madagascar, Mali, Senegal, Zambia, and Zimbabwe)

have been using mobile money payments. In Ghana,

the AIRS country team works with telecom

provider MTN. The project established a contract

with MTN, which included a total amount to be

delivered to the IRS campaign workers. The

provider charged a small (usually 2-3 percent of the

total amount) transfer fee for the service. AIRS

Ghana provided to MTN the mobile money account

numbers of each seasonal worker along with the

amount to be transferred and a transfer schedule. It

also informed the telecom provider of each

worker's location, so that MTN would ensure their

agents had adequate cash at the time of the payment

in case many recipients wanted to cash out some or

all of their earnings. In Madagascar, the AIRS

program paid seasonal workers through the mobile

provider AIRTEL via electronic wallet and a SIM

card, a risk-free and reliable payment system. If a

seasonal worker did not have a mobile money

account, the project members helped him or her to open one.

Some AIRS country programs also began using the mobile payment method with small local vendors

based in the spray districts. These vendors, usually cooks and suppliers of consumables such as milk,

juice, bread, and small IRS supplies, are used to operating in cash and often do not have a bank account.

By introducing them to mobile money, AIRS programs expedited the paperwork and payment processes

required for local procurement and ensured on-time delivery of meals, other food products, and

supplies during spray operations.

Screen of a mobile money account on a phone, Ghana. Photo credit:

Lena Kolyada

4

AIRS team member is verifying with AIRTEL staff lists of payment

recipients, Madagascar.

Photo: Anna Maria Paddack.

Results

All AIRS country programs that tested mobile

money to pay seasonal workers have continued

using this method of financial disbursement.

Specific benefits include the following:

Established regular, on time payments to

seasonal workers, which both motivated the

workers and eliminated the risk of work

stoppages when payments were delayed

Reduced administrative and logistics burden

Significantly reduced the security risk of

sending staff into the field with large amounts

of cash

Significantly reduced the cost of transacting

the payment; for example, in 2013, AIRS

Madagascar realized a 78 percent cost saving

over 2012, when it used finance assistants to

distribute cash payments

Increased transparency, thus minimizing

opportunities for financial fraud.

Lessons Learned

The main lesson learned is to consider expanding this model of payment to all aspects of the

country program whenever appropriate. It is an easy, low-cost and user-friendly approach to

execute payments if all components of the process are intact including availability of required funds

with the mobile money agents and timely transfers to a bank.

Conclusions

As a program management tool, use of mobile money greatly improved operational efficiency, lowered

transaction costs and risks, and enhanced accountability of the program. In addition to improving

program management, the introduction of mobile money to the IRS programs benefited telecom

providers and individuals.

With respect to providers, AIRS country programs helped the mobile network operators to expand the

reach of their platforms and to increase agent locations by using their systems in the remote areas on a

more regular basis.

5

For beneficiaries, the PMI AIRS Project helped expand the growing mobile money culture from urban to

rural residents, thereby increasing opportunities for overall household economic wellbeing. The seasonal

workers could use the electronic funds in various ways, such as paying school fees, buying airtime and

taxi services, and as a safe storage mechanism, thus increasing household savings.1 Further, mobile

money could empower certain household members, women in particular, who have traditionally had

less bargaining power.2 By paying the increasing number of women IRS workers in mobile money,

expansion of the mHealth application can increase their financial abilities and decision-making authority.

For many women it eliminates a third party who controls the money, when accessing the money

through the mobile phones. Direct access to the electronic funds will allow for more space for

independent financial planning, saving and purchasing.

1 Jack, William and Tavneet Suri. January 2011. Mobile Money: The Economics of M-PESA. NBER Working Paper No.

16721. JEL No. O16,O33,O55 Available online at: http://www.nber.org/papers/w16721.pdf Accessed on July 13, 2015. 2 Ibid

6

ANGOLA: USING SMARTPHONES FOR

SPRAY DATA COLLECTION

Background

Mobile applications can increase accuracy of reported data, provide access to real-time data, and

eliminate the costs for paper-based data processing and storage centers, data entry clerks, and data

transport in low-resource settings. For the 2013 IRS campaign, AIRS Angola piloted a mobile data

collection and verification system as an alternative to the traditional, paper-based data collection and

quality assurance tools in Bailundo Capital, a commune in Huambo province, Bailundo Municipality

(approximately 15,000 targeted structures).

Implementation

The AIRS pilot was designed to test the feasibility of mobile data collection and quality assurance for use

in IRS campaigns. For the AIRS Angola IRS campaign in October 2013, it began by providing spray

operators with a smartphone and their team leaders with Android tablets. Using a mobile-based

platform AIRS Angola reduced data entry errors by preprogramming smartphones with electronic

versions of the program’s standard IRS data collection tools. These forms were built with validation

checks to prevent skipping mandatory fields and to reduce data errors commonly found on paper forms.

Team leaders used tablets to immediately validate data submitted electronically in lieu of mass

verification of the entire team’s paper forms

at the end of each day’s spray activities. With

this system, the AIRS Angola M&E and

Operations teams, the chief of party, and the

U.S.-based M&E specialist could access the

data in real-time, identify issues, and

recommend appropriate and speedy

corrective actions.

Initially, AIRS Angola tasked 48 spray

operators to record data on smartphones

from roughly 12,000 households and to

electronically submit the data to the cloud-

based system. Eight team leaders were given

tablets to simultaneously supervise staff

performance and verify data quality. Twenty-

three staff exhibiting poor understanding of

the mobile system due to poor technology

literacy were reassigned to paper-based data

collection within four days of the pilot start-

up.

Photo: Elana Fiekowsky

7

Results

Once the spray operators confirmed that their spray data were uploaded to the server, the team leader

opened the tablet and viewed the verification table that listed the key indicators for their spray teams

for that day. Since the data were available immediately, the AIRS Angola M&E team was able to pull

and/or clean the data for that day and estimate spray progress in the main office in Huambo.

If the application identified a problem in the data, the team leader flagged the spray operator form

electronically, noting the issue in a text comment for the M&E team’s attention and correction. For

example, based on such notifications the M&E team noticed a higher than expected refusal rate in a few

villages. Accordingly, AIRS Angola, in collaboration with the Bailundo Municipal Department of Health,

deployed a mobilization unit the next day to circulate in those and surrounding villages. The unit used a

loudspeaker to disseminate IRS messages in both Portuguese and Umbumdo, to increase awareness and

willingness to participate, and IRS acceptance rates increased in these areas when spray teams returned

to those villages to complete the spraying.

Lessons Learned

Low-literacy workers need more training and hands-on supervision. The team recommends adding

to the three-day training from two to three extra days for these workers.

The pilot occurred in a new IRS area, thus new spray operators were learning both spray skills and

phone data entry skills. In the future, the Project recommends implementing mobile data collection

with personnel experienced in IRS and allocating more time for training.

The spray operators submitted data for all structures sprayed in one day at the end of the day

rather than after each structure, which made data validation difficult. In the future, the system

should allow spray operators to submit data after every structure is sprayed so team leaders can

validate data throughout the day rather than uploading all of the day’s data at once.

Technological “bugs” (i.e., filtering and sorting difficulties) in the team leader verification system

during the first week of spray made data supervision difficult. The PMI AIRS Project recommends

further development of a modified data aggregation system and testing it in advance of

implementation in order to catch similar bugs.

Conclusion

Despite some problems during early stages of implementation, the advantages of shorter delays between

data entry and data analysis were clear. The mobile system successfully allowed the AIRS Angola team to

detect several performance issues and adapt operations to address the problems by the next day.

However, there was no cost analysis conducted to evaluate efficiency of this method as compared to

the paper-based collection. The data verification has increased working hours for team leaders at the

end of the day.

8

A female team leader uses SMS to communicate with the

operations manager, Doma Local Government Area. Photo credit:

Nduku Iwuchukwu

NIGERIA: JOB AIDS AND

TASK REMINDERS VIA SMS

Background

One of the biggest challenges to implementing an IRS campaign is supervision, and supervision is

impossible without good communication. During a spray campaign, PMI AIRS country programs train

and hire on average 1,300 seasonal workers to do various jobs including spray homes, supervise spray

teams, manage insecticide storage, collect data, and wash uniforms. Previously, the most expeditious way

to communicate with the entire field staff was the phone tree. For example, if an operations manager

noticed that spray operators were not correctly wearing protective clothing, he would call team leaders

(up to 50) who would relay the message to their workers during the following morning’s meeting. It was

not an efficient system.

Implementation

In 2013, AIRS Nigeria hired and trained more than 300 seasonal staff for its IRS campaign throughout

two large Local Government Areas. To

tackle this communication and supervision

challenge, it decided to use mass SMS to

reach out to all team leaders and other

categories of staff at once.

The country program enrolled all spray

personnel in a mass SMS platform and began

using text messages to disseminate

reminders, reinforce training, and motivate

staff. In addition to all-staff messages, AIRS

segmented workers by job title and

geographical area, and sent them tailored

messages to improve their performance. If

project technical staff identified a problem,

they could immediately have supervisors

notify the affected staff to point out the

problem and how to correct it. For example,

when technical staff discovered that spray

operators were not using the proper

spraying technique and were applying too

much insecticide, a mass SMS was sent to all

operators to remind them of the correct

behaviors. During follow-up visits, program

supervisors confirmed that workers had

adjusted their spray technique and insecticide

application according to recommendations.

To set up the platform, the program used a local technology firm SMS Africa. It is a commercial

platform, which is open to the public upon payment of a fixed rate for bulk messages. , . The program

9

also collected phone numbers of all hired workers and organized them by job and location. To run the

SMS-based communication smoothly, the country programs had to purchase significant amount of

airtime from the telecom provider.

Over the course of its 33-day IRS campaign, AIRS Nigeria disseminated one or two messages per week.

The messages were about recruiting, the campaign start-up, correcting problems observed in field

operations, and general performance reminders. Toward the end of the campaign, when workers were

becoming fatigued, the messages included words of encouragement and recognition to motivate workers

to finish strong.

Results

While no quantitative data were collected to document the impact of this mHealth innovation, workers

reported that they found the corrections and reminders helpful and informative. Supervisors said the

SMS program eased their supervisory work.

“The text message program made supervision much easier. Once we identified a problem, we could

immediately notify spray operators and make corrections. It has definitely improved our quality of

work,” reported Nduka Iwuchukwu, AIRS Nigeria Operations Manager at the time of the SMS program

implementation.

Lessons Learned

Not all phone owners use the same telecom provider, and each telecom provider has different

service plans and coverage areas. These need to be factored in when procuring airtime from a

telecom provider.

Airtime needed was more extensive and expensive than expected. A country program needs to

plan the number of SMS it expects to send out during the campaign, so that it budgets and

purchases enough airtime for message transmission.

Open-source, low-cost, and more user-friendly platforms such as TextIt are recommended to

replace commercially-provided systems.

Not all SMS go out at the planned time. During periods of high network traffic, the order of SMS

may change, with messages sent later in a day transmitted before the earlier ones.

Conclusion

The use of the SMS platform for communicating reminders and corrections, as well as motivational

messages, to large cohorts of IRS workers promises to improve outcomes. The role of the country

program leadership in ensuring effective resources for adoption and proper integration of this

innovation remains critical to its success. In addition, the use of the SMS platform for reminders and

other messages can be easily adapted for a broader audience of public health implementers.

10

MALI: EFFICACY OF MOBILE MESSAGING

TO PREPARE FOR SPRAY CAMPAIGN

Background

In Mali, residents traditionally have learned about upcoming IRS campaigns through door-to-door

mobilization, in which community field agents (mobilizers) are engaged and trained by the country

program to deliver messages about the campaign. In 2013, over 1,000 mobilizers covered three districts.

Careful supervision of such a large number of mobilizers over a dispersed area was challenging for the

program’s operations manager and monitoring and evaluation (M&E) manager. As a result, mobilizer

success in some areas was discovered to be far inferior to what the mobilizer themselves had reported.

In addition, door-to-door mobilization is very expensive in terms of financial and human resources.

Implementation

To reduce costs while still imparting information to households about what they needed to do both

before and after spraying, AIRS Mali piloted a mobile messaging (text and voice) mobilization approach in

Koulikoro District. The pilot tested alternative methods of mobilization that were intended to

effectively and equally prepare households for IRS, but cost less than the traditional mobilization

approach. Specifically, it assessed the feasibility of replacing door-to-door mobilization with mobile

messaging.

To measure the alternative approaches’ efficacy, the pilot study assessed household preparedness in

three villages in Koulikoro District (Tienfala Village, Tienfala Gare, and Fougadougou) where mobile

messaging was piloted against three comparison villages (Fassa, Wolongotomo Socura, and

Wolongotomo Socoro) that were mobilized solely through a modified door-to-door approach. The

modified approach incorporated radio outreach and town-hall style announcements in addition to

traditional door-to-door communication. Households in all three pilot villages received SMS, in two of

the villages, Tienfala Gare and Fougadougou, households also received voice messages.

The AIRS Mali M&E team went to the three pilot villages to collect phone numbers from 673 residents

in all of the 570 structures targeted for spraying. After collecting the phone numbers, a data entry clerk

uploaded the numbers into the TextIt platform, a software application that allows a single user to send

short SMS to a large number of mobile phones via a computer. Based on the locations of the individuals

whose numbers had been collected, the team determined which language to use when sending messages

to specific groups of beneficiaries as well as group numbers in specific villages, to effectively target

messages.

The AIRS Mali team sent four types of messages to beneficiary households before, during, and after the

IRS campaign. “Mobilization” messages were sent 10 days before the campaign to remind beneficiaries of

the benefits of IRS and to let them know the timing of the spray campaign in their village. “Alert”

messages were sent three days before the start of the campaign to warn them of the dangers of malaria

and how they can protect themselves from malaria, such as accepting IRS. “Instruction” messages were

sent during the spray to tell residents how to prepare themselves and their structures for the spraying,

as well as what to do immediately after spraying. Finally, “Advice” messages were sent a few days after

spraying to inform beneficiaries about signs of insecticide poisoning and other actions to avoid that

would interfere with the efficacy of the insecticide.

11

Results

Before full implementation of the mobile messaging approach began, a mini-pilot was done to test the

SMS and gauge beneficiary perceptions. This exercise showed that the addition of voice messages would

be useful to the overall mobilization approach. Thus, in two of the three pilot villages, voice messages

were sent in conjunction with the SMS. At the end of the full pilot, there were a total of 9,402 SMS sent

out across three villages, and a total of 4,474 voice messages sent out across two villages.

Both mobile pilot and comparison villages had high spray coverage (defined as percentage of found

structures that were sprayed). The initial spray coverage in mobile pilot villages was 86 percent

compared to 96 percent in the villages where traditional door-to-door mobilization was done. Similarly,

structure preparation was lower in villages mobilized via the mobile messaging (47 percent vs. 78

percent).

The most common reason for structures not being sprayed in mobile messaging mobilization areas was

household’s “refusal,” not

necessarily because they

opposed IRS but because they

did not want to miss a day in

their farm field or spend time

removing all of their belongings

from the structure. For

beneficiaries who were hesitant

about having their structures

sprayed, it was much easier to

ignore the mobile message than

a door-to-door mobilizer on the

day of spray. In person, the

mobilizer can work directly with

the beneficiary or village leader

to gain acceptance.

In this pilot, the mobile

messaging mobilization approach

was both more costly and less

efficient than the door-to-door

approach. As the table shows, the

average percentage of structures

prepared upon first visit by spray teams in the mobile messaging-mobilized villages was 47 percent

compared to 78 percent in villages mobilized door-to-door. The average cost incurred per structure

prepared (meaning the structure had been mobilized and was ready to receive the IRS team on the day

of spray) was $6.55 per structure in the mobile messaging villages compared to $1.08/structure in the

villages mobilized door-to-door.

Photo: Elana Fiekowsky

12

Lessons Learned

Mobile messaging mobilization for IRS had limited effectiveness because much of the pilot population

was unable to read and understand the messages (although some illiterate residents could ask other

residents to read the SMS message or explain voice message to them and thus they still got the

information about IRS timing).

Mobile messaging is more effective if used in combination with a mobilizer or some other person

who can answer questions about the spray campaign. Numerous beneficiaries wanted more

information after receiving the messages, but there was no number to call or a point of contact to

ask. The AIRS Mali team could partner with the National Malaria Control Program to set up a toll-

free number that would provide more information.

The enumeration of phone numbers was very costly. In the future, it would be more cost efficient

to work phone number collection into another activity already occurring in these villages, to share

costs across activities.

Future attempts at mobile messaging should investigate partnering with locally based telecom

providers since this pilot partnered with a US-based provider. By working with local partners, there

is the potential to reduce the cost of sending mass messages.

Conclusion

This pilot study was useful in that it provided an example of an mHealth approach that was not

appropriate given contextual constraints. As noted above, the low literacy and strong dependence on

human interaction and social ties in Mali do not provide an atmosphere receptive to impersonal mobile

messaging. Traditional IRS mobilization in Mali leveraged this preference for social interaction; by

eliminating the human touch, the overall success of the intervention suffered, with lower rates of

acceptance and structure preparedness. While the results of this pilot do not encourage doing

mobilization solely through text or voice messages for mobilization in Mali (or similar settings), the pilot

is a stepping stone toward other, more effective implementation designs. For instance, mobile messaging

might be complemented by a mobilizer traveling with an IRS team on the day of spray. The pilot also

highlights the importance of performing a contextual analysis before choosing to implement an mHealth

intervention.

13

SENEGAL: SMARTPHONE-BASED

SUPERVISION FOR REAL-TIME

MANAGEMENT

Background

Supervision of IRS campaigns is vitally important for maintaining safe and effective spray performance. In

Senegal, team leaders, who directly manage the spray operators, are overseen by supervisors from the

government’s National Hygiene Service and AIRS Senegal site supervisors. These supervisors used to

collect data on paper forms – on topics such as spray preparations, spray technique observations, and

stockroom performance – to manage environmental compliance and spray operations throughout the

campaign. The forms were submitted to the AIRS country team for project monitoring and

improvements. Using paper forms had several problems: errors in noting down the data on the report

forms and challenges to promptly submitting the forms meant the country team could not process and

share information in real time, nor could they efficiently identify and correct systemic shortcomings in

spray operations. Consequently, improving spray behaviors and procedures was ad hoc.

Implementation

To standardize project supervision across the project staff and government partners, the PMI AIRS

Project adapted the paper supervisory forms into a mobile application. At the end of each spray day,

government and project supervisors in the field use the application to complete and submit their forms

to a cloud-based open-source platform. The AIRS Senegal database manager compiles the data and

emails a comprehensive daily report to both the country- and international-level stakeholders. The

reports are used to spot and address systemic deficiencies or “red flag” areas immediately.

Work started in April 2014 to build, test, and deploy a mobile system for the July 2014 IRS campaign

that would allow the Senegal and U.S.-based teams to digitize the spray supervision forms to ensure

standardized supervision and immediate analysis. Dimagi, the PMI AIRS Project partner, built the system

using its CommCare platform, an open-source mobile application that enables non-technical users to

access Dimagi’s cloud platform and build their own mobile systems. Dimagi staff worked with members

of AIRS Senegal’s Operations, M&E, Data Management, and IT teams to design a prototype CommCare

Mobile Supervisory Application, test it, and refine it. The Mobile Supervisory Application was finalized

after multiple rounds of feedback from the AIRS Senegal staff, the Dimagi team, the AIRS home office

staff, the IRS Steering Committee on project prototypes, and stakeholders.

Dimagi also held a training of trainers for selected AIRS Senegal staff. The IT and M&E team received

more specific training to learn how to access daily reports from the mobile system. AIRS staff then led

trainings for supervisors and district coordinators on how to use the mobile application. Dimagi staff

also developed all of the necessary training guides and other materials so that the Senegal team could

run their own campaigns in the future, with only limited remote support from Dimagi. Different guides

were developed for 1) team leaders (primary users), 2) district coordinators, and 3) AIRS Senegal staff.

14

Results

The mobile application enabled in-country and U.S.-based program senior management to better manage

spray operations. The AIRS Senegal team recognized the importance of the automated and standardized

supervision forms, and was making real-time corrections in the field based on incoming data. For

example, the AIRS Senegal team was able expedite the morning mobilization and transport the spray

operators to the field sooner. Thus, they had more time for spraying in the villages.

Specific results for the CommCare Mobile Supervisory Application include:

Standardized supervision protocols and quality controls for all campaign supervision efforts.

Ownership by government and AIRS Senegal supervisory staff of the use of supervisory tools.

Empowered government supervisors who use the automated, user-friendly job aid to do their jobs

effectively, such as immediately sending all spray teams a reminder on setting correct spray pump

pressure level after observing such an error repeatedly among spray operators in one team.

Improved campaign management from site level to the central level, as a result of immediate actions

to address deficiencies found with the supervisory tool from the real-time field data.

Lessons Learned

Allow adequate testing time for primary users to transition from the use of paper forms to mobile

forms (i.e., changed question order, new skip patterns, and the way the results are reported in the

mobile form)

Design a system for deploying the supervisory application updates during the spray campaign, while

phones are in use

Increase the training and practice time for supervisors from one to at least two full days of skill

building

Automate the reports in the future to ease the use of the system so the AIRS Senegal team will not

need to pull the data, and enter the data into the tracker before sending it to the applicable

stakeholders every day

Conclusion

By automating data reporting, the mobile system gives field staff and AIRS management immediate access

to data on spray campaign quality, which leads to a better-run spray campaign. Given the success of the

Senegal pilot, the PMI AIRS Project will scale up this work to all spray countries. It should be noted that

the project did not evaluate cost efficiency of switching to smartphone-based supervisory inspections

but it was observed that government counterparts were much more involved and actively supervised

when doing the work with the smartphones.

15

MADAGASCAR: SMS-BASED

PERFORMANCE MONITORING TRACKING

FOR REAL-TIME MANAGEMENT

Background

In the past, the PMI AIRS Project tracked performance using a paper-based system to collect

performance data at every operational site. One of the system’s tools was the Spray Performance

Tracker Sheet, a daily performance report that is filled out by hand by a team or sector leader or site

manager and displayed at each site. It is also used to report data upward. At the end of the spray day,

the country program’s operations manager would call each site to ask about their spray performance

and then enter that data into an Excel spreadsheet. For the operations manager, the process of

collecting individual site data via phone was cumbersome and time consuming, and because s/he could

not reach every site each day, did not provide a complete picture of the project’s spray status on a

regular basis. Another tool that helped with tracking performance was the spray data form that spray

operators fill in daily and a team leader is required to physically deliver it to a data entry site at the end

of the day for uploading to the PMI AIRS database. In Madagascar, the Project sprays in some very

remote areas reachable only by boat and/or long hike. This makes it impossible for spray teams to

deliver the completed spray data forms to the data entry site with enough time for data clerks to enter

the data and analyze the spray coverage before the teams have moved on to other areas.

AIRS Madagascar decided that for the 2014 spray round, it needed a reliable way to collect the daily

operational data so the country team could immediately analyze and improve spray coverage. In that

year, AIRS Madagascar also sprayed a new region of the country. In the initial year of IRS, spray teams

often encounter a high number of refusals because beneficiaries are not familiar with IRS and with being

asked to empty their house prior to spraying. AIRS Madagascar needed a way to closely monitor spray

coverage in case the team needed to use more mobilizers or work with local village chiefs to increase

acceptance.

Implementation

Tapping the skills of its mobile technology partner, Dimagi, Inc., the PMI AIRS Project piloted the

Performance Monitoring Tracker (PMT), an automated system that enables AIRS Madagascar’s team

leaders to send SMS with their team’s daily output data to a cloud-based database. With that, the

Malagasy and U.S.-based project senior management teams track and analyze spray progress data within

districts and for the country as a whole at the end of each day. The real-time data allow them to spot

where specific spray teams are not reaching their objectives and to make programmatic decisions to

ensure high-quality spray coverage.

In August 2014, Dimagi and AIRS Madagascar modified an SMS system that AIRS Senegal had used for

two spray campaigns earlier in the year. The modified SMS system digitized the Spray Performance

Tracker Sheet into an SMS-based PMT tool. As in Senegal, the system uses Dimagi’s open-source SMS

tool CommConnect. CommConnect enables non-technical users to access Dimagi’s cloud platform and

build their own mobile systems for SMS campaigns. A paper Spray Performance Tracker Sheet is still

filled out by hand and displayed at each site for the spray actors in the field.

16

A Team Leader sends PMT SMS at the end of the day after

validating the data. Photo credit: Elana Fiekowsky

Over the course of the spray campaign,

Dimagi staff worked with members of AIRS

Madagascar’s Operations, M&E, Data

Management, and IT teams to modify and test

the SMS system. After a few rounds of

feedback from the AIRS Madagascar staff, the

Dimagi team, and the AIRS home office staff,

the SMS system was finalized.

One day of the week-long IRS training for

sector leaders and district coordinators was

devoted to SMS deployment. Members of

the IT and M&E teams attended their own

specialized training to learn how to access

daily reports from the mobile system. Dimagi

staff also developed all of the necessary

training guides and materials so that the

Madagascar team could run their own SMS

campaigns in the future, with limited remote

support from Dimagi. Different training

guides were developed for 1) sector leaders

(primary users), 2) district coordinators, and

3) AIRS Madagascar staff. For the second

deployment, in the central highlands, all

trainings were led by AIRS Madagascar staff

and the deployment itself required only

remote support from Dimagi.

Following the two SMS deployments, Dimagi staff worked regularly with AIRS Madagascar to answer

questions and address issues.

Results

Immediate, aggregated data were available to AIRS Madagascar decision makers and PMI.

The data were used to more carefully follow the performance of the sprayers and monitor their use

of insecticide.

The SMS tool’s built-in logic checks decreased the number of errors in submitted data. When data

that was submitted above a certain threshold or was not logical given the number of spray

operators, the system would send a message stating there was an error and asking the supervisor to

resend.

Daily campaign progress could be monitored by a larger number of implementers including project

management, government counterparts and PMI..

It was clear that the Madagascar team saw the importance of the immediate access to operational

data, could monitor spray progress of remote spray teams, and was making corrections in the field

based on incoming data.

17

Lessons Learned

PMT system training should take place shortly before the spray campaign starts. If there is a long gap

between the training and campaign start-up, users may need refresher trainings.

Linking the SMS system to the AIRS database would allow assessment of the reliability of the data

being submitted.

Setting up a system that will generate and automatically send a daily record of all SMS sent and

received by email would greatly facilitate monitoring of users.

Spray teams, and supervisors in particular, would benefit from more training on interpreting the

data.

Teams should be trained to handle technical difficulties with the phones.

Conclusion

The PMT pilot was successful in Madagascar. Due in part to the data reporting system, the AIRS

Madagascar team managed a difficult spray campaign that achieved 96 percent coverage of targeted

structures. With the lessons learned, the PMI AIRS Project and Dimagi are implementing the PMT

system in all spray countries.

18

WAY FORWARD

Starting in 2015, the PMI AIRS Project has been rolling out three mobile systems in countries that are

implementing IRS: (1) mobile supervisory forms to assist in the monitoring of environmental compliance

and performance supervision, (2) SMS-based daily reporting of operational data, and (3) the mass

communication of daily reminders and motivational messages. The systems use a range of phone types,

from Android to the simpler SMS-enabled brick phones. Implementing the three mobile systems will

enable the project to engage with personnel at all levels of operations and in more remote areas, where

internet connectivity is not available. In addition, the PMI AIRS Project is working on increasing number

of country programs to adopt mobile payment systems. Through the experiences described in this

publication, PMI AIRS will be able to capitalize on lessons learned and ensure improved performance

across all countries.

20

.