Transforming NGO Management: Real time Information for Decision Making

Workshop Report

Transforming NGO Management:

Real time Information for Decision

Making

Date: Thursday, 29 September 2011

Venue: Directorate General of Health Services (DGHS) Conference Hall Mohakhali, Dhaka-1212

Transforming NGO Management: Real time Information for Decision Making

Content

Intorduction:................................................................................ 1‐2 Workshop Objective .......................................................................................................2 Participants.....................................................................................................................2 Arrangment of the workshop...........................................................................................2

Session: 1: : Introduction and Setting the Context..................................................................................3-4 Session: 2: Current practices of information collection

and usage: Strengths & Weaknesses...................................................................................5-6 Session: 3: Presentation on CMS2 : PNGO Perspective...........................................................................7 Session: 4: Presentation on evolution of information collection

and usage using technologies...........................................................................................8-9 Session: 5: Create decision-making tools using real-time information.................................................10 Session: 6: Wrap Up.....................................................................................................................................11 Annex:..........................................................................................................12‐21


1

Introduction People involved with development initiatives, need to have accurate and timely information to assess the value of what they are doing through periodic assessment of a project’s relevance, performance, efficiency, and impact in the context of its stated objectives. Generally collection of this type information in combination with its analysis and use in a project/programme of an organization is collectively referred to as ‘Monitoring and Evaluation’. The M&E system can be described as a logical chain of linked ideas starting (and continuing) with information users. Information users include persons who are influenced by projects (beneficiaries) as well as those who influence the project (decision-makers, staff). If accurate information is planned, but it is not feasible to collect it or it can’t be collected in time, then the project may go off-track, i.e., it can become ineffective or irrelevant to priorities. If the information is gathered to answer such needs, but it is not analysed, then it cannot be used by the project. Finally, if the information is collected and analysed but not available to the persons who need it, critical decisions about the project may not be made or may be poorly taken.

Nowadays, the use of monitoring and evaluation has become an essential part of all development initiatives. However, approaches to M&E nevertheless vary and over the years different organizations and groups have produced different ways of conceptualizing what M&E involves, what methodologies and indicators and tools can be used, and how. Nevertheless, M&E in Bangladesh has to some extent failed to effectively and efficiently contribute to the decision-making process of projects due to its traditionally lengthy data collection and information generation methodologies. A development professional of a leading organization in Bangladesh, explained his organizational monitoring system as follows:

It’s a huge data set that we have to handle--38 million people--8 million households. Now what’s happening is the data that comes from the field is compiled by the branch manager in a notebook and then he fills in a form and sends it to us at the end of the month. It takes us about 15-20 days to enter it and then start to analyze it, so we basically lose a month. The more human interventions you have and when you’re using calculators, the chances of errors creeping in increase, and then we have to then compile it to the head office and, if we suddenly have a request for a specific data set, they may or may not have kept copies of the original data sheet that comes from the villages.

This situation is to some extent true for most of the organizations and projects in the country. This workshop was arranged to highlight how the use of technology (mobile) as a tool of data collection can transform the monitoring system of organizations and projects which subsequently may facilitate transformation of NGO management.


2

Workshop objectives The key objective of the workshop was to realize the potential of using real-time information to enhance efficiency and effectiveness in decision making.

Specific objectives

- To gain understanding of the current mode of information gathering for decision-making by NGO management and related technological readiness;

- To get the NGO participants to start actively thinking about how real-time household/ individual data can be used for operational and managerial decisions;

- To know the effectiveness of Change Monitoring System (CMS) 2 from the PNGO perspective.

Participants Representatives from all Scale Fund partner NGOs including SF Round 2, Innovation Fund Round 1, 2 & 3 apart from Helen Keller International participated the workshop. Participants from various departments of shiree also attended the workshop. There were a total of 90 participants. Arrangement of the workshop The workshop was organized by shiree along with mPower. The Decision Support Unit (DSU) of shiree was mainly responsible for overall arrangement of the workshop. The support from operation, communication and administration teams of shiree along with technical team of mPower help in making workshop as success. The workshop, under the title of 'Transforming NGO Management: Real-time information for Decision-Making' was held on 29th September at Directorate General of Health Services (DGHS) Conference Hall, Mohakhali, Dhaka-1212. The workshop continued for the day in following schedule:

Session Time/duration Registration 09:30 am-10:00 am Session 1: Introduction and Setting the Context 10:00 am-11:00 am

Tea Break Session 2: Current practices of information collection and usage: Strengths & Weaknesses

11:15 am-12:45 pm

Session 3: Presentation on CMS2 : PNGO Perspective 12:45 pm-01:30 pm

Session 4: Presentation on evolution of information collection and usage using technologies

01:30 pm-02:00 pm

Lunch Session:5: Create decision-making tools using real-time information

02:30 pm-03:45 pm

Tea Break Continuation of Session 5 04:00 pm-05:00 pm Session:6: Wrap Up 05:00pm-05:30 pm

Real-time data denotes information that is delivered immediately after collection. There is no delay in the timeliness of the information provided. Some uses of this term confuse it with the term dynamic data. In reality the presence of real-time data is irrelevant to whether it is dynamic or static.


3

Session: 1: Introduction and Setting the Context

Objective: a) to know the participants; b) to explain objective of the workshop and setting the

context The session began with raising a query over a hypothetical situation by DSU Manager, Avizit: "Imagine a situation where the side mirror of a car in a busy road provides mirror image 10 second later than real time. In such situation what one may expect? Probably chaos on the road. Similarly lack of timely information may diminish the probability of achieving the objective of a project. Hence, the workshop is organized to highlight the need of real time data in managing project and organization." Subsequently, Mr Munirul Islam initiates the formal inauguration of workshop and session. Once the introduction of participants was over, Colin (CEO, shiree) began his presentation on “Transforming NGO management - A big claim”. Issues raised in presentation were:

‐ Transformation is a scary word; ‐ Is transformation of NGO management required given the fact quality of management

varies across NGO? ‐ Information Technology can play important role in transformation of management; ‐ Introduction of information technology require involvement of both technical and social

system of a organization; ‐ Introduction of new IT may invite recalcitrant behavior even in a stable organization as fear

and stress based on probable effect of the intrusion on individual interest; ‐ Need an information system that provides smart, timely, creative and effective decisions

from field-level to strategic management.

Presentation ended with the circulation of a quick survey of participants' perception on introducing new information technology (mobile technology under CMS2) for data collection in their organization/project. Immediately after the collection of the survey questionnaire, Mridul (CEO, mPower) made his presentation on Real-time Digitized Information in Transforming M&E and Management of NGO. The key issues of the presentation are:

‐ Real-time data has the potentiality to transform NGO management as it would: Ensure quick and effective decision as enough information available about

immediately after the collection of data; Treat each beneficiary as an individual, not as a statistical figure; Facilitate individualize interventions and tune it in accordance with the ups and

downs of people lives; Reduce gap between M&E and operational departments.

‐ In using real data, a few things needed to be considered:

Facilitators: Material: Multimedia, Survey Questionnaire Method: Speech, Multimedia presentation, Quick Questionnaire Survey

Colin Risner, CEO shiree Mridul Chowdhury CEO, mPower Avizit Reaz Quazi, DSM, shiree Munirul Islam, CS, mPower


4

Real time information is redundant if not incorporated into the context; Program managers need processed data for making executive decisions;

BRAC Health program is an instant example where real-time information transformed project management. BRAC real time information generation involves:

‐ Use of cell phone for preparation of work schedule ‐ Cell phone software enable Field level information in the form of data, image and voice

recorded data ‐ Data is processed automatically ‐ No cost involved in changing the process of information collection, a dynamic system ‐ The doctor can receive information immediately ‐ Doctor message send back to Community health worker about action for specific patient ‐ Monitoring system can follow through on whether appropriate steps have been taken or not

Mridul ended presentation with questions: real time data collection through cell phones can transform BRAC health program, can it be just as used in other sectors? How?


5

Session: 2: Current practices of information collection and usage: Strengths & Weaknesses

Objective: To make participants aware about strengths and weakness about information

collection process and its usages by their organizations

Facilitators divided the participants into four groups and asked to find out strengths and weaknesses of their organizations in reference two following issues:

I. Data collection and process II. Information usage for decision-making

After 30 minutes, each groups made a presentation using a flipchart. Group findings were compiled through open discussion presentations. Overall, the situation of organization related to data collection, processing and usages varies from organization to organization and it is found that an issue might be strength for one organization while for other it is weakness.

Findings: Strengths

‐ Almost all organizations collect data directly from grassroots level; ‐ Most organizations practice validation and cross-checking of rough data; ‐ Most of organizations have M&E and central MIS units in place; ‐ Some organizations have skill and experienced M&E personnel; ‐ Some organizations are already practicing online data-processing for some specific projects ‐ Logistics for data collection and processing exist; ‐ Almost all organizations use different data collection tools and methods, including PRA,

which enriches the system through triangulation; ‐ In developing data collection tools most of the organizations ensure beneficiary-friendly

formats; ‐ Periodic report are prepared based on data collected on a regular basis; ‐ Most of the organizations find the M&E/MIS information as the only source to refer to before

taking big decisions regarding project and program; ‐ Some participants think that development partners use this information as most important

internal source to take decisions about progress of projects; ‐ Most of the participants felt that introduction of CMS2 has created a situation to take quick

and timely decisions. Weaknesses

‐ Data collection processes of most of the organizations are traditional, lengthy and time-consuming, using paper-based systems;

‐ Some organization experience high turnover of skilled and efficient M&E staff, which hampers smooth functioning of M&E systems;

Facilitators: Material: Flip Chart, Marker Method: Group Work, Group presentation

Abdus Salam PM shiree Masud Rana PM shiree


6

‐ Some organizations lack skilled and trained field staff which sometimes hampers the quality of data collection and processing;

‐ Tendency of data manipulation to depict desired impact was raised by some participants; ‐ Most of the participants observed that sometimes unnecessary data is collected; ‐ Most of the organizations lack skill in collection and processing of qualitative data; ‐ Some formats of some organizations are not beneficiary-friendly; ‐ Language and communication barrier particularly in CHT area effect the data collection

and data interpretation process; ‐ Some of the organization do not have dedicated M&E and MIS departments; ‐ Absence of feedback on monitoring reports is common for most of the organizations; ‐ In some of the organizations, effective measures are not taken to cross-check data validity; ‐ Timely decisions based on M&E information is restricted by the lengthy time consuming data

collection and processing in most of the organization; ‐ In some cases in some organization M&E data are not considered or accepted to make

decision as information is perceived as manipulated; ‐ In some organization formative reporting system restrict dynamic decision-making; ‐ In some cases too much M&E information create a complex situation for decision-making; ‐ Project based information does not help to take decision at organizational level.


7

Session: 3: Presentation on Change Monitoring System 2 : PNGO Perspective

Objective: To have an idea about the NGO perception on Change Monitoring System 2

introduced by shiree

Mr Jahid, opened the shiree website and presented changes that are taking place among beneficiaries of Intercooperation’s Innovation Fund-supported project, using CMS2 data and reporting. After the presentation, an open discussion was initiated. Views of partner NGOs on usages of CMS2 reflected by Jahid's presentation and subsequent discussion are given below:

‐ PNGOs are in a position to get access to individual project information using access codes; ‐ Most of the PNGOs are accessing CMS2 information and some are using it in taking

decisions; ‐ Technically, real-time data access and interpretation is possible through mobile phone and

webpage; ‐ PNGOs perceive the introduction of real time information is a value added initiative; ‐ Need to improve the ways the CMS2 information is presented in web; ‐ There exist some discrepancy between figures and table which needed to be rectified; ‐ More segregation of information in presenting the CMS2 data is needed; ‐ If NGO information is made available graphically/branch office-wise it would be more

effective in operational level decisions; ‐ Quantitative information particularly about income, assets, activities etc. is required for

more effective decision making.

Conclusions: Piloting of CMS2 will continue till possible commencement of full roll out Improvement will be done on presenting of CMS2 data Content analysis of CMS2 will be done and needed improvement will be ensured A detail planning of CMS2 roll out will be done during last quarter of this year

Facilitators: Material: Multimedia, Internet Connection Method: Presentation, Open discussion

Jahid Hasan, Inter Cooperation


8

Session: 4: Presentation on Change Monitoring System 2 : PNGO Perspective

Objective: To highlight how mobile phone technology is more effective to collect information

for decision making Mridul presented how mobile-based data collection systems can be more effective than Optical Mark Recognition (OMR) technology used for data collection. The key points of his presentation are:

Issue OMR System Mobile based data collection (mPower System)

Expected Benefit

Error in Entering Data by User

Scope for error in properly circling the bubbles in field environment without a table or support

Allows for automated checks on the mobile phone for errors in data entry

Less time required; Error rates reduced – both from entering data and OMR reading.

Handling data card

Handling of data card in the field is extremely sensitive – risk of crease, getting wet, and other mishandling

All the mPower system requires is the handling of a cell phone, which is intuitive and commonly used

Zero risk of rejection of data by system

Incomplete questionnaire

Surveyor has the option of submitting without filling the whole questionnaire

Surveyor cannot submit a case on the mobile application without filling out all questions

Zero risk of incomplete survey response

If-then questions

“If-then skip to #x” are hard to follow and take up space

Irrelevant questions do not come up on the mobile screen; since the triage is controlled automatically

No scope for distraction or errors from irrelevant questions

Change of questionnaire at any point as needed

Change of questionnaire is not possible without printing new cards and reformatting OMR machine

Change of questionnaire is very simple and can be updated at the server within a day; the new questionnaire will be automatically downloaded on the mobile phone the next day

New questionnaire sync automatically done without human intervention, making it possible to change questionnaire at any point (no space constraints)

Open-ended questions

No scope for open-ended questions

Open-ended questions can be voice-recorded, which can be transcribed later, if required, and added to an individual’s data using a simple interface

Open ended questions easily incorporated

Picture/ voice No scope for getting picture/ Scope for picture and voice Picture or voice

Facilitators: Material: Multimedia Method: Speech and Presentation

Mridul Chowdhury


9

Issue OMR System Mobile based data collection (mPower System)

Expected Benefit

of beneficiary voice of beneficiary (for non-anonymous surveys) for evidence-based research

of beneficiary recording from interviewee made possible if needed

Tracking and monitoring of surveyors

No way of checking genuineness of filled-out questionnaires

Genuineness of data automatically validated since surveyor has to take picture of beneficiary, or GPS readings can be taken

Drastically reduced risk of false data. Cost of field based monitoring eliminated almost completely.

Real time data and performance tracking

Lag time between the time data is entered and the time that the data is digitized and reported

Data is instantly digitized and reported on a real-time basis

Performance of surveyors instantly tracked; real-time data from the ground;

Logistics Significant logistical hassle to send OMR survey forms to surveyors and then collect them back

The mobile phone based system is given to surveyors at the beginning of survey and all other interactions can be made through the mobile phone

No logistical hassle in dealing with paper-based survey forms; cost of transporting OMR forms safely is eliminated, more productive time on the ground


10

Session: 5: Create decision-making tools using real-time information Facilitators: Md. Towhid Khan ; Abu Saeem Arif; -mPower Hasib Ahsan; Bidhan Chandra Pal;- mPower Md. Touhidul Imran Chowdhury; - mPower Tahmina Khanam; Mohd. Tauseef Anwar- mPower

Materials: Laptops, Multimedia Method: Group work, Presentation

Objective: To know how NGOs' want to see the CMS2 dashboard to access project information

Representatives of different NGOs were divided into four groups, with technical facilitators from mPower, and each group was assigned a topic on which to prepare a dashboard:

‐ Income General Activity ‐ Access to Services and Resources ‐ Vulnerability Assessment ‐ Nutrition and Health

In preparation of the dashboards, all group followed three steps: (1) what decision they might need to taken based on given theme project, (2) what information is needed to take decisions, and (3) how information is presented for the purpose of decision making. Theme/group-wise example dashboards are provided in Annex 1.


11

Session:6 : Wrap Up

Objective: To summarize the issues raised in the workshop Conclusions

Data collection can only be useful when it helps with decision-making and data should be customized for different tiers of management and organizations.

Too much focus on technology is undesirable: everything else is process-based. The findings of the quick survey (see Annex 2) carried out in the first session of the workshop

indicated that people are generally enthusiastic about mobile-based monitoring. Based on workshop findings, initiatives will be taken to improve the questionnaire and

presentation of CMS2 data. Moreover, based on findings of the workshop, piloting of CMS2 would continue till a possible

full roll out, maximizing the benefits of current efforts to set up the system. Content and process analysis of CMS2 will be done with necessary improvements made. A detail plan of CMS2 roll out will be made during last quarter of this year and shared with

all partner NGOs.

Facilitators: Colin Risner, CEO shiree Mridul Chowdhury CEO, mPower

Material: Multimedia, Survey Questionnaire Method: Speech, Multimedia presentation,


12

Annex 1: Examples of NGO dashboards


13


14

Annex 2: Findings of CMS2 Receptiveness Survey


15

Annex: 3: Visual Documentation of the Workshop

Workshop in Progress


16

Workshop in Progress

Group Work in Progress


17

Group Presentation in Progress

Open Discussion.


18

Workshop Facilitators


19

Annex: 4: Participants' attendance


20


21

Date post:	22-Mar-2016
Category:	Documents
Upload:	eep-shiree
View:	220 times
Download:	1 times

Transforming NGO Management: Real time Information for Decision Making

Documents