An outcomes evaluation of Search For Common Ground’s · PDF fileAn outcomes evaluation...

An outcomes evaluation of Search For Common Ground’s

The Team Tanzania November 2011 – September 2013

December 2013

Dunstan Kishekya

John Mauremootoo

&

Richard D Smith

RDS Consulting Ltd

[email protected] &

www.InspiralPathways.com

1

Acknowledgements A great many community members gave their time as informants. For this, we are very grateful. Supporting the fieldwork in Mvomero and Kilwa in her capacity as Research Assistant, Ansila Marandu made a vital contribution to the data collection and participated in useful discussions on the interviews for which we offer special thanks.

We are very grateful to all of the staff at Search for Common Ground Tanzania, in particular Paul Glick (Country Director), Cornelia Wamba (Social Media Coordinator) and Stella Msami (The Team Manager), who supported our work through the timely provision of background materials and valuable discussions during the evaluation process.

Our work in the field was made possible by the following SFCG partner organisations and their representatives; The Mvomero Organizations Coalition (in Mvomero), Women and Girls Fight illiteracy and Poverty Organisation (in Kilwa) and Save the Children Tanzania (in Tarime) Lastly, we thank Kate Dyer and Layla Ghaid of the AcT and International Development Advisory Services programmes at KPMG for sharing their expectations of the report and providing valuable suggestions particularly on the value for money analysis.

Abbreviations

AcT Accountability in Tanzania Programme CSO Civil Society Organisation DfID UK Department for International Development

FG Focus Group FGD Focus Group Discussion FGM Female Genital Mutilation GBV Gender Based Violence M&E Monitoring and Evaluation

OM Outcome Mapping SFCG Search for Common Ground

SMART Specific, Measurable, Achieved, Relevant and Timely SMS Short Message Service VfM Value for Money

3R SFCG’s Reach, Resonance and Response theory of change

2

Contents Abbreviations ....................................................................................................................................... 1

Contents ............................................................................................................................................... 2

1 Executive Summary ......................................................................................................................... 4

2 Introduction ..................................................................................................................................... 8

2.1 Background ............................................................................................................................... 8

2.2 Evaluation team ........................................................................................................................ 8

3 Evaluation Approach ....................................................................................................................... 9

3.1 Data collection .......................................................................................................................... 9

3.2 Credibility of outcomes .......................................................................................................... 11

3.3 Analysis ................................................................................................................................... 12

3.4 Limitations .............................................................................................................................. 12

4 Project Description ........................................................................................................................ 14

4.1 The Team ................................................................................................................................ 14

4.2 Context for gender equality interventions in Tanzania .......................................................... 15

5 Findings: Effectiveness of The Team and validity of the theory of change ................................... 16

5.1 Resonance and Response ....................................................................................................... 18

5.1.1 Relative frequency of outcomes among males and females ........................................... 19

5.1.2 SFCG’s 5 priority issues .................................................................................................... 19

5.1.3 Clustering of outcomes by location ................................................................................. 20

5.1.4 Financial benefits & inheritance ...................................................................................... 20

5.1.5 Schooling ......................................................................................................................... 21

5.1.6 Participatory approaches to decision making and planning ............................................ 21

5.1.7 Football ............................................................................................................................ 22

5.2 Unintended results ................................................................................................................. 22

5.3 Negative outcomes ................................................................................................................. 22

5.4 How The Team has contributed to changes in attitudes and behaviours .............................. 23

3

6 Findings: Value for Money ............................................................................................................. 28

6.1 Monetary value of selected case studies .................................... Error! Bookmark not defined.

6.2 Potential value for money of The Team’s focus groups ......................................................... 32

6.3 Potential value for money of The Team ..................................... Error! Bookmark not defined.

6.4 Incremental cost of scaling up ................................................................................................ 36

6.5 Limitations and assumptions .................................................................................................. 36

7 Findings: obstacles and suggestions .............................................................................................. 39

7.1 Obstacles ................................................................................................................................ 39

7.1.1 Project implementation ................................................................................................... 39

7.1.2 Dissemination and content of the TV and radio shows ................................................... 39

7.1.3 Social issues ..................................................................................................................... 40

7.2 Suggestions for improvement ................................................................................................ 40

8 Lessons learned and recommendations for discussion ................................................................. 42

Annex A: Short outcomes – Tarime

Annex B: Short outcomes – Kilwa

Annex C: Short outcomes – Mvomero

Annex D: Case studies

Annex E: Value for money calculations – case studies

Annex F: Value for money calculations – focus groups

Annex G: Information sources

Annex H: Analysis of Facebook responses

4

1 Executive Summary The Team Tanzania (hereafter The Team) is one of 15 completed or ongoing projects implemented in Africa and Asia by SFCG using The Team concept. Starting in 2008, SFCG has used The Team is a vehicle to tackle polarizing societal issues by stimulating learning in a persuasive, but non-confrontational manner. For Tanzania the issue chosen was gender equality and the goal of The Team was to contribute to strengthening the implementation and enforcement of gender-sensitive legislation in Tanzania.

The Team started in November 2011 and is due to end, after a no cost extension, in December 2013. It has been financed by DfID with a grant of £875K. The grant is managed for DfID by KPMG through the Accountability in Tanzania programme. Monitoring and evaluation is a component of the project and in this context an end-of-project evaluation was required to identify results and assess the project’s effectiveness and value.

On many measures – legal, ratification of international declarations, representation in Parliament, women’s knowledge of rights – Tanzania ranks highly in gender equality. The challenge faced by those seeking to promote gender equality is to overcome the cultural constraints and customary laws and practices that stand between high-level commitments and achieving gender equality in the real world of peoples’ lives.

RDS Consulting Ltd was commissioned to undertake this evaluation during the period September 2012 – December 2013. The evaluation is expected to contribute to accountability and learning needs of SFCG, KPMG and DfID. The outcomes assessed in this evaluation occurred in the period November 2011 and October 2013.

The principal approach used in this outcomes evaluation was the identification, description and interpretation of outcomes defined following the definition of ‘outcome’ used in the Outcome Mapping methodology developed by the Canadian International Development Research Center: a change in the behaviour, relationships, activities, or actions of the people, groups, and organizations with whom a program works directly. Use of this definition of outcomes sets a high bar for assessing the effectiveness of The Team’s contributions to gender equality: we were not looking for evidence that participants in the interventions had only learned something new or had a new attitude, we were looking for observable evidence that they had actually applied their learning or demonstrated their attitude change in terms of changed behaviour. Accurate and robust data were generated using the Outcome Harvesting tool.

We evaluators identified and described – harvested - outcomes from some of the social actors the project had been seeking to influence directly: the target groups of The Team outreach work in 3 of the 12 rural districts where focus groups had been shown The Team TV show at mobile cinema screenings. We called these our ‘treatment groups’. Outcomes were harvested from the treatment group during focus group discussions and interviews in Mvomero district of Morogoro region (central zone), Kilwa district of Lindi region (southern zone) and Tarime district of Mara region (northern zone) from 15th – 27th October 2013. During the same period, in Mvomero and Kilwa we also harvested outcomes from target group representatives that had not participated in the mobile screenings and focus groups. We called these our ‘control groups’.

On analysis, we found that some of our data did not qualify as descriptions of observable and hence verifiable behaviour changes (outcomes) but were in fact descriptions of claimed

5

changes in attitude, awareness, knowledge or capability. We classified such descriptions as ‘proto-outcomes’ and defined outcomes and proto-outcomes together as the ‘results’ of the project. We described 54 outcomes and 10 proto-outcomes in total.

We expanded 9 of the 54 outcomes as case studies, each of which we substantiated with at least one and usually two or more informants. Additionally, we added a further dimension to substantiation of some outcomes through visual inspection of physical structures and documentation. Lastly, where possible we substantiated short outcome descriptions with other sources. We evaluators conclude that the outcomes are valid and as a minimum are the most significant outcomes known to the informants consulted and are very likely to be indicative of the achievements of the project overall up to October 2013.

Through the collection, analysis and interpretation of the outcomes data, informant interviews and documents analysis, the evaluators sought answers to the evaluation questions agreed with SFCG and KPMG to assess:

• The effectiveness of the interventions in contributing to intended or unintended outcomes relevant to the purpose of the interventions

• The validity of The Team theory of change • The implied value for money of The Team • Obstacles to the achievement of results

FINDINGS AND CONCLUSIONS

Effectiveness and validity of the Theory of Change

Considered together, the results – 54 outcomes and 10 proto-outcomes - demonstrate that the rural outreach component of The Team has generally been very effective, particularly considering the short duration of the project, having contributed to numerous examples of changes in women, men, boys and girls, that either demonstrate or are relevant to the achievement of greater gender equality.

The 9 case studies elaborated from the outcome descriptions provide convincing evidence not only of changes in behaviour but of the consequences of such changes for the economic empowerment of women, school attendance, growth of businesses, avoidance of debt, dissemination of gender equality ideas and the application of participatory approaches to planning and decision making in communities.

The outcomes demonstrate changes relevant to all 5 priority issues except rape, an issue that SFCG did not expect to find changes in at the sites where we sampled.

Assessed against the objectives of the intervention the outcomes demonstrate that the project was most effective at influencing behaviour changes at the individual level. We anticipate that there is potential for the outcomes to contribute further over time to objectives at the community level and to contribute to other initiatives beyond the CSO partners, for example through the dissemination of this evaluation and its supporting data.

The Resonance and Response elements of the Theory of Change were, insofar as it has been described, clearly validated: all the proto-outcomes demonstrate Resonance and all the outcomes demonstrate Response.

6

Rarely if ever was The Team the sole contributing factor to results but we found that The Team TV show and mobile screenings and focus groups made an important or very important contribution to 39 of the results we described and a useful contribution to 25. That no informants cited the radio shows as an influence is surprising and warrants further investigation.

The sustainability of the results is not clear from our data. Further investigations, including of capacity changes in CSO partner organisations, would be useful for investigating this.

Value for money

The outcomes demonstrate the project has contributed to the kinds of (behaviour) changes that are likely to be necessary for the achievement of sustained higher-level outcome / impact changes in gender equality. Further, we have been able to make detailed descriptions of how the project’s outputs have contributed to outcomes.

A very high proportion of treatment group informants – 63% - reported results.

The outcomes we harvested from the treatment groups were qualitatively much richer than those of the control group, suggesting the mobile cinema and focus group discussions are more effective than broadcasts of the TV show alone.

Based on our findings, participants across all 12 focus groups may be able to identify in the order of 697 outcomes, 328 (39%) of which may be monetisable.

We estimated the value for money of mobile cinema and focus groups expressed as the ratio of financial input to output to be 3.67.

The incremental cost of scaling up the mobile screenings and focus groups is approximately US$12,000 / group of approximately 70.

It is likely that these figures underestimated the monetary value of the mobile cinema and focus group discussions and greatly underestimated the monetary value of The Team overall.

LESSONS LEARNED AND RECOMMENDATIONS FOR DISCUSSION

The following recommendations are intended to stimulate discussion and where appropriate decisions by SFCG and relevant stakeholders:

Scaling up: research (i) the relative effectiveness of the focus groups compared to TV show viewing, (ii) the incremental cost of more focus groups which is relatively low now that the TV show has been produced, (iii) the useful project participant’s suggestions for further dissemination.

Choice of media: Investigate (i) the costs and benefits of producing a second series of The Team compared to promoting further responses from the first series; (ii) the resonance and response of the radio audience.

Support for other gender equality initiatives: consider sharing this report and its supporting data.

7

Sustainability: Initiate as a priority a strategy for sustaining and building on results achieved.

Monitoring, evaluation and action learning: consider (i) updating SFCG guidance on how to document resonance and response to include qualitative approaches and Outcome Mapping in particular; (ii) collecting and using monitoring data from the outset of the project that will be needed for evaluations; (iii) increasing the evaluation budget for similar projects.

Design of future work: (i) consider describing a theory of change that articulates how a future intervention can build on the types of changes we have recorded to achieve the 5 priority issues or similarly higher-level results; (ii) include project partners in project descriptions and reporting to better reflect the SFCG contribution.

8

2 Introduction

2.1 Background Search for Common Ground’s (SFCG) The Team Tanzania (hereafter The Team) project started in November 2011 and is due to end, after a no cost extension, in December 2013. It has been financed by DfID with a grant of £875K. The grant is managed for DfID by KPMG through the Accountability in Tanzania programme. Monitoring and evaluation is a component of the project and in this context an end-of-project evaluation was required to identify results and assess the project’s effectiveness and value.

RDS Consulting Ltd was commissioned to undertake this evaluation during the period September 2012 – December 2013. The evaluation is expected to contribute to accountability and learning needs of SFCG, KPMG and DfID. The outcomes assessed in this evaluation occurred in the period November 2011 and October 2013.

Primary intended users and uses

1. SFCG in Tanzania, who wish to understand more about the effectiveness of the intervention and learn lessons for the potential new phases of the intervention.

2. SFCG Common Ground Productions, who wish to use the evaluation findings to inform SFCG’s gender-related work in Tanzania, and in East Africa as a whole.

3. KPMG Advisory Limited (KPMG) Tanzania, who wish to use the evaluation findings to know more about the effectiveness of the theory of change, identify communicable stories about the project’s results and gain insight into the value for money of the initiative.

Other evaluation users and uses Other social actors who may be interested in the evaluation findings to enhance their understanding of the effectiveness of The Team Tanzania include the SFCG-Tanzania partners and other social actors involved in this evaluation, and DfID who is the major funder of this initiative.

Process uses – The evaluation was undertaken using a participatory methodology involving partners and other informants in the process of outcome description and substantiation. This activity is expected to help to enhance The Team Tanzania's partners' understanding of the links between their activities and outcomes achieved. Substantiation can serve as a way of strengthening connections between SFCG and the external partners who are invited to participate in the evaluation process.

2.2 Evaluation team The evaluation team was co-led by Richard Smith (Director, RDS Consulting Ltd), John Mauremootoo and Dunstan Kishekya. Richard is an international consultant based in the UK, with expertise in the use of Outcome Mapping for evaluation and planning and with experience of policy advocacy and advocacy evaluation. John is an international consultant based in the UK, with expertise in the use of Outcome Mapping for evaluation and planning. Dunstan Kishekya, is a consultant based in Arusha, Tanzania with experience and expertise in applying Outcome Mapping in evaluation.

9

3 Evaluation Approach

3.1 Data collection The principal approach used in this outcomes evaluation was the identification, description and interpretation of outcomes defined following the definition of ‘outcome’ used in the Outcome Mapping methodology developed by the Canadian International Development Research Center1: a change in the behaviour, relationships, activities, or actions of the people, groups, and organizations with whom a program works directly. Informant interviews and document analysis supplemented the outcomes data. The supplementary information has been used to understand the project, its outcomes and their context and as sources of data for Value for Money analyses.

Use of this definition of outcomes sets a high bar for assessing the effectiveness of The Team’s contributions to gender equality: we were not looking for evidence that participants in the interventions had only learned something new or had a new attitude; we were looking for observable evidence that they had actually applied their learning or demonstrated their attitude change in terms of changed behaviour.

Inspired by Outcome Mapping, Outcome Harvesting2 is a monitoring and evaluation tool which we used it to guide the identification and description of outcomes. Outcome Harvesting helps generate accurate and robust data because it requires descriptions of outcomes and programme contributions to be precisely formulated such that it is clear who changed in what way, when and where, and how the change agent – in our case, SFCG and their contracted implementing partners - contributed to each outcome. The outcome and contribution statement must be sufficiently specific to be verifiable.

To be verifiable, outcome descriptions need to be SMART:

o Specific – someone lacking specialised subject or contextual knowledge will be able to understand and appreciate the description

o Measurable - contains objective, verifiable quantitative and qualitative information o Achieved - a plausible relationship and logical link between the outcome and the change

agent’s actions o Relevant - a significant step towards the impact that the change agent seeks o Timely - the outcome occurred within the time period being evaluated

Identifying, describing and analysing the 54 outcomes and 10 proto-outcomes

Typically, Outcome Harvesting starts with the evaluators screening reports, websites and other materials and recording outcomes. SFCG advised from the outset that the potential for identifying outcomes from reports was limited as a key objective of the evaluation was to

1 Earl, S, Carden, F & Smutylo, T (2001). Outcome Mapping. Building learning and reflection into development programs, IDRC, Ottawa. http://www.idrc.ca/EN/Resources/Publications/Pages/IDRCBookDetails.aspx?PublicationID=121 (accessed 22 November 2013)

2 Outcome Harvesting, 2012, Ricardo Wilson-‐Grau & Heather Britt, Ford Foundation. Available from http://www.outcomemapping.ca/resource/resource.php?id=374 (accessed 22 November 2013)

10

collect and interpret such data. Nonetheless, we evaluators sought to identify and describe outcomes by examining the documents we were provided with by SFCG. We examined a total of 10 documents (Annex G for a list of information sources consulted) and found them entirely lacking in evidence about outcomes that the project may have contributed to. Concerning the monitoring data in particular, the mobile phone response reports of PUSH Mobile provided data on attitudes but none on outcomes. The data on Facebook responses contained many claims of changes in knowledge and attitudes but lacked information sufficient to describe one or more outcomes i.e. behaviour changes (see Annex H for a collation of the claimed changes).

SFCG identified the outreach component of the project as a potentially rich and accessible source of outcome information. It was therefore agreed that we evaluators would seek to harvest outcomes from some of the social actors the project had been seeking to influence directly: the target groups of The Team outreach work in the rural districts where focus groups had been shown The Team TV show at mobile cinema screenings. We also agreed, depending on opportunities available in the field and resources, to convene and harvest outcomes from people that had not participated in the mobile screenings in order to have the opportunity to make observations on any qualitative differences between these informants and those who had participated in focus groups. With the sampling strategy agreed, we conducted focus group discussions and interviews in Mvomero, Kilwa and Tarime from 15th – 27th October 2013.

Our informant groups were convened with the assistance of SFCG’s partners in the three locations:

Central zone: Mvomero district, Morogoro region: Mvomero Organisations Coalition

Southern zone: Kilwa district, Lindi region: Women and Girls Fight illiteracy and Poverty Organisation

Northern zone: Tarime district, Mara region: Save the Children Tanzania

Locations for data collection were selected to be as representative as possible with available resources of (a) the 12 districts in 6 regions where the mobile screenings took place, (b) of Tanzania as a whole. Our sampling approach was therefore sensitive to some of the major cultural, social and economic regional variations in Tanzania. However, a more wide ranging sampling design would cover more of the diversity but this was beyond the scope of this evaluation.

We worked with two sets of informants: treatment groups and control groups. In each case, the groups convened were gender-balanced and representative of the four target groups: adults, youth, teachers and community leaders. The treatment group members had participated in The Team focus groups and watched the TV show at mobile screenings. The control group members were citizens from the same target groups in the same locality who had not participated in any part of the project. They were not true controls, however, as The Team intended that focus group participants would share messages and initiate discussions around key gender equality issues with others outside the focus group.

In Mvomero and Kilwa we harvested outcomes from both treatment and control groups; in Tarime because of resource constraints we agreed with SFCG to harvest outcomes only from the treatment group. On average, our informant treatment groups contained about 30%

11

of the members of each target group (ca. 6-9 people). The informant control groups were of similar sizes.

Following the informant group work, individuals who had described outcomes were consulted further as necessary on a one-to-one basis to obtain the data needed to complete a precise and verifiable outcome description.

The evaluators then formulated the outcomes so they met the criteria of SMART outcome descriptions, that is they each contained a precise description of the change the project had influenced and a concise description of how it had contributed to the change. On analysis, we found that some of our data did not qualify as descriptions of observable behaviour changes but were in fact descriptions of claimed changes in attitude, awareness, knowledge or capability. We consider the proto-outcomes to have credibility as they were described, in each case, by the social actor who claimed to have changed. However, they are not as credible as observable, hence verifiable, outcomes. We classified such descriptions as ‘proto-outcomes’ and defined outcomes and proto-outcomes together as the ‘results’ of the project. The number and location of outcomes and proto-outcomes we described as summarised in Table 1.

3.2 Credibility of outcomes The credibility of outcomes as evidence for the evaluation depends on what the intended users require to consider the outcomes to be accurate. The outcomes we described meet four criteria that support their credibility:

• Informants were knowledgeable about the outcomes. For all outcomes, the informant was the social actor that had changed or had a close relationship with the social actor.

• Informants agreed to go on record with the information they provided about outcomes.

• The description of outcomes and how the intervention contributed are specific enough to be verifiable.

• The relationship between how The Team contributed and the outcomes was judged by the evaluators to be plausible.

In order to further enhance the credibility of the outcomes, we agreed with SFCG that we would seek substantiation of a proportion of the outcomes with independent sources. To do this, we expanded 9 of the short outcome descriptions into case studies with sections on the outcome, its significance and the contribution of The Team. Results of the substantiation are summarised in Table 2.

Additionally, we sought substantiation of short outcome descriptions opportunistically. In each case, the source consulted confirmed the outcome described.

In conclusion, we evaluators consider that the outcomes are valid because they meet the credibility criteria described above and, furthermore, because most were described by the social actors who changed, not by the change agents who may have claimed to have

Table 1 The number and locations of the outcomes and proto-outcomes

Results - total

Outcomes Proto-outcomes

Mvomero 36 28 8 Kilwa 15 14 1 Tarime 13 12 1

12

influenced them. We also consider that as a minimum the most significant outcomes have been identified from the informants consulted and that the outcomes are indicative of the achievements of the project in the 12 rural locations where outreach work was focused up to October 2013.

Table 2 Results of substantiation of case studies

Case study

Location Outcome Substantiation

1 Mvomero

School girl truancy falls after girls’ football team is created

Outcome and contribution fully substantiated by three sources

2 Mvomero

School building construction becomes significantly more effective by using a participatory approach

Outcome and contribution fully substantiated by four sources

3 Mvomero

Widows’ group’s new planning process boosts income and savings

Outcome and contribution fully substantiated by three sources and outcome substantiated by observation.

4 Mvomero

One woman’s business grows as she gains confidence and ambition

Partially substantiated by two sources. Both confirmed the business had grown during the period described but were not know the detail of how much it had grown nor how The Team was said to have contributed.

5 Kilwa Self-awareness as the result of The Team motivates Hamidi to rejoin schooling

Outcome fully substantiated by two sources; contribution fully substantiated by one source, the second source did not know

6 Kilwa Mobilising fellow school children to fight together for gender equality

Outcome fully substantiated by one source; source was unaware of the contribution

7 Kilwa A woman’s realisation that it is unacceptable to marry school age girls leads to a marked reduction in her conspicuous consumption

Outcome and contribution fully substantiated

8 Kilwa A widow’s enhanced self-reliance and financial freedom

Outcome and contribution fully substantiated by one source and outcome substantiated by observation.

9 Tarime A wife enjoys greater well-being and income after her husband grants her shared ownership of assets and income

Outcome fully substantiated by two sources; contribution partially substantiated – lack of knowledge

3.3 Analysis Short descriptions of outcomes / proto-outcomes and contributions (about 2 sentences each) together with details on the source, location and date of the outcome harvest were entered into an MS Excel database. Each outcome and proto-outcome was then coded according to its relevance to answering the evaluation questions. Additionally, we assessed a subset of case studies for value for money. The analysis is presented in Chapter 6.

3.4 Limitations For reasons stated above – see: Identifying, describing and analysing the 64 outcomes and proto-outcomes - all the outcome descriptions used in the evaluation came from information sources in the locations targeted by the outreach component of The Team, a component of the project that was delivered entirely in rural areas. As a result, we were unable to consider any outcomes to which The Team may have contributed in urban areas. An implication of this is that contribution of the social media component of The Team, work that was directed largely at urban populations, is not assessed in this evaluation.

13

An inherent limitation on the data we could collect is the very short time between the start of the project and the evaluation. Achieving a major shift in gender equality is almost certain to take longer than 18 months. The results that may be observed after a short intervention are therefore likely to be lower level awareness, attitude, knowledge and behaviour changes. An assessment sometime in the future will be needed to determine if any changes initiated by The Team lead to further, deeper and sustained changes.

It is typically challenging to identify negative outcomes. People are reluctant to mention them and organisations tend not to record them, although SFCG informed us they do not exclude them from their monitoring data. We were only able to describe two.

The time required to move between sites whilst maximising use of available resources for the evaluation limited the potential to sample more widely.

The lack of existing monitoring data or reports that described outcomes meant that all data for the evaluation had to be collected from scratch, therefore limiting the potential to sample more widely.

We did not, by design, harvest outcomes from sources internal to The Team i.e. SFCG and its CSO implementing partners. Our data therefore does not support an analysis of capacity development or any other changes that may have occurred during implementation.

While we were able to record data on some highly sensitive issues including GBV and ‘sugar mummies’ we recorded very little on probably the most sensitive issues of all: we had only one data point on FGM and did not record any data related to rape.

14

4 Project Description

4.1 The Team The Team Tanzania is one of 15 completed or ongoing projects implemented in Africa and Asia by SFCG using The Team concept3. Starting in 2008, SFCG has used The Team is a vehicle to tackle polarizing societal issues by stimulating learning in a persuasive, but non-confrontational manner4. For Tanzania the issue chosen was gender equality and the goal of The Team5 was to contribute to strengthening the implementation and enforcement of gender-sensitive legislation in Tanzania.

Implementation of The Team is guided by the project’s mission statement:

To champion women’s rights in Tanzania by highlighting and modeling locally-rooted solutions to: 1) The customary and traditional attitudes that undermine the country’s gender-sensitive laws; and 2) The prevailing attitudes of men that are a barrier to women’s rights in Tanzania.

SFCG intended to use five types of activity to realise the mission:

1. Consultation workshops and baseline assessment. These aimed of developing key messages and themes of specific relevance to Tanzania and establish current knowledge and attitudes.

2. The television series of 13 episodes. Intended for national broadcast, this was set in a secondary school aimed to catalyse discussion of key gender equality issues.

3. The radio series of 50 episodes, also call Wamoja. This aimed to challenge gender norms and is set in a rural area as most of its intended audience.

4. Outreach campaign. This used screenings, focus groups, SMS, social media and other tools and events to motivate the target audience to watch and listen and engage with the programmes.

5. Monitoring and evaluation. This has comprised the baseline study, collection and analysis of SMS and Facebook responses and this evaluation.

Implementation of key components of the project was subcontracted to:

• 11 local partners to organise mobile cinema screenings and facilitate focus group discussions in 12 districts across 6 regions covering southern, central and northern zones of Tanzania. SFCG selected the partners, organised training for the facilitators and provided an experienced trainer and advice.

3 http://www.sfcg.org/programmes/cgp/the-team.html (accessed 29.11.13)

4 A proposal from Search for Common Ground in Tanzania to The UK Department for International Development, July 2011.

5 In this report, we use The Team to refer to the The Team Tanzania intervention as a whole not just the TV and radio shows.

15

• A TV production company to produce the 13 episodes. SFCG led development of the ‘curriculum’ of key messages and themes through a consultative process in three regions.

• A radio production company to produce the 50 episodes. The radio shows were similarly guided by the curriculum developed by SFCG.

• Marketing and media partners to promote the shows through multiple strategies.

4.2 Context for gender equality interventions in Tanzania On many measures – legal, ratification of international declarations, representation in Parliament, women’s knowledge of rights – Tanzania ranks highly in gender equality. DfID’s operational plan for Tanzania 2011-156 states that:

Tanzania has undertaken substantial policy and legal reforms to address gender inequality and empower girls and women. As a result, Tanzania has achieved gender parity in primary education and increased the participation of women in politics and decision making authorities, particularly in the public sector. However, despite these achievements, the majority of women in Tanzania are still locked into traditional roles and […] have limited participation in economic activities so have benefitted very little from growth.

In a 2011 assessment mission, SFCG recorded evidence of the gaps between high-level concern for gender-equality and reality:

• Girls’ low retention rates in secondary schools. • Barriers to women owning property e.g. land inheritance. • High incidences poverty and its consequences among women. • High rates of gender based violence (GBV) and rape. • Barriers to participation in political decision-making.

The challenge faced by those seeking to promote gender equality is to overcome the cultural constraints and customary laws and practices that stand between high-level commitments and achieving gender equality in the real world of peoples’ lives.

6https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67404/tanzania-2011.pdf (accessed 29.11.13)

16

5 Findings: Effectiveness of The Team and validity of the theory of change

This chapter answers two evaluation questions:

1: Do the outcomes indicate that the project was effective in terms of: i. Contributing to pre-defined objectives and results and ii. Contributing to unintended objectives and results consistent with the purpose of the project?

2: Do the outcomes and their associated contribution descriptions validate the Team (TV and radio)’s theory of change (3R approach)?

Most of the 54 outcomes and 10 proto-outcomes identified (see section 3.1 for definitions) are relevant to the pre-defined objectives (Box 1 and Table 3) and theory of change (Figure 1). Considered together, they demonstrate that The Team has generally been very effective, particularly considering its short duration, having contributed to numerous examples of changes in women, men, boys and girls that are relevant to the achievement of greater gender equality. However, there are three caveats:

(a) While the project has contributed to an impressive number of outcomes relevant to its pre-defined objectives, the outcomes suggest there is a marked difference in the extent to which each of the three objectives have been realised: the great majority of outcomes identified are relevant to pre-defined Objective 2 and very much fewer relate to Objectives 1 and 3 (Table 3)7. Despite this imbalance, we do not, as explained below, consider the effectiveness overall to have been compromised.

7 We interpreted results relevant to Objective 1 as changes related to female roles in decision making at the community level i.e. groups of people, villages or other administrative unit.

Box 1 Pre-defined objectives of The Team*

1. To foster an enabling environment to advance female roles in decision-making at the community level

2. To promote knowledge and attitudes favourable to women’s well-being at the individual level; and

3. To contribute to local and international initiatives aimed at advancing gender equality.

* SFCG proposal for The Team, page 8.

Figure 1 The SFCG Three Rs framework for media*

*Adapted from: Telling the story of The Team: A framework. SFCG.

17

(b) All but 8 of the outcomes and proto-outcomes (results) were identified from the treatment groups8. Nonetheless, the fact that our sampling identified results from the two control groups indicates that The Team has had effects beyond the focus groups. For further discussion of the relative effectiveness of the treatment and control groups see section 6.3.2.

(c) As agreed with SFCG, we did not set out to determine who has seen the show in the population at large hence did not seek to validate the Reach element of the theory of change. To do so would require, as SFCG recognise9, a representative survey on a regular basis.

The 6 outcomes and 1 proto-outcome not relevant to the pre-defined objectives are nonetheless positive changes hence should be viewed as achievements.

Table 3 Relevance of results to the pre-defined objectives10

Total number of results

identified

Results relevant to the pre-‐defined

objectives

Unintended results

1 2 3

Outcomes 54

5 45 7

6

Mvomero 28

3 23 4

4 Kilwa 14

1 11 1

2

Tarime 12

1 11 2

0

Proto-‐outcomes 10

2 6 0

1 Mvomero 8

2 4 0

1

Kilwa 1

0 1 0

0 Tarime 1

0 1 0

0

Examples of results demonstrating an enabling environment to advance female roles in decision-making at the community level – Objective 1 - include the male teacher who now accepts that new ideas should be judged by their qualities not by who puts them forward (Mvomero 32) and behaviour changes such as the leadership shown by a woman in creating a new women’s group to support ten women farmers (Mvomero 9).

It is perhaps not surprising that there are relatively few results relevant to advancing female roles in community-level decision-making (Objective 1) because these can be expected to come some time after changes at the individual level (Objective 2), once more women have

8 Mvomero: 4 outcomes and 2 proto outcomes; Kilwa: 1 outcome, 1 proto; Tarime – no control.

9 Telling the story of The Team: A framework. Internal SFCG document.

10 Some results are relevant to more than one objective so the total number of results may not equal the sum of the other columns in each case.

18

the confidence and freedom to identify with community-level decision-making roles and more men and women accept women in such roles.

The relatively low number of outcomes demonstrating The Team contributing to other local and international gender equality initiatives - Objective 3 – is at least partly a factor of the short duration of the project. That we found changes at the individual if not many at the institutional level is a sign of good progress. A further factor limiting the number of outcomes about influencing other initiatives is that the principal sources that SFCG agreed we use for the evaluation was individual members of the communities we sampled in. With more time, we could have systematically assessed the influence of The Team on other initiatives. Instead, the data we collected on this was opportunistic, as part of our outcome harvesting from target groups. This explanation aside, we did succeed in identifying and describing 7 outcomes that demonstrate different ways in which The Team has contributed to local initiatives concerned with gender equality. We also suggest that it is quite possible that the project will contribute further to gender equality initiatives if its results and lessons learned – such as those described in this report - are shared.

Examples of outcomes contributing to Objective 3 include the significant way SFCG’s partner in Mvomero, The Mvomero Organizations Coalition, has changed its strategy for working on gender equality by involving women much more and engaging a woman to coordinate gender issues (Mvomero 25), and a number of outcomes that show women joining or reviving business or savings groups (Tarime 4, Kilwa 4), starting a group to support income generation through crop growing (Mvomero 9) and introducing groups to participatory approaches to income generation (Mvomero 8).

It is the influence the project has had on promoting knowledge and attitudes favourable to women’s well-being at the individual level11 (Objective 2) where the project has excelled, as demonstrated by 45 outcomes and 6 proto-outcomes.

Outcomes and proto-outcomes are now explored to exemplify how the results validate the Resonance and Response elements of the theory of change.

5.1 Resonance and Response By definition, proto-outcomes are changes in awareness, understanding or attitude. Therefore we consider that all proto-outcomes relevant to the pre-defined objectives are examples of resonance as defined by the SFCF theory of change i.e. evidence that the audience has related to, understood, engaged, or gained new knowledge from the show.

Similarly, outcomes are behaviour changes hence all those that are relevant to the pre-defined objectives are examples of response as defined by the SFCF theory of change. Because a response first requires a change in awareness, understanding or attitude, all response-related outcomes are also illustrations of resonance.

The outcomes demonstrate a number of behavioural change patterns. In this section we look at the relative frequency of outcomes among males and females, how the outcomes did or did not contribute to SFCG’s five priority issues and other themes displayed by the outcomes. 11 We interpret ‘well-being’ as having multiple dimensions including satisfaction with the health, physical, emotional and financial dimensions of life.

19

5.1.1 Relative frequency of outcomes among males and females Not so predictable is the roughly even split between outcomes concerning behaviour changes in females and males. Disaggregating the outcomes by the gender of the social actor that changed shows that 24 of the outcomes concern changed behaviour of girls and women and 30 changes in boys and men (Table 4). There is clearly resonance and response with both genders.

Table 4 Relative frequency of outcomes by gender

Female Male

TOTAL 24 30

Mvomero 12 15

Kilwa 9 6

Tarime 3 9

Looking first at the theme of gender and football, it is interesting to note that not all football related outcomes were about enabling girls to play football. In a notable example of female leadership, a woman farmer in Kilwa led the formation of a male football team (Kilwa 12).

Several of the outcomes describing male changes of behaviour concern violence, including efforts by men to stop others from beating their wives (Tarime 1, 11 & 13), men who have stopped beating their wives (Tarime 2 & 10) and a man who sent his daughter to participate in the Catholic Church’s Tohara Mbadala programme that provides an alternative to FGM (Tarime 6).

Another set of outcomes concerns men and boys acting to resolve conflicts in the home by sharing responsibility for duties (Mvomero 11, Tarime 12), involving wives in decision making (Mvomero 12, 22, 24, 29) and taking time to learn with children (Mvomero 18).

Self-improvement of men features in an outcome in which a 26 year old man who re-joins a secondary school with the acceptance of his family in an area where madrasa schools are common (Kilwa 14) and another where a man has started to listen to his wife and stopped his habit of staying out late drinking (Mvomero 21).

5.1.2 SFCG’s 5 priority issues Through an assessment prior to the design of The Team project, SFCG has identified the following as the priority issues around gender equality in Tanzania that it is seeking to address:

1) Inheritance and women’s consideration in inheritance issues 2) Women’s leadership 3) Gender-based violence 4) Rape 5) Retention of girls in secondary school

20

Classifying the outcomes and proto-outcomes by these issues, we found a low number of related outcomes indicating that the changes represented by the results are generally not at the level of directly influencing these five issues (Table 5)12. However, although limited in number, there were some notable outcomes relevant to inheritance, leadership, violence and schooling; these are highlighted below.

Table 5 Relevance of outcomes and proto-outcomes to five priority issues

Inheritance and women’s

consideration in inheritance

Women’s leadership

Gender based violence Rape

Retention of girls in secondary schools

Outcomes 4 6 6 0 2 Mvomero 3 3 0 0 2 Kilwa 0 3 0 0 0 Tarime 1 0 6 0 0

Proto-‐outcomes 0 1 0 0 2 Mvomero 0 1 0 0 2 Kilwa 0 0 0 0 0 Tarime 0 0 0 0 0

5.1.3 Clustering of outcomes by location It is clear from the data that there is a tendency for outcomes concerned with particular issues or themes to originate from one location. For example, violence (Tarime) and retention of girls in school (Mvomero). These patterns reflect (a) differences between sites and hence differences in the issues of importance to informant groups; (b) a tendency of some informants to follow the example of others when presenting outcomes, perhaps because they are stimulated to do so, perhaps because they are not as articulate at thinking of examples as others; and (c) characteristics / specific interests of each group.

5.1.4 Financial benefits & inheritance A very high proportion of results - 20 outcomes and 1 proto-outcome - demonstrated changes contributing to gender equality that had financial implications. These fall into four groups: economic empowerment, sharing responsibility, inheritance and conspicuous consumption.

5.1.4.1 Economic empowerment Women starting a business growing and selling vegetables (Mvomero 9, 26; Kilwa 5), including a jobless woman now growing vegetables for sale and pursuing her interests by joining art and drama groups (Kilwa 9);

12 SFCG advised that we may have found results relevant to reducing rape if we had sampled in some of the other outreach locations.

21

Women reviving or re-committing to their businesses or business groups and using their profits for their interests, not giving everything to their husbands (Tarime 3 & 4; Mvomero 14, Annex D);

Husbands involving their wives in business, resulting in growing income and home improvements including construction of a well (Tarime 5, Kilwa 10).

5.1.4.2 Sharing responsibility Women being involved in decision making including family finances (Mvomero 2 & 12); a man giving money to wife (Mvomero 5).

5.1.4.3 Inheritance and women’s consideration in inheritance Assuming that sharing of responsibility, income and decisions on finances are steps towards recognising women in inheritance, then the previous three examples of outcomes should be seen as contributions to addressing the inheritance issue. Particularly striking is the example of the joint ownership of assets and division of income that followed a discussion of inheritance in a focus group (Tarime 5).

5.1.4.4 Conspicuous consumption A woman who made a substantial reduction in expenditure on clothes and jewellery (Kilwa 2, case study).

5.1.5 Schooling Retention of girls in secondary school is a priority issue for SFCG. We identified 3 related outcomes and 2 unrelated proto outcomes on this issue, all in Mvomero. Two of the outcomes are part of the same story: an elder sister fought with her parents (Mvomero 16) for her 14 year old sister’s right to go to school, a fight that she won (Mvomero 17). The third shows how the formation of a girls’ football team led to a dramatic rise in attendance (Mvomero 7 and Annex D)

The two proto-outcomes are striking as each concern women whose attitude to girls attending school has been dramatically changed: the first, Mvomero 37, no longer thinks it is right for parents to deny a girl the right to go to school; the second, has decided to save for her future children to go to school as she now realises that it is useful for everybody.

5.1.6 Participatory approaches to decision making and planning A striking set of outcomes exemplify ways in which The Team has encouraged people to try participatory approaches to reach their goals. Anecdotally, we suspect that the strapline used to promote the TV show – Better Together – has been highly effective. Numerous times informants talked about the show as The Team – Better Together, making the concepts inseparable. It may also be that this was a key message promoted by the facilitators.

Although the use of participatory approaches is not an explicit contribution to gender equality, we interpret the introduction of participatory processes as an achievement of the project that is consistent with promoting gender equality because participatory processes help to empower the marginalised. However, some of the outcomes exemplifying the

22

introduction of participatory processes, whilst seemingly positive changes, do not clearly relate to pre-defined objectives:

A female farmer who has been able to help resolve a conflict between her neighbours by using a consultative, participatory approach (Kilwa 13).

A male teacher who has introduced participatory approaches to his teaching (Mvomero 3).

Use of a participatory process to solicit community contributions to a school construction project with remarkable results (Mvomero 6; Annex D).

The application of participatory methods in football coaching that has improved results (Mvomero 13).

Other examples are in line with objectives:

A female community leader who has started to sensitise the community on the importance of planning together as families (Mvomero 10).

The leader of a widow’s groups who has started using a participatory approach to planning with significant income earning results (Mvomero 8; Annex D).

5.1.7 Football There were a total of 8 outcomes related to football. For instance, 6 outcomes from Mvomero show that The Team has (a) inspired and given confidence to girls resulting in them joining football teams and (b) inspired women and men to form girls’ teams and encourage girls to join. A remarkable consequence of one of these outcomes is the increase in school attendance of over 30% that has followed formation of a girls’ team at Hembeti Secondary School (Mvomero 7). That there are outcomes about girls playing football for the first time, a theme central to the storyline of The Team, is perhaps unsurprising. Nonetheless, the pace at which these changes have happened and the enthusiasm evident is notable although how long the motivation will be sustained remains to be seen.

5.2 Unintended results In addition to the 4 examples of participatory planning and decision-making described above, only two further outcomes or proto-outcomes do not conform to the theory of change or pre-defined objectives. Again, we view both as achievements as they appear to be positive changes:

Protecting community resources – Hedge planting was undertaken to protect school grounds from encroachment, as in The Team (Mvomero 4).

Self-improvement – one man reported having a renewed determination to progress in life, no matter what problems he encounters (Mvomero 33).

5.3 Negative outcomes Negative outcomes are typically under-reported because informants generally assume that evaluators want to learn about successes, negative outcomes may involve a host of

23

sensitive issues and organisations tend to under-report negative outcomes hence limiting the knowledge base available to evaluators. In this evaluation we succeeded in identifying only one negative outcome: a loss of freedom suffered by a school girl when her father took away his daughter’s telephone because he felt that it promoted bad behaviour (Kilwa 1). His action was promoted by seeing girls using phones to communicate with men in The Team. We do not think this is a particularly significant outcome; rather it is symbolic of the struggles between parents and teenagers that are common in much of the world.

We also learned of a potentially significant and negative type of change that was linked by several informants to The Team: the growing prevalence of ‘sugar mummies’, economically empowered women who have intimate relations with much younger men (Tarime 19). One woman recounted in detail the story of a woman over 50 who had a lasting sexual relationship with an 18-year old man. Another woman cited the very recent case of a husband who fled his house after his wife brought a young man to stay. Following a conflict with the husband, the youth beheaded one of the couple’s sons and fled. We did not classify either of these as outcomes as the precise link to The Team was not clear and we were not able to substantiate this possible outcome.

5.4 How The Team has contributed to changes in attitudes and behaviours

There will be many contextual factors – personal, financial, cultural, etc. – contributing to how important the ideas and role models introduced through The Team have been to realising the outcomes for particular people and organisations. Other factors may include existing attitudes and receptivity to ideas of the target groups and other gender equality initiatives past and present. We did not set out to assess the relative importance of the various contributing factors to the outcomes but did aim to characterise as precisely as possible the contribution of The Team to each outcome. In all cases it was possible to identify the component(s) of The Team intervention that had contributed to the outcome, whether it be the mobile screenings, TV shows, or other. In a number of cases, as we shall see below, it was possible to specifically link outcomes to scenes, characters or themes from The Team TV show and / or discussion of them during focus groups.

People in the control groups described just 4 outcomes and 3 proto-outcomes out of the 54 outcomes and 10 proto-outcomes described. Each of these cited seeing The Team on TV as the contribution The Team had made to their change. Although few in number, the control group results illustrate how The Team has had effects outside of the focus groups. Some sources were able to give particularly specific examples of what had motivated their change (Table 6).

For a very small proportion of results – 3 of the 65 outcomes and proto-outcomes - it was difficult to be confident about what aspect of The Team had contributed during the time available for informant interviews. However, we did in each case obtain the informant’s confirmation that The Team had contributed and an expression of the contribution that was not implausible.

The contribution descriptions of the great majority of outcomes and proto-outcomes indicate, unsurprisingly, that most responses to The Team are to be found among those receiving the most intensive engagement with the project: the target groups that viewed the mobile screenings and participated in the focus groups. More surprising is that none of our

24

informants identified the radio broadcasts or any other media such as Facebook through which The Team has been disseminated and discussed.

Table 6 Contributions of The Team to control group results

Summary examples of outcomes and proto-outcomes from the control group

Summary descriptions of how The Team contributed

Man who has reduced his drinking (Mvomero outcome 21) The notion of being open to ideas of women

Man who now involves his wife and children in planning and decision making (Mvomero outcome 22)

Discussions catalysed by The Team about GBV, violence against children and lack of involvement of women in decision-making

Woman who no longer thinks it is right for parents to deny a girl a secondary school education (Mvomero proto-outcome 37)

Portrayal in The Team of girls running away from home or getting pregnant or even committing suicide if prevented from enrolling in secondary school.

Woman who decided to save to send her future children to secondary school (Mvomero proto-outcome 38)

Seeing girls playing football showed the woman that anybody can do anything.

The illustrations in Table 7 give a real taste for how, exactly, the TV shows and associated FGD contributed to outcomes. They do not tell us, however, how important The Team contribution was to the person who changed. Estimating the importance of contributions is not a precise exercise. We asked each informant how important he or she felt The Team had been to the result they identified. We then categorised the contributions as: useful contribution, important contribution and very important contribution; the results are presented in Table 8. We defined these categories as follows: useful contribution – one that strengthened and / or brought forward changes that may have happened to some extent anyway; important contribution – one that helped realise a change that may not otherwise have happened or would have happened very much more slowly; and very important contribution – one that was either essential to the change or greatly accelerated it. The contribution importance results show a more or less even spread of outcomes for which The Team contribution was useful, important or very important.

25

Table 7 Contributions of The Team to focus group results

Summary examples of outcomes and proto-outcomes from the focus groups

Summary descriptions of how The Team contributed

Male farmer counsels neighbours the change attitude to beating women after having previously believed beating of women to be acceptable (Tarime 11)

Viewing the scenes in which Kalalu, a male teacher, harassed female teachers when drunk, together with FGD on GBV, changed this farmer’s beliefs about GBV.

A man who sent his daughter to participate in the Catholic Church’s Tohara Mbadala programme that provides an alternative to FGM (Tarime 6)

The Tohara Mbadala programme was discussed in FG after a screening.

Moribund women’s business groups revived (Tarime 4) Seeing the girls play as a team convinced the business group members they can do anything if they do it together.

Man stops beating his wife (Tarime 2) Friends of the man were inspired to advise him to use peaceful reconciliation approaches after episodes featuring GBV and FGD on negative consequences of GBV.

Woman substantially reduces expenditure on clothes and jewellery (Kilwa 2)

Seeing Waridi to leave school to marry a rich man to repay husband’s loan.

Woman farmer joins saving and borrowing group (Kilwa 4) Watching The Team and discussing it in FG helped her realise she also had economic responsibilities as a mother.

Male sub-village chair mobilises women to try various strategies to fight poverty (Kilwa 8)

Seeing how the teacher Wito used different strategies to achieve her goal showed him that confident women can succeed.

Wife convinces husband to enrol child in international school (Mvomero 1)

The determination and success of Wito in overcoming ridicule of men was inspirational.

Leader of widow’s group initiates participatory planning of income generation activities (Mvomero 8)

She learned about participatory planning and how it brings efficiency for the first time watching The Team.

Special seat councillor starts women’s group to support farming (Mvomero 9)

Watching the cinema screenings and participating in the focus group strengthened the councillor’s capacity in gender equality awareness-raising and she went on to use The Team DVD in this work.

Girl fights parents to allow her sister to go to school (Mvomero 16)

The episode about girls being taken from school for marriage and the FG discussion on girls’ identity motivated her.

Man consults wife and learns with children (Mvomero 18) Viewing and discussion taught him the negative consequences of chauvinism.

26

Table 8 Importance of The Team contributions to outcomes and proto-outcomes

Useful contribution

Important contribution

Very important contribution

Mvomero 9 10 17

Kilwa 6 3 6

Tarime 10 1 2

TOTAL 25 14 25

5.5 Conclusions The results were described from data collected during face-to-face group work and interviews with the social actors whose behaviour the project has sought to influence. These are the sources most knowledgeable about the results and it is therefore likely that they comprise at least the most significant results known to the informants consulted; and that they are at least indicative of the types of results to which the project has contributed in rural areas to date. Substantiation adds significantly to the credibility of all the outcomes.

Considered together, the results – outcomes and proto-outcomes - demonstrate that the rural outreach component of The Team has generally been very effective, particularly considering the short duration of the project, having contributed to numerous examples of changes in women, men, boys and girls, that either demonstrate or are relevant to the achievement of greater gender equality.

Assessed against the objectives of the intervention – the results, in particular the outcomes as they are observable hence more credible changes - demonstrate that the project was most effective at influencing behaviour changes at the individual level. Other than through the support it provided to its partner CSOs, the project has so far contributed relatively little to local and international initiatives – the third objective. It has also had relatively little influence on creating an enabling environment to advance female roles in decision-making at the community level, a change that can be expected to come later once individual attitudes and behaviours change.

The outcomes demonstrate changes relevant to all 5 priority issues except rape, an issue that SFCG did not expect to find changes in at the sites where we sampled. Common types of changes identified related to:

• Financial benefits and inheritance

• Schooling

• Participatory approaches to decision making and planning

• Female participation in playing and organising football

The Resonance and Response elements of the theory of change were, insofar as it has been described, clearly validated: all the proto-outcomes demonstrate Resonance and all the outcomes demonstrate Response.

27

We found that The Team TV show and mobile screenings and focus groups made an important or very important contribution to 39 of the results we described and a useful contribution to 25. For all but 3 of the results, we also succeeded in describing which part / scene / character / theme of The Team TV show or focus group discussion had contributed.

No informants cited The Team radio show as an influence on their behaviour or attitudes. This was unexpected as the radio shows were targeted at rural areas such as those where we sampled.

Over a very short time, the project has achieved many impressive results. Further potentially more significant results may well occur in the future, especially if any of the changes in behaviour prove to be lasting. For instance, a woman now given shared ownership of household assets may go on to inherit these assets.

Conclusions about how The Team’s rural outreach activities compare to the effect of The Team in urban areas will require the collection of data and further analyses beyond the scope of this evaluation.

28

6 Findings: Value for Money This chapter answers evaluation question 4: To what extent do selected outcomes imply that The Team Tanzania represented value for money?

6.1 Approach When deciding how to address this question, we were guided in part by the views of DfID and AcT. For AcT, “VfM can be described as a belief or conclusion by a buyer or seller of goods and services that the goods and/or services received were worth the price that was paid”13. Consistent with this, the AcT ‘how to’ guide to writing a VfM case study identifies both the qualitative as well as monetary values of an intervention to be relevant to a VfM assessment14.

DfID recognises that it is not always possible to measure value for money at the outcome and impact level. When embarking on this evaluation, we considered that this is likely to be the case for this short, 18 month project, that is seeking to influence changes that may very well take longer than 18 months to materialise and which are likely to be influenced by a range of factors in addition to the project. In such circumstances, DfID asks for “…good evidence that shows how what we are doing will lead to the intended outcome.”15

The outcomes we have described provide, we suggest, good evidence of how outputs have led to outcomes and a rich resource for understanding the qualitative value of the intervention in rural areas because:

i) They describe the kinds of (behaviour) changes that are likely to be necessary for the achievement of sustained higher level outcome / impact changes in gender equality;

ii) In many cases, we were able to identify specific outputs of The Team that contributed most to the outcomes.

Considering the qualitative values of outcomes or changes resulting from the outcomes we described, we found the following to be the main types evident in our data:

• Increased secondary school attendance of girls • Increased secondary school attendance of boys • Increased in women's communal savings levels for hard times • Increased gender integration in schools

We consider these changes to have significant qualitative value in their own right. Furthermore, we expect there are models that could be adapted to estimate monetary value

13 Dr Honest Prosper Ngowi (Mzumbe University) & AcT Programme (KPMG), Value for Money (VfM) of AcT Partners Results, December 2012.

14 How to write your own Value for Money Case Study, Accountability in Tanzania, 2013.

15 DfID’s Approach to Value for Money (VfM), 2011, p6. “Where it proves impossible to get sensible measures of value at the impact or outcome level then we need to make sure we are measuring inputs and outputs and have good evidence that shows how what we are doing will lead to the intended outcome”, Accessed 03.07.13 at https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/49551/DfID-approach-value-money.pdf

29

for most or all these types of changes. However, such an exercise was beyond the scope of this evaluation.

Challenged by AcT to go further, we set out to answer the following questions:

I. What is the Value for Money Effectiveness of The Team’s outreach activities in rural areas?

II. What is the relative Value for Money Effectiveness of The Team’s outreach and broadcast activities in rural areas?

III. How many monetisable outcomes of what overall value might one expect to find from the focus group participants as a whole?

6.2 Effectiveness of The Team’s outreach activities in rural areas The principal sources of the 64 outcomes and proto-outcomes we identified were individuals from our treatment groups i.e. people who had been participants in the mobile screenings and focus groups that The Team undertook in 12 rural districts. Some of the outcomes described were changes in the informants themselves; others were changes in the behaviour that informants had witnessed in others. The number of outcomes described per informant varied. Considering these characteristics of the data, three measures of effectiveness of this component of The Team are possible:

1. the percentage of treatment group informants that described results;

2. the number of results (outcomes and proto-outcomes) per treatment group informant; and

3. the total number of results that might be expected from all 12 of The Team’s focus groups.

From our data we provide answers for each measure as follows:

1. Percentage of treatment group informants that described results =

(Number of treatment group informants that described results / Total number of treatment group informants) X 100

= (40 / 64) x 100 = 63%

2. Number of results per treatment group informant =

Total number of results described by treatment group informants / Number of treatment group informants

= 53 / 64 = 0.83

This figure is likely to be an underestimate for two reasons: (a) harvesting of outcomes was undertaken with groups of informants so it is possible that with additional one-to-one time we would have been able to identify more results; and (b) some of our outcome descriptions could have been split into multiple outcomes but we chose to present them as they were presented to us in order to maintain the original context.

3. Total number of expected results for all The Team’s focus groups =

30

Total number of The Team focus group participants x Number of results per treatment group informant

= 840 x 0.83 = 697

Assumption 1: our informants were representative of their focus groups; see section 3.1.

Assumption 2: our treatment groups were representative of the other 9 focus groups; see section 3.1.

6.3 Relative effectiveness of The Team’s outreach and broadcast activities in rural areas

In addition to the results identified by rural participants in the mobile cinema and focus groups, we were able to describe results identified by control groups in two of the three districts where we sampled, Mvomero and Kilwa16. Comparison of the results from these control groups to those of the treatment groups provides a basis for measuring the efficiency of the outreach activities. As our first step, we calculate measures of the effectiveness of the project at influencing results in the control group. We then compare this effectiveness to that of the treatment group.

6.3.1 Effectiveness of The Team at influencing results reported by the control groups

From our data we provide two measures of effectiveness17:

1. Percentage of control group informants that described results =

(Number of control group informants that described results / Total number of control group informants) X 100

= (7 / 36) x 100 = 19%

2. Number of results per control group informant =

16 Control groups were comprised of target group representatives that had not participated in the mobile cinema and focus groups. Both of the control groups were comparable to our treatment groups: in each case, a similar number of people were selected by SFCG’s partners to form informant groups containing a gender-balanced mix of The Team’s four target groups.

The ‘control’ groups were not true controls as part of the The Team outreach strategy is that focus group participants should disseminate ideas through discussion with other members of their community. Therefore, some of those in our control groups may have been influenced indirectly by the mobile screening and focus group activities. Of those describing results, only a proportion (16/36) had seen The Team on TV and as none reported hearing The Team on the radio, The Team had influenced a proportion through interactions with focus group participants.

17 We are not able to estimate a number of results for potential control groups across all 12 districts where The Team implemented outreach activities because we do not have population data for the target groups in these locations.

31

Total number of results described by control group informants / Number of control group informants

= 8 / 36 = 0.22

6.3.2 Comparative effectiveness of the treatment and control groups To illustrate the comparative effectiveness of the mobile screenings and focus groups, we can compare the number of results per informant in the treatment group (0.83) with the number of results per informant in the control group (0.22). On this measure, The Team’s outreach activities were about four times as effective at producing results as the radio and TV broadcasts. However, this is not the whole picture.

Only 16 of the control group participants had watched one or more episodes of The Team TV show and all the results identified by the groups were provided by these participants. (Incidentally, none cited the radio show as an influence on a result.) Considering only those informants that had watched The Team, the number of results per informant was 0.5.

Our data therefore indicate that the mobile cinema and focus groups were more effective at contributing to outcomes and proto-outcomes than broadcasts alone (0.83 results per focus group informant against 0.50 results per informant who had viewed The Team but not participated in a focus group). Further sampling would be needed to determine if this difference is statistically significant. Regardless of the difference, our data indicate The Team probably had a measurable and potentially sizeable effect among those that watched the TV show. However, our sample size and the lack of precision in the responses we obtained are not sufficient to assess the significance of the focus group members, while noted by some in the control, at influencing control group results.

The comparison of results per informant does not tell the whole story. We found a much greater richness in the results and understanding of the issues addressed by The Team among the treatment groups compared to the control groups. In particular:

• We observed the treatment groups to be notably more animated and dynamic and

altogether better able to articulate the changes they had experienced or observed in

others18;

• Compared to treatment groups, the sources lacked the information needed to

describe the outcome more fully;

• Several treatment group informants were, unlike the control group informants, able to

describe outcomes that had catalysed or led directly to further, higher-level results;

• No control group outcomes indicated any monetary benefits, compared to 39% of

treatment groups’ outcomes.

18 Treatment group participants may be better able to articulate the outcomes having become accustomed through the focus groups to discussing issues about The Team. We were not able to test this hypothesis.

32

Because of these factors, we did not choose control group outcomes for any of the case studies.

6.4 Monetisable outcomes from The Team’s focus groups This section addresses the question: How many monetisable outcomes of what overall value might one expect to find from the focus group participants as a whole?

Our answer to this question is based on the monetary value we obtained for four of the outcomes we described. First we estimate the monetary value of these outcomes, then we use these findings and to estimate the total potential monetary value of outcomes for The Team’s 12 focus groups.

Each of these estimates involves making several assumptions in order to simplify the scenario to allow estimates to be made. We have stated those we have identified (section 6.7) but accept that there may be others that a fuller investigation beyond the scope of our Terms of Reference could reveal.

6.4.1 Monetary value of 4 outcomes We were able to obtain credible monetary data for 4 outcomes that we then wrote up as expanded case studies to gain a more in-depth understanding of their qualitative and monetary values (Annex D). For all 4, The Team contributed through the mobile cinema screenings and focus groups:

Mvomero:

Case study 4: One woman’s business grows as she gains confidence and ambition

Kilwa:

Case study 7: A woman’s reduces her conspicuous consumption to avoid debt

Case study 8: A widow’s enhanced self-reliance and financial freedom

Tarime:

Case study 9: A wife receives shared ownership of assets and income from husband

Using data supplied by the informant for each of the 4 case studies, we calculated estimates for the monetary value due to the contribution of The Team (Table 9). In each case, the monetary value due to The Team was a proportion of the total monetary because the informants indicated other contributing factors had also been relevant. For case studies 1 & 2 we either asked the source to estimate the percentage contribution from The Team or to indicate the contribution on a 1-10 scale. For case studies 3 & 4 we interpreted contribution percentage from the statements of the sources.

Our estimates for annual monetary value for these four case studies range from US$57 for a woman reducing spending to avoid debt that she has seen force others to marry their school age daughters, to US$1,434 for the increased income from egg production for a couple where the husband is sharing assets and income for the first time.

In addition to the case studies presented here, we learned about the impressive monetary and other values of the Hembeti school construction outcome (Mvomero 6). Though the

33

case is not readily modelled to estimate value for money, a simple presentation of the story is instructive because it vividly illustrates the remarkable inspiration The Team provided for mobilising community contributions to a school construction project.

Faced with the challenge of soliciting financial contributions from residents of 7 villages for the construction of 4 classrooms, Ward Education Officer Charles Kikullu, embraced a participatory approach to planning and organising with great effect. Previously, it had taken him 4 years to raise funds for building project. This time he has succeeded in just 6 months to raise over $30,000 in cash and in-kind contributions from residents, parents and students. Four classrooms have already been constructed and toilets and offices are to follow because the commitments have outstripped the requirements for phase 1 of the building works. The particular episode of The Team that was inspirational to Charles was when a businessman was trying to purchase school land without participation of teachers and the community. From this he understood that good leadership benefits from participation. His use of the participatory approach has inspirited others in the school and among the students and parents to do likewise.

Table 9 Monetary value of case study outcomes over one year19

TZS US$ Notes Case study 4: One woman’s business grows as she gains confidence and ambition

Additional profit due to The Team over one year

816,000 534 80% of total additional profit; outcome source estimated 80% contribution from The Team

Case study 7: A woman reduces her conspicuous consumption to avoid debt

Total saving due to The Team over one year

86,400 57 90% of total savings; outcome source estimated 90% contribution from The Team

Case study 8: A widow’s enhanced self-reliance and financial freedom

Total capital accumulation due to The Team over one year

360,000 236 60% of total capital accumulation; evaluators’ estimate a 60% contribution from The Team based on sources statement that The Team helped her gain confidence and courage more quickly than otherwise

Case study 9: A wife receives shared ownership of assets and income from husband

Total profit due to The Team over one year

2,190,000 1434 50% of total savings; evaluators’ estimate a contribution from The Team based on the source’s statement that The Team greatly contributed to the

19 Conversion from TZS to US$ made using oanda.com on 28.11.13

34

pace of change.

As with all the outcomes, there are also qualitative dimensions to the value of case studies; we summarise these in Table 10.

Table 10 Qualitative values of case studies

Qualitative values

Case study 4 Increased self-confidence and reference to the case as a role model by a Special Seat Councillor

Probably a sense of well-being from the security and freedom provided by the income

Case study 7 Reduction of stress in families and reduced temptation to marry off young daughters for financial gain

Increased potential for enrolling girls in secondary schools

Potential role model

Case study 8 Confidence to act when she had been despairing

Ability to save for child’s school fees

Ability to save for the construction of a house

Probably a sense of well-being from the security and freedom provided by the income

Case study 9 Sharing of ownership and income sets a precedent as it is very unusual

Probably a sense of well-being from the security of co-ownership and being valued for contributing in new ways to the household

6.4.2 Potential number and value of monetisable outcomes for all The Team’s focus groups

Assuming that the focus groups from which we identified the 4 case studies are representative of the focus groups in the nine other districts where The Team’s mobile cinema outreach was delivered, what would be the total number and monetary value of the outcomes resulting from all the The Team’s 12 focus groups in 6 regions20?

Potential total number of outcomes with monetary value that participants in The Team’s 12 focus groups could identify =

The percentage of the outcomes identified by the treatment groups which we evaluators consider suggest a monetary benefit to women e.g. starting a banana selling business x The total number of focus group participants =

20 The 4 case studies considered above came from focus groups in three of the twelve districts where The Team has held mobile cinema screenings and focus group discussions. While we sought to be representative when selecting the three districts we will only have succeeded in sampling some of the diversity of the twelve districts and a much, much smaller proportion of the diversity in Tanzania.

35

39% x 840 = 328 outcomes

The potential monetary value of the 328 potential outcomes =

Mean monetary value of monetised outcomes weighted by the mean contribution of The Team X Potential total number of outcomes with monetary value =

US$396 x 328 = US$129,888

6.5 Input to output ratio of the mobile cinema and focus group activities

To estimate these values we used the following variables (see Annex E for details of how A-C were calculated; D is data from SFCG):

A. The cost of the intervention’s outputs that contributed to the 4 case studies. This is comprised of the total production costs of 13 episodes of The Team TV shows + the cost of the mobile cinema screenings and focus group discussions. Cost of the intervention = $476,000.

B. The mean value for the monetary benefit described in the 4 outcomes over one year, multiplied by the estimated mean percentage contribution of The Team to the outcomes. Mean monetary value X mean contribution of The Team = $396

C. The percentage of the outcomes identified by The Team’s focus group participants which suggest a monetary benefit to women e.g. starting a banana selling business. Percentage of outcomes with a monetary benefit = 39%.

D. The total number of those who took part in the focus groups in the 12 districts. Number of focus group participants = 840.

From the values of A-D, we were able to estimate the ratio of financial inputs to outputs at 3.67. i.e. to derive $1 of value required $3.67 of investment (Box 2).

Box 2 Value for money estimate of The Team’s focus groups

Value for money (the ratio of financial input to output): 3.67

Calculated from $476,000 : $129,730

Calculation: A : E

where

A = $476,000 = cost of intervention TV shows, mobile screenings and focus groups. Costs were calculated from data provided by SFCG.

E = the total value of outcomes across the 12 focus group locations in one year.

E was calculated from B x (C x D) = $396 x (39% x 840) = $129,888.

where

B = $396: Mean monetary value of monetised outcomes X mean contribution of The Team

C = 39%: Percentage of focus group outcome sources that described an outcome that suggests a monetary value

D = 840: The total number of those who took part in the focus groups in the 12 districts.

36

6.6 Incremental cost of scaling up The estimates of the value for money of The Team overall (section 6.3) significantly overstate the incremental cost of extending dissemination and community engagement through focus groups with mobile cinema screenings or using other strategies. This is because the screenings and focus groups were only about 1/3 of the one-off cost of producing the TV shows (Annex F). Whereas each screening and focus group cost an average of $40,000 (1/12 of $476,000), any further screening and focus groups, each engaging 70 people, would now cost in the order of only $12,000 including partner direct costs, partner training, SFCG support and monitoring and participant logistic costs. Scaling up costs could be reduced if some of the informant suggestions (section 7.2 below) are followed such as doing the screenings in schools with follow-up discussions and having large outdoor screenings. Small group FGDs using trained community members could substantially increase efficiency and increase community ownership.

For an illustration, assuming a cost of US$12,000 for screening and convening focus groups to discuss all 13 episodes of series 1, it would take approximately 30 outcomes with a mean monetary value of $396 to “break even.” This of course, ignores less easily monetised outcomes, potential effects beyond one year and dissemination and multiplier effects.

6.7 Limitations and assumptions Each of the estimates above is likely to be an underestimate of the monetary value of the mobile cinema and focus group discussions and an even greater underestimate of the monetary value of The Team overall. The estimates do not take into account:

a) Potential longer term outcomes and impacts b) Monetisable value that could, with the application of appropriate models for increased

school attendance and other changes recorded, be derived from the qualitative values of the outcomes

c) Outcomes in urban areas resulting from TV broadcasts, radio broadcasts, festivals, Facebook or community radio. How much these components of The Team may have contributed to monetary value is not possible to determine in the absence of data on reach and the nature of any outcomes

d) The number of results per target group viewer in Tanzania’s rural areas as a whole, i.e. beyond the districts where outreach activities were undertaken, is likely to be lower because there will have been no influence of focus group participants.

e) The sample size for the control group was smaller than the treatment group: 36 controls vs 64 treatments; only 16 control group informants had watched the TV show.

A critical question when considering these estimates is: how representative are they? For instance:

How representative were The Team’s focus groups of the district’s where they are located? We cannot be sure. However participants came from multiple villages, are gender balanced and comprise an even representation of the four target groups.

37

How representative are the case studies of the focus group participants in the districts where we sampled? Here we can be more confident. In each of the three districts, we were able to work with gender-balanced groups of informants, evenly representing the four target groups and involving about one-third of all focus group participants. Our team is skilled in facilitation and was able to create an environment in which most people felt free to contribute. We therefore consider our sampling of and within the three districts to be representative of focus groups in these districts.

How representative are the 4 case studies of the 840 focus group participants in the 12 districts where The Team was implemented on the ground? We selected the three districts to collect data in order to be representative of the major social-economic and cultural difference between the north, central and southern zones of Tanzania where the focus groups of The Team were located. To this extent, the focus group participants and in turn the case studies we described are representative of the 840 focus group participants.

How representative are the 4 case studies of the target groups in Tanzania as a whole or of The Team audience in Tanzania as a whole? We evaluators do not have the information required to answer these questions.

Assumptions include:

• Financial data provided by informants and SFCG are accurate.

• Data provide by informants can be extrapolated to annual figures. In many cases we

are likely to have an underestimated annualised monetary gains, e.g. in cases of

increasing profit and capital we did not assume the profit and capital levels would

continue to increase at the same level but would plateau.

• No major inflation/deflation in the time period or changes in purchasing power.

• No negative opportunity costs – that increasing time spent in activity X resulted in a

corresponding financial or qualitative penalty in reduced time spent on activity Y. This

seems unlikely as the interviewees were asked about any negative consequences of

the changes and did not mention any.

6.8 Conclusions Our analysis describes value for money of the project to date in several ways:

• The outcomes demonstrate the project has contributed to the kinds of (behaviour)

changes that are likely to be necessary for the achievement of sustained higher level

outcome / impact changes in gender equality. Further, we have been able to make

detailed descriptions of how the project’s outputs have contributed to outcomes.

• The outcomes have several qualitative values, some and perhaps all of which could

be monetised with significant further modelling work.

38

• A very high proportion of treatment group informants – 63% - reported results. This is

markedly higher than the proportion of control group informants – 19% - reporting

results.

• The number of results reported per informant was higher in the treatment groups –

0.83 – than in the control groups – 0.22. Yet considering only those informants that

had watched The Team, the number of results per informant was 0.5.

• The outcomes we harvested from the treatment groups were qualitatively much

richer than those of the control group, suggesting the mobile cinema and focus group

discussions are more effective than broadcasts of the TV show alone.

• As our ‘control groups’ had intentionally been influenced by The Team’s focus group

participants, further investigations beyond the scope of this evaluation are needed to

examine the extent to which the TV broadcasts alone have influenced target groups.

• Based on our findings, participants across all 12 focus groups may be able to identify

in the order of 697 outcomes, 328 (39%) of which may be monetisable.

• We estimated the value for money of mobile cinema and focus groups expressed as

the ratio of financial input to output to be 3.67.

The incremental cost of scaling up the mobile screenings and focus groups is approximately US$12,000 / group of approximately 70.

It is likely that we have underestimated the monetary value of the mobile cinema and focus group discussions and greatly underestimated the monetary value of The Team overall.

39

7 Findings: obstacles and suggestions This chapter answers evaluation question 3: What were some of the factors (institutionally and within the country) that prevented objectives and expected results from being achieved?

In this chapter we describe the obstacles to achieving results that we were able to identify from discussions with informants and collection of data for the outcome descriptions. In addition, informants provided suggestions, which we summarise below, for how The Team or related initiatives could be made more effective.

7.1 Obstacles General challenges to addressing gender inequality in Tanzania are described in Chapter 4. In this section, we summarise the main obstacles we learned about directly through this evaluation. This is not intended to be a comprehensive assessment of obstacles external to or internal to SFCG and its implementing partners for The Team. We did not, for instance, set out to systematically dissect and identify lessons from management processes as might be expected for a management report. Nonetheless, we anticipate that the special privilege we had to learn from the informants will have identified topics for reflection and learning for those considering the effectiveness of The Team and those considering related work in the future.

7.1.1 Project implementation The project was planned to run for only 18 months and delays have compounded the challenges already set by such an ambitious timeline. The 18-month duration seems over-ambitious, particularly if there was an expectation that the project not only had to produce and disseminate its outputs but should promote learning from and sustainability of its achievements.

Implementation of the project was delayed such that it has proven necessary to obtain a no cost extension to December 2013. A major reason for the extension has been the change in SFCG Tanzania personnel during the project: the current staff are not those that designed and started the project. Such a lack of personnel continuity is, of course, not unique to SFCG. Compounding the delay caused by personnel changes was the initial baseline study that was found to be unsatisfactory necessitating a further baseline study.

Some concern was expressed by the three partners we engaged with in Mvomero, Kilwa and Tarime that delays in implementation were not communicated early enough to them to allow good planning with the focus groups. These partner organisations also reported that follow up from SFCG did not meet their needs for support with monitoring of progress in target groups. More support from SFCG would have been welcomed by the Tarime partner organisation, potentially reflecting concern of other partners located far from the SFCG offices in Dar es Salaam. Set against this is very positive testimony about the prompt and useful guidance current project staff have generally provided, particularly by phone and email.

7.1.2 Dissemination and content of the TV and radio shows Broadcasts of the TV shows were mainly available only to urban residents in Tanzania because of the lack of TVs in rural areas, an issue compounded by the recent analogue-digital switchover. Dissemination of messages to rural areas therefore depended largely on

40

urban residents sharing their impressions about the show with rural people and through Facebook viewing and discussion.

Some felt the radio show content was too safe; for example, they stopped short of scenes involving actual rape. In contrast, one participant in the Mvomero Teacher's control group who had seen The Team felt that the TV programme was too radical a shift for more conservative people to absorb: "It represents an abrupt shift rather than a gradual one. For modern people it is quite easy to understand but for the old people it is difficult."

Initially, SFCG had assumed they would get sponsorship for the radio broadcasts. However, no sponsorship was found so it was necessary to pay for radio broadcasts. Ideally, the show would have gone out on one or more stations with good reach in urban audiences as well as a national broadcaster. Because of the lack of sponsorship, the potential audience could not be maximised as airtime had to be bought. Radio did not reach all locations because Radio Free Africa and Tanzanian Broadcasting Corporation are not easily accessible everywhere.

7.1.3 Social issues Engaging women was, as expected, more challenging in some areas such as Mtwara than others such as Kilwa. In areas like Mtwara women were unable or unwilling to express their concerns regarding the gender issues either verbally or through questionnaire forms. Despite the challenges, we evaluators were able to obtain a good level of participation from female informants.

7.2 Suggestions for improvement Focus group participants (treatment and control) made a number of suggestions, summarised below, relevant to maximising the value of The Team or related initiatives in the future.

All informants were very much supported the idea of another TV season of The Team.

Draw more on local culture and context in programmes and facilitation

Facilitation questions could be improved by relating them to the local context and build the discussion up to the target issues.

Greater resonance may be achieved by drawing on local events such as the practice of women sponsoring son in-laws to marry women to give birth to male children on their behalf to sustain continuation of a family’s lineage.

Open the mobile cinema & FGDs to a larger group

This was a universal plea from the informant groups. People cited the use of outdoor broadcasts in campaigns against HIV and malaria as examples to be emulated.

Integrate showing of The Team with other events

It should be accompanied/preceded by other entertaining activities like a sports competition so as to attract more people.

Organise events (showings, etc.) around girls’ sports competitions, dialogue, debate, various things.

41

Dissemination

● Make the DVDs more available ● Produce DVDs for dissemination to people in Tarime district. ● Identify community facilitators to disseminate materials ● Form The Team Dissemination groups ● Show the programmes in schools ● The pioneer The Team group/club should be formed, empowered and supported to

reach communities cannot access TV and radio facilities. The support entails having large TV screen, display equipment, generator, fuel, transport, identification materials (e.g. T-shirts and caps). Outcomes for the youth.

● Support a group to disseminate The Team to other wards. ● More training and support to partner organizations to enable them disseminate the

themes of The Team to remote 99 village of 30 in District of Tarime. ● Tee shirts, caps and footballs were much appreciated.

42

8 Lessons learned and recommendations for discussion This chapter answers evaluation question 5: Based on the evidence of the evaluation what are the lessons learned and recommendations for discussion to improve programming in Tanzania?

Rather than recommendations, we here provide recommended points for discussion by The Team actors. We have deliberately not sought to recommend what SFCG and its partners should do next. Decisions on next steps for SFCG and its implementing partners to take will be informed by the findings and conclusions of the evaluation, but equally will be informed by political, legal, public-perception, financial, programmatic, and ethical considerations which sit beyond this evaluation. In addition, this evaluation has been focused on assessing effectiveness of The Team. By design, it has not sought to assess the quality of processes used by SFCG and its implementing partners, including how they have worked together. Thus there is a constraint on the scope for identifying areas of learning. However, we expect that the process of reaching the evaluation findings and conclusions will have resulted in some observations which may be of use when considering any future Team or related programmes in Tanzania and potentially elsewhere.

The following lessons and recommendations for discussion our based on our reflections on our findings, the data and discussions with informants in communities, partners and SFCG.

Potential for scaling up using existing materials for screenings and focus groups

Lesson 1a: We found indications that The Team has been particularly effective when mobile screenings are coupled with focus groups, though our sample size was not sufficient to reach a conclusion on how much more effective focus groups are compared to TV broadcasts alone.

Lesson 1b: With the TV shows already produced, there is considerable potential to work with communities not yet reached for a reduced incremental cost / participant and benefitting from the lessons learned to date (Section 6.6).

Recommendation 1.1: Undertake research on the relative effectiveness of the focus groups compared to TV show viewing.

Recommendation 1.1: Focusing further work in rural areas may achieve most because of the limited rural access to broadcasts to date. However, further work in urban areas should also be considered because it is likely to be most cost effective in terms of numbers of people that can be engaged.

Recommendation 1.2: Review suggestions from project participants (Section 7.2) for improving the content, focus group discussions, and dissemination of the TV shows.

Choice of media

43

Lesson 2a: Our findings indicate that The Team TV show was popular and those who saw it - all informants who had seen the show wanted to see a new series produced.

Lesson 2b: Our findings indicate that The Team radio show had a limited reach, resonance and results which was a surprise given the popularity of radio as a media to disseminate messages in Africa.

Recommendation 2.1: Investigate the costs and benefits of producing a second series of The Team compared to promoting further responses from the first series.

Recommendation 2.2: An assessment of resonance and response amongst those that heard the radio programmes but did not see the TV show is needed to understand the effectiveness or otherwise of the radio programmes.

Support for other gender equality initiatives

Lesson 3: The outcomes indicate that support for other initiatives through The Team – Objective 3 - has so far been limited to village-level women’s and other community groups.

Recommendation 3: Consider sharing this report and the outcomes data with relevant organisations nationally and internationally to stimulate discussion and learning.

Sustainability

Lesson 4a: The response-level (behaviour) changes we described are very positive indications that the project will have a lasting effect. How lasting the effect is and how far if at all the changes recorded will catalyse further steps towards gender equality will only be known if follow up work and monitoring is undertaken by partners, SFCG or others.

Lesson 4b: Achieving gender equality is a long-term process, consideration of what comes next is of great importance to ensuring the intervention has long-term value.

Lesson 4c: The SFCG proposal to DfID did not contain any consideration of sustainability of results.

Recommendation 4: SFCG initiate as a priority the development of a strategy for sustaining and building on the results achieved to date. This may best be developed with its partners, DfID and other relevant stakeholders.

Monitoring, evaluation and action learning

Lesson 5: This evaluation demonstrated that seeking outcomes directly from social actors influenced by an intervention can be highly effective at describing resonance and response results.

44

Recommendation 5.1: SFCG considers updating the 3Rs guidance document on response to recognise the value of using OM-inspired M&E approaches even when country teams have limited experience with OM and /or limited time to support data collection.

Recommendation 5.2: SFCG considers updating the 3Rs guidance document on resonance to reflect that it is not only quantitative approaches that can be useful: qualitative approaches as used in this evaluation are also very valuable.

Lesson 6a: Examination of SMS monitoring revealed indications of attitude and awareness and selected Facebook monitoring data revealed changes in attitude and awareness. Neither these, nor any other report on outreach, social media, broadcasts, contained information that could be used to describe outcomes.

Lesson 6b: Partners consulted indicated they would have liked support in monitoring the effectiveness of the focus groups.

Recommendation 6.1: Considering the need to collect data from the outset of the project that can later be used for mid-term and final evaluations.

Recommendation 6.2: Consider using outcomes harvested through monitoring and other monitoring data for near real-time adjustment of intervention strategies, between partners and as the basis for periodic self-evaluation.

Recommendation 6.3: Consider how to improve information flow from the field using data collection tools that that are carefully designed to optimise integration with workflow and are usable with minimal, largely remote support.

Lesson 7: Data and resource constraints limited the scope of this evaluation to the effectiveness of rural outreach component of The Team, prevented assessment of the Reach element of the theory of change and limited the potential to assess its value for money.

Recommendation 7.1: Consider regularly obtaining data on the Reach of The Team in any future work as this is critical for a full understanding of the intervention’s effectiveness.

Recommendation 7.2: Reconsider the budget available for evaluations.

Design of future work

Lesson 8: The changes we have identified do appear to be valuable contributions to gender equality but how they may contribute to the 5 priority issues is not clear in many cases. On the one hand, outcomes go beyond predefined objectives, on the other they fall short of achieving results at the level of the 5 priority issues.

Recommendation 8: Consider developing a theory of change that includes the pre-defined objectives and 5 priority issues and articulates how a future intervention can build on the types of changes we have recorded to achieve results at the level of the 5 priority issues or similarly higher-level results.

45

Lesson 9: Other actors, not least the project’s partners, will have existing experience and know-how and / or related on-going programmes yet this information is not described in the project documents hence is not available for project design.

Recommendation 9: Include descriptions in project documents and monitoring data of how others, including partners, are anticipated to contribute to the results sought by the intervention. This may assist programme conceptualisation and inform management decisions.

Annex C Tarime short outcomes

NoTreatment / control

Outcome description (1 sentence: who has changed, in what way, where, when)

Contribution statement (what from The Team influenced the change)

OutcomeAnecdote (i.e. potentially an outcome but lacking specific or detailed data)Proto-‐outcome (i.e. an attitude change only)Observation (reinforcement of a proto-‐outcome)

1 Treatment

Following The Team Marwa Mirumbe, one of the elders (Mzeewa Mila) of Tarime and his friends advised his neighbour Marwa Weibina against his practice of regularly beating his wife.

The negative effects of GBV were highlighted in The Team episodes the focus group discussions. Alternative gentle approaches to reconciliation such as consultations with elders were discussed. Outcome

2 TreatmentFollowing The Team and up to October 2013 Marwa Weibina had stopped beating his wife.

After The Team, Marwa Mirumbe and his friends and advised Marwa Weibina against beating his wife. This followed the Team episodes and the focus group discussions in which the negative effects of GBV were highlighted. Alternative gentle approaches to reconciliation such as consultations with elders were discussed. Outcome

3 Treatment

Following The Team several women (three named examples were given) have become increasingly involved in the banana sales business and have instead of the money being used exclusively for the benefit of men, they have retained the income generated from the business.

Both the women as well as men who participated in The Team mobile cinema screening realised that women can practice business for their own benefit besides the traditional gender roles of caring for the family, rearing children, household chores and farming, and have adopted a different attitude towards women’s participation in family and community work. Outcome

4 Treatment

Following The Team several previously moribund women's business groups in Mogabiri have been revived through an increase in activity of its members. This has led to increased economic empowerment and decreased conflict within families.

The women who viewed The Team and took part in FGDs saw the capacity of girls to play football as a team as an indication that women are capable of doing anything if they work as a team. Outcome

5 Treatment

Following The Team Mairo Keria Mugesa has involved his wife in decision-‐making and joint ownership of their business. Their dairy cow is now under joint ownership, they bought 200 chicks together in July 2013, constructed a well jointly from July 2013 and have managed to build an improved home.

After The Team (specifically the incidences when the female teacher being ridiculed by male teachers and even male students) and the FGD in which women's right to own property was discussed, Mairo could see the potential socio-‐economic benefits of involving his wife in decision-‐making and ownership. Outcome


6 Treatment

Following The Team Joseph Marwa Gesara sent his daughter Happiness to Masanga village (45 km from their home village of Mogabiri) for the “Alternative FGM” (Tohara Mbadala) programme of Roman Catholic Church which enables girls to get married without stigma.

The Tohara Mbadala programme was discussed in the FGDs that followed from the mobile cinema showing of The Team when the group brainstormed on ways forward to address the challenge of gender violence including female genital mutilation Outcome

7 TreatmentFollowing The Team Pastor Ibrahim Pete began to preach against violation of children’s rights during his church services in Kibumaye.

Pastor Ibrahim saw that The Team highlighted issues such as some parents forcing girls to get married and adults sexually harassing girls. Outcome

8 Treatment

From April 2013 Pastor Tobias OluochI of Mogabiri fought against a man who had illegally built upon the Kende secondary school playground. The man was finally ordered to demolish the building in October 2013. To be completed: response pending ??

9 Treatment

Following the Team, Tobias Chacha, a teacher at Mogabiri Primary School, has facilitated the interaction of girls and boys in the process of learning life skills.

Viewing girls play football in The Team was an eye opener for Tobias and was the main inspiration for him taking the initiative to integrate girls and girls in this way. Outcome

10 Treatment

Since July 2013 and up to October 2013, Mwita Jacob has stopped beating his two wives following counselling given by his sister Dorith J. Range, a teacher at Kibumaye Primary School.

Dorith participated in the focus group discussion during which the participants brainstormed on a number of issues related to women’s empowerment and made reference to domestic violence within the Kulya community. Outcome

11 Treatment

Following The Team Sabai Lameck, a male farmer, has counselled neighbours and teachers to change their attitude towards the beating of women.

Sabai came to the realisation that beating women is a human rights’ violation. But prior to seeing the scenes from The Team in which the male teacher (Kalalu) harassed female teachers when he got drunk, and taking part in the follow-‐up discussions on gender based violence, Sabai had believed that the beating of women was an acceptable practice. Outcome

12 Treatment

Following The Team Rose Thomas, a 19 year old ex-‐student of Mogabiri Secondary School, has encouraged her brother Marwa Thomas, a student to participate in household and farm duties that had traditionally been allocated exclusively to women. In response Marwa has been participating in these tasks.

The mobile cinema screenings and the FGD contributed to this change because they reduced the risk of a boy being ridiculed by the parents and elders for undertaking “girls tasks.” Outcome

13 Treatment

Following The Team Elias Chacha and his friends advised his neighbour Marwa Weibina against the practice of beating his wife time and again and now living happily with his wife.

The episodes of The Team on gender violence related to men’s bullying and the male teacher’s threats to female teachers triggered a discussion about women-‐beating practices among the Kulya tribe. Outcome


14 Treatment

Following The Team more women than before have shown an interest in contesting for leadership roles in the coming year’s local government elections.

The episode that displayed Mwl Wito’s leadership talents for initiating changes in the school by introducing the girls’ football team and the supportive role of the Head teacher has helped to inspire Kulya women to take leading roles in undertaking socio economic activities like banana sales business and community work including leadership. Anecdote

15 Treatment

Following The Team, the male participants in the FGD further realised that women in the Kulya community could do things other than domestic household chores.

From the Team episodes the men saw girls who were initially prohibited by parents from playing football but then saw that the girls were very capable of doing something that traditionally had only been done by boys. The contribution of the FGDs were also crucial. Proto-‐outcome

16 Treatment

Following The Team Marwa Weibina has given his wife the freedom to sell bananas for long periods of time away from home unlike in the past when she could only stay go away from her home for a specific time earmarked for the purpose by Marwa.

The FGD on GBV, specifically discussions about women being beaten up when they came home late was a major contribution. Anecdote

17 Treatment

Married women without male children have been able to maintain their social status in the family by adopting the Nyumba Mboke approach. With the power of money earned from business, they sponsor the in-‐law youths to get married to a girl so that the couple give birth to male child for the woman who then owns the male children to regain entitlement to family property inheritance through the son(s). This same approach applies to women who have had her son die before getting married. An example of a youth (name withheld) currently working as a lawyer was cited to support the occurrence of such cases. N.A. Observation

18 Treatment

Following The Team, the male participants in the FGD have further realised that FGM of girls in the Kulya community is an unacceptable practice.

The topics of gender violence and discrimination from participation in economic as well as social activities was raised during the discussion of one of the episodes. Observation


19 Treatment

The incidence of older women (known as sugar mummies) attracting boys and young men for sexual relationships seems to be on the increase in Tarime with growing economic empowerment of women as the result of business undertakings, in particular the banana business.Witness Johanes recounted a recent story (that happened after The Team) to illustrate the situation. Agnes Marwa, a woman of over 50 years with a successful banana business, attracted an 18 year old man into a sexual relationship and both adamantly refuse to separate despite the repeated appeal of the man’s parents. Another example was cited by Witness Johanes. Derifina Magoiga the owner of a banana / liquor business, and wife of Magoiga, attracted a youth from the village of Mjalelo about 20 km away. The two lived together for a year while the husband fled the house. A week ago (mid-‐October 2013) the conflict between the husband and wife erupted and in the struggle the youth beheaded one of the couple’s sons and ran away. At the time of the interview the youth had not been apprehended.

N.A. Observation

Annex B Kilwa short outcomes




OutcomeAnecdote (i.e. potentially an outcome but lacking specific or detailed data)Proto-‐outcome (i.e. an attitude change only)Observation (reinforcement of a proto-‐outcome)Statement (original interview)

1 Treatment

From the beginning of the second school semester in July 2013 Shaban Mohammed, a male prison officer decided to take away the phone he had bought for his daughter so that she concentrates on her studies as he felt that phones promoted bad behaviour among girls who were not concentrating on their studies any more.

From seeing The Team Shaban learned that Sophia and her sisters were very free and they could use phones to communicate with men who were hanging out with. Outcome

2 Treatment

Following The Team Amina Msham Mwangu, a 38 year old woman has substantially reduced her spending on new clothes and jewellery for every celebration. New items used to cost a minimum of TShs 120,000.00 per month but Amina's demand has now decreased to about one-‐third of that.

Amina realised the negative effects of her former attitude from seeing the mother of Sophia who was forcing Waridi, who was at school, to get married to a rich man to get money to repay her husband’s loan. Outcome

3 Treatment

Following The Team, Fatuma Said Kindamba (a female farmer) collaborated with two other colleagues to organise community meetings to mobilise youths to revive the men's football team which was reformed in June 2013. It is now doing well in the District Level Football League.

From The Team Fatuma learned the importance of group cohesion -‐ in order for any group (being a family or community) to succeed, it is important to behave and work as a single group and not a larger group with subgroups. Outcome

4 Treatment

In May 2013 Fatuma Said Kindambaa, a 37 year old female farmer joined a community credit saving and borrowing group (VICOBA) where she can save and borrow easily.

Watching The Team and taking part in the FGDs helped Fatuma to understand her economic as well as social responsibilities to her family as a mother. Outcome

5 Treatment

In June 2013 Mwanahamisi Issa, a 37 year old widow, started a business selling vegetables with capital of TShs 50,000 borrowed from her brother in law. Today her capital is worth about TShs 250,000 and she has been able to buy blocks for a new house and is saving for her child to go to secondary school.

From watching The Team and participating in the FGDs Mwanahamisi saw the way teacher Malaika Wito went through difficult circumstances but finally succeeded. The Team helped to give Mwanahamisi the confidence and courage to get something done. She had done business before but not with this level of determination. Outcome


6 Treatment

Following The Team Mwanaisha Sheweji a 42 year old housewife has been able to bring her neighbours together to talk about their differences which has helped them to forgive each other and resolve their conflicts.

Mwanaisha learned from the scenes involving Mama Sophia and Mama Upendo who had conflict to deal with. Outcome

7 Control

Following The Team Issa Magambo observed the women who have been encouraged by Twalah Mbwangali, the 47 year old male Chairperson of Miembe Miwili sub-‐village to take part in income generating activities are spending their income on collective family needs.

From The Team both men and women came to understand their roles as fathers and mothers of the families -‐ that everyone is accountable to their spouses and their children. Anecdote

8 Treatment

Following The Team, Twalah Mbwangali the 47 year old male Chairperson of Miembe Miwili sub-‐village started to mobilise women to use different strategies to fight poverty, for example through engaging in income generating activities such as trade.

When watching The Team Twalah saw how teacher Wito used different strategies to achieve her goal and could see how confident women can succeed. Outcome

9 Treatment

Following The Team Dhamira Athumani who was jobless young lady began to grow vegetables which she is now selling. She has also joined a theatre and performance art group where she is learning drama.

Dhamira Athumani was one of the women that Twalah Mbwangali (the 47 year old male Chairperson of Miembe Miwili sub-‐village) counselled to use different strategies to fight poverty. This action was the result of watching The Team and seeing how Miss Wito used different strategies to achieve her goal. Outcome

10 Treatment

Following The Team Ibrahim H. Fereji, the 68 year old male Chairperson of Mpara sub-‐village, now works together with his wife in income generating activities including charcoal selling, farm work and fishing and takes care of his family as a responsible father.

Following The Team Ibrahim came to understand the shared responsibilities of both men and women in family life. Outcome

11 Treatment

Following The Team Shomari Abdul has changed his attitude to work and now involves himself in wage earning activities to earn income for the family -‐ particularly loading baggage onto ships port loading, loading and offloading luggage from lorries and salt harvesting.

From The Team Shomari came to understand the shared responsibilities of both men and women in family life and changed his attitude to work. Outcome

12 Treatment

Following The Team Zainabu Said Zuberi, a 33 year old female farmer led the formation of a men's youth’s football. The Team, called Glasi was formed on 13 July 2013 and it is now participating in District competitions.

Through The Team Zainabu saw Miss Wito’s consultative approach to leadership that gave her the confidence that she could interact with men in this way without being suspected of adulterous behaviour. Miss Wito’s behaviour taught Zainabu that everything is possible if you are really determined to go for it. Outcome


13 Treatment

Following The Team Zainabu Said Zuberi, a 33 year old female farmer changed her approach to dealing with conflict in the family and with neighbours to a participatory approach that involves being humble. By adopting this approach she was able to help resolve a misunderstanding between her neighbours.

Through The Team Zainabu saw Miss Wito’s consultative approach to leadership. Outcome

14 Treatment

In April 2103 Hamidi Mohammed Khalufa (a 26 years old male) rejoined Kilwa Day Secondary School to obtain a good education for a better life free from issues like the urge to beg or the temptation to push drugs, abuse women, steal or roam the streets.

Through The Team Hamidi changed his perception of the importance of education and saw how many evils are due to limited education. This confirmed his aspiration to be a teacher. Outcome

15 Treatment

Following The Team Hamidi Mohammed Khalufa (a 26 years old male) who had rejoined Kilwa Day Secondary School in April 2013 mobilised fellow youths, both boys and girls, to fight for gender equality by engaging them in group discussions in evening tuition sessions.

Through The Team Hamidi became aware of the importance of gender equality between girls and saw how many evils (e.g. sexual harassment and promising girls for marriage at an early age) are due to limited education. Outcome

16 Treatment

Following The Team the children of Mwanaisha Sheweji a 42 year old housewife are showing much more respect towards each other than before.

Mwanaisha used the knowledge she had gained from The Team episodes and FGDs to educate her family about respecting each other. Anecdote

17 Control

Following The Team Issa Magambo observed that many women have been encouraged to take part in income generating activities such as selling fish.

From The Team both men and women came to understand their roles as fathers and mothers of the families -‐ that everyone has a role to play in the family for the welfare of the family. Anecdote

18 Control

After watching The TEAM on TV Ashura Said Likumbage learned that conflicts are inevitable in the communities. Some conflicts are very necessary for development. The good thing is to solve the conflicts and live in peace. The Team on TV Proto-‐outcome

Annex A Mvomero short outcomes




OutcomeAnecdote (i.e. potentially an outcome but lacking specific or detailed data)Proto-‐outcome (i.e. an attitude change only)Observation (reinforcement of a proto-‐outcome)

1 Treatment

Following The Team the husband of Joyce Peter, a female teacher, agreed to his wife's request that they enrole their four year old daughter into the international school next year.

One of the characters (Mwalimu Wito) was ridiculed by fellow teachers and male pupils who said that she couldn’t make it but she finally sorted out the situation. This proved to be an inspiration to Joyce. Outcome

2 Treatment

Following The Team Kasi Dawi (a male teacher) is now working together with his wife in decision making, including financial decision making as exemplified by their recent joint decision to buy a plot of land on which to build their house.

During the screening of The Team Kasi saw men and women teachers planning to sell off school grounds but the woman teacher (Mwalimu Wito) consistently fought against this and the playground was saved. It demonstrates that women can have good ideas and that they can transform a man. The FGD helped reinforce the message of the TV programme of the importance of respecting women's ideas. Outcome

3 Treatment

Following The Team Selemani Rukonge (a male teacher) has now introduced participatory decision-‐making processes in his teaching activities to ensure that conflicts are avoided.

Selemani learned about the misuse of power through the incident in The Team programme when a business man tried to bribe pupils not to follow up when he was trying to expropriate school land. He learned this through the show and the discussions that followed. Outcome

4 Treatment

Following The Team Magorromary Victor (a male teacher) has started to plant live hedges around school grounds to protect them from development and is trying to encourage other schools to do the same.

One of the episodes of The Team showed a business man trying to grab land from the school compound. In the discussions that followed the issue of encroachment was discussed. Magorromary proposed a solution to this was to plant live hedges because most open school areas are under threat of encroachment. Outcome

5 TreatmentAfter watching The Team on TV with his wife Sulaiman Mwinyi (a community leader) is giving money to his wife for purchases.

From watching The Team, especially the actions of Miss Wito who pursued the idea of forming a girls’ football team in spite of ridicule, Sulaiman realised that when women are empowered they can do many things. Outcome

6 TreatmentFollowing The Team Charles Kikullu initiated a participatory planning process for community contributions to school construction.

In The Team Charles saw a business man trying to purchase the school land without the participation of the teachers and the community. He learned from this that there should be participation for good leadership. The TV show was the main contribution. Outcome

7 TreatmentSince April 2013, girls have been training for the newly formed girls’ football team at Hembeti Secondary School.

From watching The Team Charles learned that girls could play football and not just netball. The TV show was the main contribution. Outcome


8 Treatment

From June 2013 Tatu Mbonde the leader of the Mvomero widow's group has initiated a participatory planning process for the group to help them to earn income from agriculture and save to help members in trouble.

Before watching The Team and being involved in the focus group discussions Tatu and her group had no idea of planning together at all. From watching The Team and the discussions that followed, Tatu learned that teamwork among community members brings about efficiency. Outcome

9 Treatment

In May 2013 Odilia M. Lugendo, a Women’s Special Seat Councillor in Mvomero, started a women’s group called Kumekucha whose 10 current members are engaged in sunflower and padi rice farming.

Odilia watched the mobile cinema screenings of The Team and participated in the focus group discussions. She has shown The Team DVD as a tool to raise awareness about the negative impact of relying on marrying off school girls to earn income, financial dependency on husbands and also to help members conduct participatory planning within the family. Outcome

10 Treatment

From March 2013 Tatu Mnyali, a farmer and community leader from Mvomero, has started to sensitise the community on the importance of families planning together.

Tatu watched the mobile cinema screenings of The Team and participated in the focus group discussions. These inspired Tatu to introduce the concept of planning together using scenes from The Team to facilitate learning about the fact that women sacrifice a great deal for the family to the detriment of their welfare when there is no planning together. Outcome

11 TreatmentFollowing The Team Elena Daudi’s husband has helped with household duties and apologised for his lack of help before.

The scenes in the TV programme showing the inequality in roles between men and women and the group discussion influenced Elena’s husband. Outcome

12 Treatment

Following The Team Saidi Shaha and his wife started to make family decisions together including those regarding finances and the welfare of their 13 year old daughter. The Team has helped Saidi and his wife learn and change how they live. Outcome

13 Treatment

Following The Team Saidi Shaha, a football coach, has improved his decision-‐making process by using participatory methods which has resulted in a better level of performance.

Saidi observed the participatory decision-‐making approach used by the teacher in The Team and has been able to adapt it. Outcome

14 Treatment

Following The Team Semeni Kilongoro, a shopkeeper in Mvomero, has increased her confidence, hard work, and concentration, resulting in greater monthly profit levels which in September were over five times what they had been before her participation in The Team process.

The Team inspired Semeni a great deal, changing reversing her attitude that business was meant for men only. Through watching The Team she increased her confidence. Both the viewing and the subsequent discussion helped. Outcome

15 Treatment

Following The Team Semeni Kilongoro, who is involved with a sports club for orphans aged 6-‐13 (UMOJA), decided to start involving girls in football and having girls and boys play football together. Previously boys and girls played separate sports.

The club was already in operation but The Team has added the dimension of playing together. Outcome

16 TreatmentFollowing The Team Amina Daudi fought with her parents for her sister’s right to attend year one of secondary school.

Watching the aspect of The Team TV programme of women trying to pull girls from school for marriage and the discussions on girls’ identity helped Amina to realise that girls have unique opportunities for schooling and motivated her to fight for her sister's right to attend school. Outcome


17 TreatmentIn April 2013, Amina Daudi's parents allowed her 14 year old younger sister to attend one year of secondary school.

Watching the aspect of The Team TV programme of women trying to pull girls from school for marriage and the follow-‐up discussions on girls’ identity helped Amina to realise that girls have unique opportunities for schooling. This motivated her to fight for her sister's right to attend school. Outcome

18 TreatmentSince June 2013 Abed Majid now consults with his wife, talks to his children and learns together with them.

Through the viewing and the discussion Majid learned of the negative impacts of practices such as male chauvinism which leads to conflict between the wife and husband which affects the children through a lack of proper parenting. Outcome

19 TreatmentSince June 2013 Mariam Shukuru has started to play football for a team and now has ambitions to play for the national team (Twiga Stars) as a goalkeeper. Mariam learned through the TV show that women can play football. Outcome

20 TreatmentFollowing The Team Ashura Said has overcome her shyness and started to play football for a team.

The particular aspect of The Team that inspired Ashura was the determination of the teacher who only started with one girl but encouraged others to join. Outcome

21 Control

After watching The Team on TV Lenius Mkude has stopped his habit of drinking and coming home at midnight on the advice of his wife and is now attending prayers and learning sessions.

The TV show had influential aspects including the story of land grabbing and the notion of being open to ideas of women. Outcome

22 Control

Following discussions with people who had seen The Team Mohammed Mdenya is now involving his wife and even his children in planning and decision-‐making while formerly he used to dictate.

Through discussions catalysed by The Team Mohammed was able to speak about issues such as GBV, violence against children and lack of participation of women in decision-‐making and grew to understand the importance of involving women and even children in decision-‐making. Outcome

23 ControlAfter watching The Team on TV a group of teachers formed a girls’ football team for 10-‐15 year olds at Hembeti Secondary School. The teachers watched The Team on TV and observed the girls playing football. Outcome

24 ControlFrom about April 2013, after watching The Team on TV Verdiana Sanga’s husband is now sharing plans and valuing her ideas.

Verdiana’s husband watched The Team on TV and told her that it dealt with gender but did not go into detail. Outcome

25Other (SFCG Partner)

Following their involvement with The Team as a SFCG local partner The Mvomero Organizations Coalition (MOC) has changed the way it has dealt with gender equality, by involving women much more and from February 2013 engaging a woman member of staff (Elizabeth Priscus) on contract to coordinate gender issues.

From participating in The Team process (coordination of mobile cinema screenings and facilitated discussions in Mvomero and Kilosa) Stanford Kalala, MOC’s Executive Director has learned that gender-‐related challenges in particularly inheritance are common to all communities. From the process Stanford is now confident of being able to deal with such issues. Outcome


Following his organisation’s involvement with The Team as a SFCG local partner, The Mvomero Organizations Coalition (MOC) Executive Director Stanford Kalala is now involving his wife in decision-‐making and she has started a vegetable garden which is saving the family about Tsh4,000 per week.

Through his involvement in the Team process Stanford realised the importance of involving women in decision-‐making. He was particularly inspired by the head teacher who gave Miss Wito a chance to try out her ideas. Outcome



Following his organisation’s involvement with The Team as a SFCG local partner, The Mvomero Organizations Coalition’s (MOC) Accountant Hassan Kuga is now involving his wife in decision-‐making and since September 2013, together with MOC’s Executive Director and his wife, they have begun to farm 5 acres of paddy rice which is projected to give a profit of Tzs5 million over a season.

Through his involvement in the Team process Stanford and Hassan realised the importance of involving women in decision-‐making. Outcome

28 TreatmentFollowing The Team Mwajuma Mpambayage (a female teacher) no longer mixed her life at home with her life as a teacher.

The issue of carrying your work home featured in The Team TV series and in the discussions that followed Anecdote

29 TreatmentFollowing The Team Kasi Dawi (a male teacher) is making enough time to spend with his wife and children.

During the screening of The Team Kasi saw men and women teachers planning to sell off school grounds but the woman teacher (Mwalimu Wito) consistently fought against this and the playground was saved. It demonstrates that women can have good ideas and that they can transform a man. The FGD helped reinforce the message of the TV programme of the importance of respecting women's ideas. Outcome

30 Treatment

From September 2013 Generosa Mfuruki, the female Community Development Officer for the villages of Makuyu, Mgudeni and Mvomero, has reinforced her commitment to help to educate women in these villages.

Generosa's work has been enhanced by the courage of Mwalimu Malaika Wito who has been used as a role model by the community members who viewed the TV episode. Generosa watched The Team in the mobile cinema screening which was the main contributor towards this change. Anecdote

31 TreatmentFollowing The Team Selemani Rukonge (a male teacher) has now recognised that he has responsibilities to his family and to ensure that conflicts are avoided.

Selemani learned about the misuse of power through the incident in The Team programme when a business man tried to bribe pupils not to follow up when he was trying to expropriate school land. He learned this through the show and the discussions that followed. Proto-‐outcome

32 Treatment

Following The Team Kayuwi James (a male teacher) has changed his attitude to women, for example by accepting that new ideas should be judged by their qualities and not by who put them forward.

The key contribution of The Team was the mobile cinema and the key aspect of The Team programme was the determination of the teacher who started The Team despite being discouraged. Proto-‐outcome

33 Treatment

Following The Team Kayuwi James (a male teacher) has changed his expectations and he now looks to make things happen no matter how many challenges he faces. He plans to undertake further studies within three years.

The key thing was the cinema and the key aspect of The Team was the determination of the teacher who started The Team despite being discouraged. Proto-‐outcome

34 TreatmentFollowing The Team Godfrey Urassa (a male teacher) has increased his awareness that gender inequality is learned from the family.

A scene from The Team where the mother was discouraging and the father encouraging the girl to play football led him to this belief. The root cause of the belief of the mother was that girls could not make it and she had confidence in boys only. Proto-‐outcome

35 TreatmentFollowing The Team women in the Village Community Banks (VICOBA) credit savings scheme have increased their self-‐confidence.

The discussion helped to create a solid common understanding of the issue of undervaluing women. Both the TV show and the discussion were very important. Proto-‐outcome


36 TreatmentFollowing The Team Selemani Manyangula realised that cooking, washing, etc. are also jobs for boys and men.

Watching the scene in The Team with the boy who didn’t like to cook or wash influenced Selemani’s change in attitude. Proto-‐outcome

37 ControlAfter watching The Team on TV Zaitun Ayubu no longer thinks that it is right for parents to deny a girl child the right to go to secondary school.

The Team showed how parents refused to let children enrol for secondary education after standard 7. Under such circumstances the girls tend to run away from homes or get unwanted pregnancies – or the girls may end up committing suicide. Proto-‐outcome

38 Control

After watching The Team on TV Hadija Ramadhani has learned that it is useful to everybody to send a girl to secondary school and has decided to do this for her future children by saving towards affordable education.

Seeing The Team and the fact that girls could play in a football team showed Hadija that things can be done by anybody. Proto-‐outcome

39 ControlHalima Salum is passionate against gender violence and The Team has solidified this attitude.

Halima remembers the incident when the old man beat and wounded the young girl when he was trying to force himself upon her. Similar things have happened to her and her sister. Observation

40 ControlAfter watching The Team on TV Theodori Mkimbila believes that it is acceptable for women to play football.

Seeing The Team on TV and observing conflict among teachers particularly regarding whether or not girls should play football was influential. Observation

Annex D: The Team Tanzania – Case studies

Contents 1. Mvomero: School girl truancy falls after girls’ football team is created ........................................ 1

2. Mvomero: School building construction becomes significantly more effective by using a participatory approach ...................................................................................................................... 3

3. Mvomero: Widows’ group’s new planning process boosts income and savings ........................... 5

4. Mvomero: One woman’s business grows as she gains confidence and ambition ......................... 7

5. Kilwa: Self-‐awareness as the result of The Team Motivates Hamidi to rejoin schooling .............. 9

6. Kilwa: Mobilising fellow school children to fight together for gender equality. .......................... 10

7. Kilwa. A woman’s reduces her conspicuous consumption to avoid debt ........................... 12

8. Kilwa: A widow’s enhanced self-‐reliance and financial freedom ................................................. 14

9. Tarime: A wife enjoys greater well-‐being and income after her husband grants her shared ownership of assets and income ...................................................................................................... 16

1. Mvomero: School girl truancy falls after girls’ football team is created

Principal Source: Charles Kikullu -‐ Ward Education Coordinator (WEC) (0718232312)

The outcome: Since April 2013, girls have been training for the newly formed girls’ football team at Hembeti Secondary School in Hembeti ward of Mvomero division and school attendance has increased by more than 120 from the 350 that regularly attended before the formation of the girls’ football team.

Hembeti school is a secondary school for forms 1-‐4 with 800 enrolled students. Attendance is poor for a variety of reasons.

The girls started training in March 2013. Some girls volunteered on the spot but others had to be counselled on the benefits and the potential of playing football by the sports’ teachers who linked playing football now with future employment opportunities. There was a lengthy process of negotiation with many of the parents with teachers having to go to the children’s homes at times to persuade their parents that training and playing

Seven of the Hembeti Girls School Football Team with teachers and Stanford Kalala of MOC

football was a good thing. The whole process of formation of the girls’ team has been very participatory with other teachers apart from the sports teachers also involved.

The formation of the girls’ football team has encouraged students to like schooling. This has been reflected in increased levels of motivation. The girls believe that if they can play football then it is also possible for them to take part in other activities traditionally reserved for boys like brick making.

The boys have learned that playing football is not their area of dominance. Girls can do it as well. There has been a reduction in the attitude that girls are weak and now boys see girls as equally strong.

Watching the matches is entertaining for the students and this encourages them to attend school. If students attend school regularly they are allowed to watch the girls’ matches so this has been an incentive to attend school.

The contribution of The Team: Ward Education Coordinator (WEC) Charles Kikullu is responsible for encouraging more access and equity of participation in education of girls and the marginalised. After viewing The Team in March 2013 he decided to follow the example of the character Wito and establish a girls’ football team. He involved the teachers, in particular those responsible for sport (one male and one female teacher) to spread the idea. Before watching The Team Charles Kikullu had the perception that girls could only play netball. The Team taught him that girls can also play football. The concept of forming a girls’ football team arose entirely from The Team.

Substantiation:

Substantiator Outcome Contribution of The Team

Pili Issa (Girls’ sports teacher at Hembeti school)

Fully substantiated Fully substantiated

Oscar Anthony (Teacher at Hembeti school)


Wilhelm Mushi (Teacher at Hembeti school)


Additional source of substantiation: photos of the girls’ football team.

Evaluators’ statement on the value of this outcome: The outcome has a quantitative significance – a more than 30% increase in school attendance increasing the life chances of both male and female students; and a qualitative significance – increased confidence and motivation of girl students, increased respect of boys for girls, improved student motivation levels, greater discipline and enhanced integration among male and female students.

2. Mvomero: School building construction becomes significantly more effective by using a participatory approach

Principal Source: Charles Kikullu -‐ Ward Education Coordinator (WEC) (0718232312)

The outcome: Since April 2013, the Ward Education Coordinator (WEC), Charles Kikullu, has initiated the practice of planning together by involving all the head teachers and parents in a school building construction project. Productivity has increased approximately eight-‐fold. The previous toilet block took four years to build and this one will be done in less than six months (see photos below).

Charles Kikullu is responsible for planning community contributions for construction of classrooms and other school buildings for nine primary and one secondary school in Hembeti ward of Mvomero division. Because of the acute shortage of teachers the education authorities have also planned to construct one house per year for teachers.

Until he watched The Team in March 2013, Charles Kikullu had used the prevailing top down approach in implementing this responsibility. Following decisions by school management and committee members that would determine what each school needed, he would then knock on people’s doors and demand their contribution.

Under the new planning approach, Charles explained the opportunities and available resources – bricks, collection of stones, sand and concrete. As a result of planning together people are contributing more willingly, are working faster and are more eager to help. They got contributions of TShs 10,000 per parent for the civil work and labour. The construction work started in June 2013. In the first phase, 40,000 burnt bricks have been produced for four classes and an office and 4,000 burnt bricks have been produced for the completion of toilets for the teachers1.

1 The bricks for the classes and the office are all contributed by the parents. The bricks for the toilets are mostly contributed by the parents but some were provided by the students through the after-‐school self-‐reliance scheme

Hembeti School toilet blocks. Old block on the left and that being currently constructed on the right

Hembeti Secondary School started brick making for the construction of teachers houses with the participation of the pupils since December 2012. This school project is now using a similar participatory process as that described above and the same type of participatory planning process is now also being used during the after-‐school self-‐reliance scheme period. Participatory planning is proving to be a useful life skill for students, children and parents alike.

The contribution of The Team: From The Team Charles learned that there is the need for community participation in decision making for effective and efficient implementation of development plans. In The Team he saw the business man trying to purchase the school land without the participation of the teachers and the community. He learned from this that there should be participation for good leadership.

Evaluators’ statement on the significance of this outcome: The outcome has a quantitative significance – a more than 800% increase in productivity of a volunteer construction programme; and a qualitative significance – increasing use of participatory decision-‐making processes by administrators, teachers, parents and students in planning and executing activities.

Substantiation:


Shida Samuel Rukali (Headteacher at Hembeti school)


Pili Issa (Girls’ sports teacher at Hembeti school)


Oscar Anthony (Teacher at Hembeti school)


Wilhelm Mushi (Teacher at Hembeti school)


Other source of substantiation: photos of the buildings under construction.

3. Mvomero: Widows’ group’s new planning process boosts income and savings

Principal Source: Tatu Mbonde – Farmer and leader of a Mvomero widow’s group (Tel: 0718870725)

The outcome: From June 2013, Tatu Mbonde the leader of a Mvomero widow's group, has initiated a participatory planning process for the group to help them to earn income from agriculture and save to help members in trouble. The group is currently constructing a chicken pen which will be used communally. When it is finished it will take 200 chicks. Everybody will contribute TShs 3,000 to help raise the TShs 110,000 needed to construct it and contribute an equal number of chicks for the production of meat and eggs.

The group has also decided to set up a communal savings scheme so that there is money available for the group members when they get in trouble. They have agreed to contribute TShs 500 per person per month and now have TShs 36,500.

Tatu Mbonde has been a leader of a widows’ group since it was formed on July 19 2012 with five members. It has grown gradually and now there are now twenty-‐six members. The aim was to earn their own living and therefore reduce the temptation to get money by other means. Before watching The Team and being involved in the focus group discussions starting in March 2013 the group did not understand the concept of planning together.

The contribution of The Team: From The Team (the mobile cinema and focus group discussions) Tatu learned that teamwork among community members brings about efficiency.

Through The Team, Tatu saw how the girls are cheated to leave schooling and studying and the poor advice they can get from the family.

The father took a loan and when he failed to recover the loan, the wife advised him to marry off their daughter. This reinforced her attitude toward girls’ education and she has since endeavoured to encourage her fellow women to opt for alternative income earning opportunities in order to avoid such things.

Tatu estimated that The Team contributed about 80% to the changes outlined with other factors being responsible for the remainder.

She is aware of at least one of the group members (Odilia Lugendo), who participated in the mobile cinema and focus group discussions, who has also influenced other women to form a group -‐ Kumekucha.

Tatu Mbonde, wearing a red The Team tee-‐shirt and fellow widow's group member next to the chicken pen which is under construction

Evaluators’ statement on the significance of this outcome: The outcome has a quantitative significance – the promise of increased earning and savings of the group; and a qualitative significance – increasing use of participatory planning processes, reduced temptation to seek money by marrying off school-‐aged girls and enhanced educational opportunities for their children.

Substantiation:


Tatu Mnyali (Secretary of the group)


Mwana Hamisi Salum (Treasurer of the group)


Charles Kikuli Fully substantiated Fully substantiated

Other source of substantiation: Photos of the chicken pen under construction.

4. Mvomero: One woman’s business grows as she gains confidence and ambition

Principal Source: Semeni Kilongoro –a shopkeeper of Mvomero (Tel: 0718658835)

The outcome: Since March 2013, Semeni Kilongoro, owner of a grocery store in Mvomero, gained confidence and ambition and began to work harder with more focus, spirit and concentration with the result that her business has grown dramatically. Before watching The Team and being involved in the focus group discussions in January and February 2013 Semeni had about TShs 200,000 in capital and made TShs 10,000 – 20,000 per month in profit from the store. Now she has TShs 1.5m in capital and makes a profit of TShs 100,000 per month. It is still a challenge to raise funds but her hard work and spirit has enabled her to reach the level she is now at and she has even higher targets for the future. Her ambition is to own a wholesale unit worth TShs10 million

Semeni Kilongoro established her grocery store in November 2012 but it was not going well as she lacked confidence in her ability to run a business. She could not afford the basics in life like soap and she used to wait for handouts.

Now, she is standing on her own two feet and she can do what she wants with the money she earns. For example, she has been able to complete her sister’s house following her death.

The contribution of The Team: The Team inspired Semeni a great deal. She used to have doubts but is now convinced that she can do more than just household work. She used to think that trade and business was meant for men but came to realise that women can be effective business people. Through watching The Team she came to realise who she was and her potential roles as a woman. The Team also had another educational aspect for Semeni, in terms of entrepreneurial knowledge and the concept of setting targets.

The section of The Team that particularly influenced her was the part when the woman teacher [Malaika Wito] was trying to coach the girls to play football and showing that playing and coaching was not just a man’s job. Both the viewing and the subsequent discussion helped her to wake up to the challenge of doing things normally thought to be men’s work. Semeni estimated that The Team made an 80% contribution to the changes outlined with her own ideas [not stated] being responsible for the remaining 20%. Semeni knows of another woman from the target group who has made similar changes.

Evaluators’ statement on the significance of this outcome: The outcome has a quantitative significance – increased earning; and a qualitative significance – increased self-‐confidence and Semeni’s function as a role model.

Substantiation

Substantiator Outcome Contribution of The Team Odilia Lugendo (Special Seat Councillor)

Partially substantiated -‐ As any other customer she has seen Semeni's business grow gradually but cannot tell its volume. Given her impressive progress, Odilia as a leader sometimes make reference to Semeni's achievements when mobilising women especially young ones to emulate her to start and manage income generating activities.

Not able to substantiate

Michael (Customer and owner of neighbouring shop)

Partially substantiated -‐ Regarding the start of business and growth of business, I cannot tell for sure but the volume of business has somehow increased in number and variety of items unlike in the past.

Not able to substantiate

Amisa (Neighbour) No response No response

5. Kilwa: Self-‐awareness as the result of The Team Motivates Hamidi to rejoin schooling Hamidi Mohammed Khalufa (Youth, 26 years old 0787208480 / 0658208480 of Kilwa Masoko

Mjini)

The outcome: In late April 2013, Hamidi Mohammed Khalufa (aged 26) re-‐enrolled into the Kilwa

Day Secondary School soon after participating in the mobile cinema screening of The Team and focus

group discussions. He wanted to rejoin to obtain a good education for a better life free from issues

like the urge to beg or the temptation to push drugs, abuse women, steal or roam the streets. He

shared this idea with his parents and his father (whose approval and financial support he needed)

willingly accepted, giving him financial support to the tune of TShs 44,000 to pay for the initial

tuition fees and basic requirements like text books. By the time Hamidi will sit for the National Form

Four Examinations for his six subjects in November 2013 the cost is likely to amount to TShs 550,000,

huge money by Kilwa standards. His expectation is that he will attain Division 3 by scoring a

minimum of C in each of his subjects in the 2014 National Form Examination. He now aspires to be a

teacher.

The contribution of The Team: The Team contributed greatly to inspire Hamidi to resume his

schooling. After he saw The Team he changed his perception of the importance of education. He saw

how girls were being cheated into sexual relations in one of the episode of The Team and he realised

the reason was ignorance of many evils which is largely due to limited education.

Substantiation:


Mohamed Issa (Hamidi ‘s father)

Fully substantiated Confirmed.

Mwalimu Emanuel Ruta (Teacher at Kilwa Day Secondary School)

Fully substantiated Not substantiated. He is unaware of the source of inspiration for Hamidi’s decision.

Evaluators’ statement on the significance of this outcome: Hamidi’s expected level of academic

attainment will help realise his ambition to be a teacher and to avoid the temptations outlined

above. His achievements will help to motivate his fellow youths in Kilwa who are not going to school

by showing the potential of an alternative approach to secondary education schooling (e.g. private

schooling) for his peers to emulate.

6. Kilwa: Mobilising fellow school children to fight together for gender equality. Hamidi Mohammed Khalufa (Youth, 26 years old 0787208480 / 0658208480 of Kilwa Masoko

Mjini)

The outcome: After rejoining Kilwa Day Secondary School in April 2103 Hamidi Mohammed Khalufa

mobilised fellow youths, both boys and girls, to fight for gender equality by engaging them in group

discussions related to academic studies in an evening tuition. This process was resisted at first but

the situation has gradually improved and now there are high levels of interaction between boys and

girls that do not centre on sexual relationships.

Hamidi Mohammed Khalufa’s attitude towards interactions between girls with boys changed as the

result his participation in the mobile cinema screening of The Team and focus group discussions

from feeling that interactions should only relate to sexual relationships to one that involves the

sharing of information about educational matters and life experiences.

The contribution of The Team:

Hamidi became aware of the importance of gender equality between girls and boys and the

importance of education for enlightenment about the evils of sexual harassment and the attainment

of equality as a result of viewing The Team and participating in the focus group discussions. A scene

that contributed to the change was that in which a rich adult man seduced a school girl – Upendo -‐

with donations of money and transport in a Range Rover for sexual favours and the fate of Waridi

who was promised for marriage at a young age. A scene that illustrated gender equality was that of

the girls playing football for the first time. It was very much resisted by male teachers and boys alike

but the female teacher -‐ Malaika Wito -‐ fought hard to enable the girls to play with boys. In fact the

girls played very well to the surprise of the doubters.

Hamidi estimated that The Team contributed to over 80% to the changes outlined as before this he

would not have dared to initiate efforts aimed at gender equality among the Muslim community of

Kilwa without courting trouble. Perhaps, modernisation trends could have also caused the changes

thus accounting for the other 20% or so.

Substantiation:


Mwalimu Emanuel Ruta (Teacher at Kilwa Day Secondary School)

Fully substantiated Not substantiated. He is unaware of the source of inspiration for Hamidi’s actions.

Evaluators’ statement on the significance of this outcome: This outcome can inspire others to

promote gender equity that may help to increase women’s chances of obtaining a better level of

education that can assist their economic empowerment and reduce their vulnerability to evils such

as premature marriage and abuse by men.

7. Kilwa. A woman’s reduces her conspicuous consumption to avoid debt

Amina Msham Mwangu (Housewife originally but now a Volunteer at a Wildlife Organization – 38

years old, 0783207993).

The outcome: Since participating in the mobile cinema screening of The Team and focus group

discussions Amina Msham Mwangu has changed her behaviour from conspicuous consumption to a

greater focus on the basic necessities. She used to purchase new clothes and jewellery for every

celebration which may occur many times in a month. This behaviour is rooted in norms and customs

of women in coastal areas, Kilwa in particular. This behaviour pushes women to marry off even their

school aged daughters, to earn money for the purpose of purchasing consumer commodities. Amina

has changed her attitude from thinking that it is okay to marry off school girls to get money to

supporting them to get an education.

She discussed the realization of the negative impact of her habits with her husband who agreed with

her new attitude and her readiness to accept the realities dictated by their financial situation -‐ that is

affordability to be the main determinant of whether or not new purchases are done when the

occasion presents itself. New items for events used to cost a minimum of TShs 120,000.00 per

month. Her demands have now decreased to about one-‐third of that. For example, new items for

the month of September cost about TShs 40,000.00. Amina’s husband is happy about her changed

practices.

The contribution of The Team: Amina realized the negative effects of her former attitude after

seeing the mother of Sophia who was forcing Waridi, who was at school, to get married to a rich

man to get money to repay her husband’s loan. Through The Team Amina realized that her demands

were costing her family a great deal, in particular her husband who is a District Council employee.

Amina estimated that The Team contributed over 90% to the changes outlined because they took

place after the Team which in addition to enlightening her on the negative impact of her behaviour

also created an environment of acceptability of changes of this type of behaviour among community

members.

Substantiation:


Shaban Mohammed (Amina’s husband)

Fully substantiated2 Fully substantiated

Evaluators’ statement on the significance of this outcome: The outcome has a quantitative

significance – a TShs 80,000 reduction in monthly spend on new items for celebration; and a

qualitative significance – reduction of stress in families and reduced temptation to marry off young

daughters for financial gain, increased potential for enrolling girls in secondary schools. Amina lives

in Magereza compound where competition of material acquisition among women is prevalent. This

can result in husbands contracting loans with their ramifications in terms of stress and financial

burden. There is a common tendency of Kilwa community members including women to marry off

their daughters for economic gain. This outcome, if it motivates others, can help reduce this

tendency and increase the number of girls enrolling into secondary education.

2 It was Shaban who provided the financial figures

8. Kilwa: A widow’s enhanced self-‐reliance and financial freedom

Mwanahamisi Issa (37 Years old widow: 0758461739)

The outcome: In June 2013, Mwanahamisi Issa started a business selling onions, tomatoes, potatoes,

okra and cashew nuts. Today, Mwanahamisi’s capital is about TShs 250,000. Her decision to start a

business came after participating in the mobile cinema screening of The Team and focus group

discussions when she started to think of a project that could sustain her family’s livelihood. She told

her sister that she did not like to depend on her for cash all the time and asked her to give her a

small sum of money that she could use to run a business. Her sister’s husband gave her TShs 50,000

that she invested to start the business.

Mwanahamisi normally uses part of the profit for food, paying bills, and paying her child’s school

fees. She has also been able to buy some blocks adding to the previously bought ones. Mwanahamisi

now has a total of 700 blocks. Her target is to get about 1,500 to construct a self-‐contained house

with 3 bedrooms, a kitchen, store, toilet and bathroom. However, the pace of block-‐buying is now

slow because she has started saving for her child who will complete primary education next year. He

is performing wonderfully well and she believes that he is going to do well in his final exams and he

is going to be selected for a good school outside the district. She wouldn’t like to disappoint him,

which is why she is undertaking these preparations.

The contribution of The Team: The Team gave Mwanahamisi the confidence and courage to get

something done as she had been in a state of despair. It has made her believe that she could

become successful. She has also convinced some of her friends and neighbours to take similar steps.

Without The Team it would have taken her a longer time to build the confidence and courage to

undertake this change. Although she had done business before, it wasn’t with this level of

determination.

Mwanahamisi saw the way teacher Malaika Wito went through difficult circumstances but finally

succeeded. She introduced the idea of girls playing football even though many of her fellow teachers

were against her. But in the end she had a successful girls’ football team. She also played a major

role in ensuring that the school playground was not sold to a rich man and she was also able to

mould the students’ discipline.

Significance of the outcome given by the contributor: This change has given Mwanahamisi financial

independence. Until recently she had to be dependent on her sister for support. She was also

receiving support from an Islamic organization. Before the change she would not have been able to

support her child through secondary school but her business now gives her an opportunity to

achieve this. Similar changes are also being undertaken by her friends and neighbours.

Substantiation:


Twalah Mbwangali (Chairperson of Miembe Miwili sub-‐village)


Additional substantiation: The evaluators visited the fruit and vegetable sales site and observed the

sales proceedings and results of cement blocks produced by used of income generated by the

business as indicated below.

9. Tarime: A wife enjoys greater well-‐being and income after her husband grants her shared ownership of assets and income

Principal Sources: The husband and wife -‐ Mairo Keria Mugesa and Ester Mairo

The outcome: After The Team, Mairo Keria Mugesa, a farmer, realised the potential socio-‐economic

benefits of involving his wife in decision-‐making and ownership. The dairy cow is now owned jointly

by him and his wife, is making money for both of them and they share the income. He estimated

that the dairy cow and the sale of its calf makes them a daily profit of about TShs 7,200. In July 2013,

they bought 200 chickens that they own jointly. Egg production makes them a daily profit of about

TShs 12,000.

Before participating in the mobile cinema screening of The Team and focus group discussions, Mairo

Keria Mugesa used not to consult his wife Ester about business investments because he did not

value her contribution. He built a water well alone and he was the sole owner of all family business

investments including the ones initiated and managed by his wife, such as the dairy cow that she

had obtained under the Magobiro Farm Centre Project in 2010.

a.) The husband (Mairo Keria Mugesa) feeding the dairy cow now which he did not do in the past. b.) The Wife (Ester Mairo) in front of the chicken pen

Their newly found close working relationship

is illustrated by their joint construction of a

a.) b.)

The 2nd water well which has been constructed jointly by the wife and husband since viewing The Team

second water well in three months from July 2013.

Mairo and Ester have been able to construct a new house worth TShs 12,000/-‐million within three

months from July 2013. Ester confirmed that she is happier than before as the result of the changes

following The Team.

The improved Mairo family housing resulting from joint family ownership. Photo a shows the kitchen the family used before The Team. Photo b shows the new house for the kitchen built through the joint efforts of the wife and husband after The Team.

The contribution of The Team: Aspects of gender

discrimination were highlighted in The Team episodes shown

in the mobile cinema. Specific incidences in The Team that

Mairo referred to as influences were the female teacher being

ridiculed by male teachers and even male students, and a

neglect of women’s ideas by male and fellow female teachers,

for example regarding the formation of the girls’ football

team.

These incidences catalysed discussions about discrimination against women’s ownership and

inheritance within the Kulya communities. Issues highlighted in the The Team focus group

discussions included the right to own property which Kulya tradition bestowed on men thus denying

women the entitlement to property even of their own making, as was the case of the Mairo’s

families dairy cow.

The Team has contributed to a certain extent to the change itself but it has greatly contributed to

the pace of the change which has happened in only three months.

a.) b.)

Wife and husband sharing business income which did not happen before The Team

Substantiation:


Maro Mwita Mroni (Teacher of Mogabiri Primary School)

Fully substantiated Partly substantiated -‐ He could at best say the changes have occurred after The Team and he is not aware of factors other than the Team that influenced the change

Boka Msama (Chairperson (of committee appointed to oversee the process of inheritance of properties)

Fully substantiated Partly substantiated -‐ He could at best say the changes have occurred after The Team and he is not aware of factors other than the Team that influenced the change

Further sources of substantiation: The records of the Mairo and Ester seen by evaluator Dunstan

Kishekya that refer to the purchases and sales of eggs and milk on 26th October 2013, and

photographs taken during the site visit.

Evaluators’ statement on the significance of this outcome: Traditionally, only men owned and

appropriated income from family property namely farms, dairy cow and chickens. The sharing of

ownership and income with a wife is a new development promoted by The Team.

Keria Mugesa is an example of a man who has changed his attitude and practices towards women’s

position within the Kulya community as the result of The Team. He is now more respectful of women

and support of their equal involvement in economic activities and of their right to inherit property.

Focus group informants said that some other families have undertaken similar changes but these

changes are still limited to the more progressive participants among those who participated in The

Team mobile cinema showing and FGDs. The other community members are showing a wait and see

attitude though there are signs (verbal commitments) that they may follow suit.

Annex E Monetary value of selected case studies over one year

Amount TZS US$ Notes Case study 1

Monthly profit levels before The Team

15,000 The median of the "TShs 10,000 – 20,000 per month in profit…"

Monthly profit levels after The Team

100,000

Increase in monthly profit levels since The Team

85,000

Total additional profit over one year

1,020,000

Additional profit due to The Team over one year

816,000 534 80% of total additional profit; outcome source estimated 80% contribution from The Team

Case study 2

Monthly reduction in spending on new items for events

80,000 120,000 – 40,000 = 80,000

Assumption: expenditure continues at 1/3 of previous level

Total saving over one year 96,000

Total saving due to The Team over one year

86,400 57 90% of total savings; outcome source estimated 90% contribution from The Team

Case study 3

Increase in capital in four months since The Team

200,000 250,000 (capital in October) -

50,000 (capital in June) = 200,000

Total capital accumulation over one year

600,000 Assumption: profits and outgoings allow capital to increase at same rate over a year

Total capital accumulation due to The Team over one year

360,000 236 60% of total capital accumulation; evaluators’ estimate a 60% contribution from The Team based on sources statement that The

Team helped her gain confidence and courage more quickly than otherwise

Case study 4

Annual profit from the egg business

4,380,000 Daily profit: 12,000

Annual profit = 12,000 * 365 = 4,380,000

Assumption: only the income from the egg business is considered as the cow was already owned by the husband

Total profit due to The Team over one year

2,190,000 1434 50% of total savings; evaluators’ estimate a contribution from The Team based on the source’s statement that The Team greatly contributed to the pace of change.

Annex F Potential value for money of the focus groups

A. Cost of the intervention = $476,000

The cost was calculated as follows:

Item Cost (US$)

Making the 13 episodes of the TV series

Production – direct costs 285,000

SFCG staff costs 24,000

SFCG indirect costs 20,000

Staff time of SFCG TZ Director, Finance and others – in kind

0

Subtotal 1 329,000

Mobile cinema screenings & focus group discussions

Partners – direct costs 89,000

Partners - training 16,000

SFCG staff costs – assessment and monitoring

6,000

Travel and refreshments for 840 focus group participants, 6 days

33,000

Subtotal 2 144,000

Consultancy

Selection of partners 3,000

Subtotal 3 3,000

TOTAL COST 476,000

B. Mean monetary value X mean contribution of The Team = $396

This value was calculated from:

Mean monetary value of case studies

Case study Monetary value of outcome over one year (US$)

Case study 2 534

Case study 4 57

Case study 5 236

Case study 6 1,434

AVERAGE (mean)

565

Mean contribution of The Team

Case study Estimated contribution of The Team to the outcome (%)

Case study 2 80

Case study 4 90

Case study 5 60

Case study 6 50

AVERAGE (mean)

70

C: Percentage of the outcomes identified by The Team’s focus group participants which suggest a monetary benefit to women = 39%

Calculated from the following:

Location Number of outcomes with a monetary value

Mvomero 10

Kilwa 8

Tarime 3

Total 21

Total number of outcomes identified = 54

D. The total number of those who took part in the focus groups in the 12 districts = 840

Annex G Information sources and references Information sources consulted when seeking to identify outcomes

1. SFCG (November 2012). The Team Tanzania Second Interim Report. Submitted to

KPMG Advisory Limited Accountability in Tanzania Programme. 2. SFCG (2012). Report on radio drama testing prepared by Stella Msami (The Team

Manager) 3. SFCG (January 2013). The Team: A Platform to Promote Gender Equality in

Tanzania - Baseline Study. Report prepared by Abdul-Aziz Juma, Research Consultant.

4. SFCG (February 2013). The Team Tanzania Third Interim Report. Submitted to KPMG Advisory Limited Accountability in Tanzania Programme.

5. SFCG (March 2013). Field report on local partner’s mobile cinema screening of The Team outreach component.

6. SFCG (May 2013). The Team Tanzania Progress Report. Submitted to KPMG Advisory Limited Accountability in Tanzania Programme.

7. SFCG (May 2013). The Team Tanzania Social Media Report. 8. SFCG (June 2013). The Team Tanzania Fourth Interim Report. Submitted to KPMG

Advisory Limited Accountability in Tanzania Programme. 9. Push Mobile Media Limited (2013). Content Analysis Report for The Team Tanzania

(SFCG) 10. SFCG (undated). Summary of responses to Facebook Focus Group Discussion

question.

Other references consulted

1. SFCG (July 2011). The Team: A Platform to Promote Gender Equality in Tanzania. A Proposal from Search for Common Ground in Tanzania to the UK Department of International Development.

2. Synovate Media (2012). The evolution of the Tanzania media landscape, 3. SFCG (May 2013). The Team summary. 4. SFCG (undated) Three Rs – Reach, Resonance, Response Framework for Media. 5. SFCG (undated). Final TV Summary Guide. 6. SFCG (undated). List of The Team mediums and focus groups. 7. SFCG (undated). Logframe for The Team – Tanzania.

Annex HAnalysis of Facebook responses selected and translated by SFCG

No Face-‐book Name Involvement period Sex English translation

OutcomeAnecdote (i.e. potentially an outcome but lacking specific or detailed data)Proto-‐outcome (i.e. an attitude change only)Observation (reinforcement of a proto-‐outcome)Statement (a general statement on the issue)

1 Bella Rumisha From 2nd March –up to date

F Mainly, I am now independent, I learnt to never give up and to be free without crossing boundaries Anecdote

2 Reganise Von Lettow From 8th March – up to date

M

Primarily, I learnt to respect and support women/girls in my everyday life. I realized that they also have something to contribute to societal development, starting within families. For example, Wito shows great patience through all the challenges she faces, such as being marginalized by her fellow teachers. Sophia demonstrated that Women/girls can also do things that we only thought men could do, e.g being able to play soccer. I learnt that, if women are given the opportunity to remove the patriarchal system that exist within our societies which is a barrier for many women, they can become the centre of development.-‐Secondly, what I learnt from The Team is that good academic work and sports can take your focus from negative activities, e.g crime.-‐Thirdly, not to take shortcuts in life as means to succeed, e.g when Sophia’s family were forcing their daughters to get married for money while their daughter is still very young. Upendo desiring to have material things by having a relationship with a man who is so much older than she is and trying to mix that way of life with education. Those two things do not mix.

Proto-‐outcome

3 Rabbi Francis Hume From March 8th –up to date

M

I learnt a lot from The Team. -‐The same rights should be given to male and female children, e.g same rights in sports-‐Those of us who are still in school should focus on our academic goals and not get involve in love affairs with much older people like Upendo had done.-‐ To have a strong will like Wito.-‐Parents should give their children the same rights, e.g Sophia’s parents believed in Waridi more and she is the one who ended up falling pregnant.-‐Do not judge others before judging yourself first e.g Sophia’s mother believed that Upendo had bad behaviors and her children were saints, at the end one of her children ended up disappointing her.

Proto-‐outcome

4 Issa Maulid From March 10th up to date

MI used to think like Baraka but now I have changed. I now believe that women can bring development in society, e.g what Wito did to defend the school soccer field although she faced many challenges, that was amazing. Her determination taught girls who did not know how to play soccer to become professional soccer players.

Proto-‐outcome

5 Francis Simon From April 6th up to date M

Because of Watching The Team, I have become independent and I now make decisions that I feel are right for my life. I currently study at MSJ (Morogoro School of Journalism), studying TV and radio production. The quality of your work encouraged me to keep studying harder and has convinced me that I had made the right choice. There are two things that I now believe that were not present before The Team, 1) a goal of becoming the best editor and presenter in the world. 2) Wanting to act in drama plays. The Team has been encouraging me in many ways, I now believe that I am capable of so many things. this is why I save money each week to buy airtime for internet to be able to access you-‐tube.

Proto-‐outcome

6 Mussa Mdee From April 15th up to date

MI discovered that women can also play soccer well. I used to believe that if a woman plays soccer she would develop muscles in her legs like a man. Now I have been educated and I would like to advise people not to believe that women are not able, we should not discriminate and equal rights should be given to all.

Proto-‐outcome

7 Joshua Timo From 24th April up to date

MThis is a tough question! However, I have learnt a lot. Later on in life if I get a daughter, I will make sure that I am close to her and I would ask her what she wants and what she does not want. I also copied a few things from The Team when it comes to romantic relationships. I stopped having many relationships with multiple women at the same time. Congratulations The Team.

Anecdote

8 Elisha Gwendu From 27th April up to date

M

There are many major changes. I discovered that there are so many things that can hinder one’s talent just because people believe that they are of a particular gender or due to negative norms.Parents should not be too quick to point fingers at other people’s children; rather they should try to help them.

Proto-‐outcome

Annex HAnalysis of Facebook responses selected and translated by SFCG

9 Dee Mohamed From 1st May up to date MI learnt a lot and now believe that in life sometimes the little things you ignore can create big problems later on. The Team taught me to be very careful in life.

Proto-‐outcome

10 Jacks Sparrow From February 24th to date

MI have discovered many things in my life because of The Team. For example, women can achieve many things that men can also achieve, e.g in a sport such as soccer. What they need is support.

Proto-‐outcome

11 Ismail Kisubi From April 12th to date MWhat has changed in my life is that, I now know that women can do anything, so I now enjoy collaborating and working with people of all genders.

Proto-‐outcome

12 Ambrose Law From April 16th to date M

I learnt a lot. In short, I learnt about a woman’s role in society, although I remember studying it in school but The Team has brought more depth to it.

What I now believe and did not believe before is that, a girl like Sophy can play soccer the way I saw her playing on TV. I mean she was able to do free styles and everything. She’s amazing, truly, anything is possible.

Proto-‐outcome

13 Ino Felix From March 8th to date MI believed in gender equality but not to the extent that The Team emphasized it. Women can achieve much more that men can. Women can make good decisions if they are given the opportunity.

Proto-‐outcome

14 Johnson Rutaraka From April 25th M

Firstly I would like to thank The Team for replying to my messages; I did not expect that you would reply.I learnt that anything possible, it does not matter if one is a woman or man, equal rights to all, women do well. I really Love The Team and its entire cast. I would love to be part of it but I guess there is no other way. Say hi to Sophie, Wito, Baraka, Pendo and headmaster (names of charcters). I love them a lot. Wish you all the best The Team. Mwaaaah!! (as in kisses)

Proto-‐outcome

15 Vincent Paul

M

Parents should be able to allow their children to choose what will is best for them and will help them in life.

A child should be raised by both her/his parents in order to be raised well. This is what has created problems in Pendo’s life.

It is not a good thing to discriminate against women, they are able to achieve as much if they are given an equal opportunity.

Teachers should have different methods of teaching students, like Wito has done.

Teachers should take their work seriously and not get drunk during office hours. Teachers are role models and therefore should be the examples.

The government should not be corrupt and make the society poor because of a few people’s benefit.

Journalists should stand their ground and not give in to corruption and threats. They should write truthful and honest information. Pendo’s aunt’s husband showed a very good example.

Statement

Date post:	06-Mar-2018
Category:	Documents
Upload:	leminh
View:	216 times
Download:	4 times

An outcomes evaluation of Search For Common Ground’s · PDF fileAn outcomes evaluation...

Documents