parbhani - CLR India

Centre for Learning Resources

PARBHANI Baseline Study - English Improvement Program

Contents 1. Executive Summary ......................................................................................................................................... 3

2. Background ...................................................................................................................................................... 4

2.1 We Learn English Program, Parbhani District ............................................................................................ 4

2.2 Overview of the Baseline Study ................................................................................................................. 5

3. Methods .......................................................................................................................................................... 5

3.1 Research Question ..................................................................................................................................... 5

3.2 Study Procedures ....................................................................................................................................... 5

3.2.1 Sampling Strategy ............................................................................................................................... 5

3.2.2 Sample ................................................................................................................................................ 5

3.3 Instruments and Procedure ....................................................................................................................... 7

3.4 Data Collection Procedure ......................................................................................................................... 8

4.Data Analysis (WLE – Listening and speaking skills) ......................................................................................... 8

4.1 WLE Level 1 Test ........................................................................................................................................ 8

4.2 WLE Level 2 Test ........................................................................................................................................ 9

4.3 WLE Level 3 Test ...................................................................................................................................... 10

5. Findings .......................................................................................................................................................... 12

5.1 Grade 4 results on WLE Test Level 1 ....................................................................................................... 12



6. Conclusion ..................................................................................................................................................... 15

Table of Figures

Table 1: Overview of WLE sample size ................................................................................................................ 6

Table 2: Overview of WLE Level 1 testing tool .................................................................................................... 7



Table 5: Overview of data collection procedure ................................................................................................. 8

Graph 1: WLE Level 1 Test - Grade 4 ................................................................................................................. 12



1. Executive Summary

This document describes the results of a baseline study conducted in July 2019. The presented study was

undertaken at the beginning of implementation of the We Learn English Program in Zilla Parishad

Schools in Parbhani District, Maharashtra. This programme is a joint initiative of Zilla Parishad Parbhani

District as implementation agency and Centre for Learning Resources (CLR) as technical and capacity

building resource partner. The WLE program aims to improve English listening and speaking skills in

primary school students. WLE is an audio program consisting of serialized stories revolving around day-

to-day conversations between an audio teacher and children. The lessons enable vocabulary building and

speaking with correct syntax. The WLE lessons has been integrated into the regular time table of

students of the Zilla Parishad Schools in Parbhani District.

This baseline study was guided by the questions- What is the current level of English language

comprehension and speaking skills in Grade 4, Grade 5 and Grade 6 children of Zilla Parishad Schools?

To answer this question, the WLE Testing tool consisting of Level 1, Level 2 and Level 3 questionnaire

was used to measure the listening and speaking skills in students through one on one interviews.

The study was conducted in a total of 42 schools from the 9 talukas Gangakhed, Jintur, Manvat, Palam,

Parbhani, Pathri, Purna, Selu and Sonpeth from Parbhani District. 4 schools from Gangekhed, 7 schools

from Jintur, 3 schools from Manvat, 5 schools from Palam, 7 schools from Parbhani, 3 schools from

Pathri, 6 schools from Purna, 3 schools from Selu and 4 schools from Sonpeth taluka participated in the

study. WLE tests was administered on 4-5 students each belonging to Grade 4, Grade 5 and Grade 6

from these schools.

2. Background

2.1 We Learn English Program, Parbhani District WLE program in Parbhani District, Maharashtra was initiated in 2019 by Zilla Parishad Parbhani in

partnership with CLR. The Program was officially launched on 15th August by Palak Mantri Shri

.Gulabrao Patil along with Honorable C.E.O B.P. Prithviraj and former District Collector P.

Shivshankar. WLE aims to improve English listening and speaking skills of students in the Zilla Parishad

School in Parbhani, Maharashtra. The project aims to benefit students from Grades 4-6 in Zilla Parishad

Schools. The program has been rolled out in a total of 1128 Zilla Parishad Schools in the 9 Talukas of

Parbhani District. Of these 600 schools are upper primary schools and the rest are primary schools.

To enable improved English skills in students, English content and support in the form of WLE (a

listening and speaking program) has been integrated into the regular time table of students. WLE is an

audio program consisting of interesting serialized stories revolving around the day-to-day conversations

between an audio teacher and children. The lessons enable vocabulary building and speaking with the

correct syntax. These lessons are transmitted in the classrooms at fixed timings using the services of

Aakashwani Parbhani and FM 98.8 Parbhani. In addition, the lessons are also live telecast on the FM

Parbhani mobile application at the same timings as the radio broadcast. The mobile application is a

measure to overcome the issues of poor mobile network and power outage in classrooms.

The program implementation plan was drawn taking into account student English proficiency, their

current Grade level and programmatic technicalities. This year, all three Grades 4-6 are to receive WLE

Level 1 lessons. WLE lessons have been scheduled Monday to Friday across all Grades.

In July 2019, a 50 member team of the Cluster Resource Group of Parbhani District received one round

of training on the technical aspects of the WLE program. Upon completion of their WLE program

orientation, the Cluster Resource Group members conducted similar orientation sessions for the Zilla

Parishad teachers in their Talukas. This enabled the teachers to have sufficient training in anchoring the

WLE program in their classrooms. After the completion of the intended orientation workshops the

program implementation began in schools from August 2019 onwards.

Prior to initiation of the programmatic components a baseline study of the program was conducted in

July 2019 to assess the current level of English proficiency of students.

2.2 Overview of the Baseline Study The baseline study of the program was conducted in July 2019 to assess the current level of English

proficiency of students. The study was conducted in a total of 42 schools from the 9 talukas Gangakhed,

Jintur, Manvat, Palam, Parbhani, Pathri, Purna, Selu and Sonpeth from the Parbhani District. 4 schools

from Gangekhed, 7 schools from Jintur, 3 schools from Manvat, 5 schools from Palam, 7 schools from

Parbhani, 3 schools from Pathri, 6 schools from Purna, 3 schools from Selu and 4 schools from Sonpeth

talukas participated in the study. WLE tests were administered on 4-5 students each belonging to Grade

4, Grade 5 and Grade 6 from these schools.

40 trained investigators from the Cluster Resource Group collected data from these 42 schools. WLE

tests were administered in a one on one fashion. The WLE Level 1 test was taken by 199 students from

Grade 4, WLE Level 2 test was administered to 205 Grade 5 students, and 200 students from Grade 6

took the WLE Level 3 test.

3. Methods

3.1 Research Question The study was aimed at examining the research question, “What is the current level of English language

comprehension and speaking skills in Grade 4, Grade 5 and Grade 6 children of Zilla Parishad Schools?”

3.2 Study Procedures

3.2.1 Sampling Strategy The study was conducted amongst students from schools where the WLE program was to be launched.

A sample of 42 schools were included in the baseline study from a total of 600 Zilla Parishad schools.

The WLE tests were administered on Grade 4, Grade 5 and Grade 6 students. Grade 4 was administered

the WLE test Level 1, Grade 5 was administered test Level 2 and finally Grade 6 students took Level 3

test. About 4-5 students each belonging to Grade 4, Grade 5 and Grade 6 were chosen through alternate

roll number call for participating in the baseline study.

3.2.2 Sample A total of 42 schools were included in the sample. About 4-5 students each belonging to Grade 4, Grade

5 and Grade 6 were chosen through alternate roll number call for participating in the WLE tests. The

details of the sample size for the study are provided in the table given below.

Table 1: Overview of WLE sample size

No of schools

No of school per taluka No of students Grade 4

No of students Grade 5

No of students Grade 6

Level 1 40 Gangakhed-4

Jintur-7

Manvat-3

Palam-3

Parbhani-7

Pathri-3

Purna-6

Selu-3

Sonpeth-4

199 - -


Jintur-6

Manvat-3

Palam-5

Parbhani-7

Pathri-3

Purna-6

Selu-3

Sonpeth-3

- 205 -


Jintur-7

Manvat-3

Palam-5

Parbhani-6

Pathri-3

Purna-6

Selu-3

Sonpeth-3

- - 200

3.3 Instruments and Procedure The research question was aimed at understanding the current level of English Language comprehension

and speaking ability of the students. This was accomplished using the WLE Level 1, Level 2 and Level 3

testing tool, which was administered one on one for each selected child. The table below provides an

overview of the questions and the learning outcomes covered in the 3 levels of WLE level tool.

Table 2: Overview of WLE Level 1 testing tool

Question No Questions

Learning outcome tested

1 Answering 10 simple questions in English

Listening comprehension and Speaking skills about oneself

2 Picture description Speaking skills- Speaking using picture cue, present tense and present continuous tense

3

Speaking about a friend Speaking skills – description without cues


Question No Questions Learning outcome tested

1 Answering 5 questions in English Listening comprehension and Speaking skills about oneself

2 Speaking using picture cue Speaking skills –Use of prepositions in picture description

3

Translating English sentences to Marathi

Listening Comprehension-Past Tense and Future tense

4 Speaking about yourself Speaking skills- description without cuees


Question No Questions Learning outcome tested

1 Answering 15 questions in English Listening comprehension and Speaking skills about oneself

2 Translating English sentences to Marathi

Listening Comprehension- Compound & complex sentences in past, future and Present

3

Translating Marathi sentences to English

Listening comprehension & Speaking skills-Compound & complex sentences in past , future and Present

4 Picture Description Speaking using picture cue- Action words, Present continuous tense, Preposition

5 Speaking about a teacher Speaking skills – description without cues

3.4 Data Collection Procedure A team of 40 investigators, comprising of members of Cluster Resource Group were involved in data

collection. Prior to data collection, CLR‟s master trainers conducted a one-day orientation workshop for

the investigators. During the orientation workshop the investigators were introduced to the tools,

interview techniques, data collection and entry process. The table below represents the data collection

procedures and tools used to examine the research questions. A discussion of the procedures and tools

in detail follows.

Table 5: Overview of data collection procedure

4.Data Analysis (WLE – Listening and speaking skills) While administering the WLE tests the investigators recorded the exact verbatim of student responses in

the response sheet. These duly filled in response sheets were then scored by the team at Centre for

Learning Resources based on a predetermined coding scheme. The scores were then entered in Excel

and averaged. Below we provide a glimpse into the tools and the data analysis procedure for the WLE

Level 1, Level 2 and Level 3 Tests.

4.1 WLE Level 1 Test The WLE Level 1 test had three questions, as seen in table no. 2. The first question, tested children on

listening comprehension and speaking skills. It had 10 sub-questions. Children received scores out of 10

for the number of questions they could comprehend. These scores were then averaged for all Grade 4

students. To score their speaking skills, their response to each question was categorized as „appropriate

answer in full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate

answer in one word‟. Subsequently, for each child speaking scores included „number of appropriate

answers in full grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and

„number of appropriate answers in one word‟. Each of these scores were then averaged for all Grade 4

students.

Participants in the study WLE (Listening Comprehension and

Speaking ability)

Method

Grade 4 WLE Level 1 One on one interview



In the second question children were expected to speak as many sentences as possible in English when

describing a picture. This included use of vocabulary such as colors, pronouns and action words in

present continuous tense. The children‟s responses were categorized as „appropriate answer in full

grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate answer in one

word‟. Subsequently, for each child speaking scores included „number of appropriate answers in full

grammatically correct sentence‟, „number of appropriate answers in broken sentence‟ and „number of

appropriate answers in one word‟. Each of these scores were then averaged for all Grade 4 students.

In the third question children were expected to speak as many sentences as possible telling the

investigator about their friend – his/ her name, where s/he lives, what does s/he do and like doing etc.

Again, the children‟s responses were categorized as „appropriate answer in full grammatically correct

sentence‟ and „appropriate answer in broken sentence‟. Subsequently, for each child speaking scores

included „number of appropriate answers in full grammatically correct sentence‟ and „number of

appropriate answers in broken sentence‟. Each of these scores were then averaged for all Grade 4

students.

4.2 WLE Level 2 Test The WLE Level 2 test had three questions, as seen in table no. 3. The first question, tested children on








students.

In the second question, children had to answer 5 questions based on a picture shown to them. The

questions were about the actions of the characters. They were expected to respond to the question using

action words and prepositions. The children‟s responses were categorized as „appropriate answer with

accurate sentence structure‟ and „appropriate answer with inaccurate sentence structure‟. Subsequently,

for each child speaking scores included number of „correct answer with accurate sentence structure‟ and

number of „appropriate answer with inaccurate sentence structure. Each of these scores were then

averaged for all Grade 5 students.

For the third question, children had to comprehend 5 sentences spoken in English by the investigator

and translate them into Marathi using past and future tense. The children‟s responses were categorized as

„appropriate answer with accurate sentence structure‟ and „appropriate answer with inaccurate sentence

structure‟. Subsequently, for each child speaking scores included number of „appropriate answer with

accurate sentence structure‟ and number of „appropriate answer with inaccurate sentence structure. Each

of these scores were then averaged for all Grade 5 students.

In the final question children were expected to speak as many sentences as possible telling the

investigator about their daily routines using past tense. The children‟s responses were categorized as

„appropriate answer in full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and

„appropriate answer in one word‟. Subsequently, for each child speaking scores included „number of

appropriate answers in full grammatically correct sentence‟, „number of appropriate answers in broken

sentence‟ and „number of appropriate answers in one word‟. Each of these scores were then averaged for

all Grade 5 students.

4.3 WLE Level 3 Test The WLE Level 3 test had five questions, as seen in table no. 4. The first question, tested children on








students.

For the second question, children had to comprehend 5 sentences in spoken in English by the

investigator and translate them into Marathi using compound and complex sentences in past, future and

present tense. The children‟s responses were categorized as „appropriate answer with accurate sentence

structure‟ and „appropriate answer with inaccurate sentence structure‟. Subsequently, for each child

speaking scores included number of „appropriate answer with accurate sentence structure‟ and number

of „appropriate answer with inaccurate sentence structure. Each of these scores were then averaged for

all Grade 6 students.

In the third question, children had to comprehend 5 sentences in spoken in Marathi by the investigator

and translate them into English using compound and complex sentences in past, future and present

tense. The children‟s responses were categorized as „appropriate answer with accurate sentence structure‟

and „appropriate answer with inaccurate sentence structure‟. Subsequently, for each child speaking scores

included number of „appropriate answer with accurate sentence structure‟ and number of „appropriate

answer with inaccurate sentence structure. Each of these scores were then averaged for all Grade 6

students

For the fourth question children were expected to speak as many sentences as possible in English when

describing a picture. This included use of vocabulary such as colors, pronouns, prepositions and action

words in present continuous tense. The children‟s responses were categorized as „appropriate answer in

full grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate answer in

one word‟. Subsequently, for each child speaking scores included „number of appropriate answers in full



In the final question children were expected to speak as many sentences as possible telling the

investigator about their teacher. The children‟s responses were categorized as „appropriate answer in full

grammatically correct sentence‟, „appropriate answer in broken sentence‟ and „appropriate answer in one

word‟. Subsequently, for each child speaking scores included „number of appropriate answers in full



5. Findings

5.1 Grade 4 results on WLE Test Level 1

Graph 1: WLE Level 1 Test - Grade 4

In the first question as part of the listening comprehension and speaking test, Grade 4 children were

asked 10 questions about themselves and their family in English. On an average they were able to

comprehend around 6 questions. While answering these questions, they were able to provide almost four

correct answers in one word. Their performance on the rest of the Level 1 questions indicates extremely

poor speaking skills.



In the first question as part of the listening comprehension and speaking test, when Grade 5 children

were asked 5 questions about themselves and their family in English, on an average they were able to

comprehend 2-3 questions. Further, they were able to provide correct answers in one word for about 2

of these questions. Level 2 questions under the first section of listen, comprehend and speak are

comparatively more difficult than Level 1 questions under the same section, which explains the much

lower scores of Grade 5 students. Their attempts at the rest of the Level 2 questions is poor.



As part of the basic listening comprehension and speaking on the Level 3 test, when Grade 6 children

were asked 15 questions about themselves and their family in English, on an average they were able to

comprehend close to eight questions. Interestingly, they also answered 6-7 of these questions in one

word. However, similar to the trend seen in Grade 4 and Grade 5 students, Grade 6 students were also

not very successful in answering the subsequent questions which checked their authentic speaking skills

and comprehension.

6. Conclusion

Based on WLE test results it can be concluded that students possess very rudimentary English listening

comprehension and speaking skills. The ability to comprehend familiar expressions and situations and

the more complex skills, of speaking in whole sentences and accurately were found to be lacking. It is

clear that students have very little exposure to English reading and writing in the normal school course

work.

In order to reach the desired levels of proficiency these students will require a structured English

Learning, and Speaking course and a sustained exposure over a substantial period of time. The WLE

Program, which builds familiarity with the language and helps develop, listening comprehension and

speaking skills is suitable for this target group.

The program implementation plan needs to be designed taking into account student English proficiency,

their current Grade level and programmatic technicalities. Based on the results, it is recommended that

this year, all three Grades 4-6 receive Level 1 lessons.

Date post:	10-Apr-2022
Category:	Documents
Upload:	others
View:	4 times
Download:	0 times

parbhani - CLR India

Documents