Community participation in public schools: impact of information campaigns in three Indian states

This study evaluates the impact of a community‐based information campaign on school performance from a cluster randomized control trial in 610 villages. The campaign consisted of eight or nine public meetings in each of 340 treatment villages across three Indian states to disseminate information to the community about its state‐mandated roles and responsibilities in school management. No intervention took place in control villages. At baseline there are no significant differences in school outcomes. This paper reports on the first follow up survey that took place two to four months after the intervention. We find that providing information through a structured campaign to communities had a positive impact in all three states. However, there are differences across states in where the impact occurs. The most notable impacts occurred on teacher effort, while impacts on learning were more modest. Some improvements also occurred in the delivery of benefits entitled to students (stipend, uniform and mid‐day meal) and in process variables such as community participation in each of the three states. Future research needs to examine whether there is a systematic increase in learning when the impact is measured over a longer time period and whether a campaign sustained over longer duration generates greater impact on school outcomes.


Introduction
Learning outcomes have gained increasing importance in the policy debate on basic education in developing countries. This is especially so for countries that have achieved much progress in access but continue to face low-quality schools. Findings reveal that although more than 90% of children between ages 6 and 14 years are in school in countries such as India, learning achievements are low. The problem of poor-quality outcomes is a more general one and is not limited to the education sector. Various public services across many countries suffer from poor quality (Global Corruption Report 2006 ;World Development Report 2004;Annual Status of Education Report 2005). Targeting resources efficiently to communities and getting public workers to perform their tasks have remained a challenge for several public services in developing countries. This challenge is probably due to weak mechanisms of accountability in public service delivery. For example, Chaudhury et al. (2006) document high absence rates among publicly funded health workers and school teachers in six developing countries including India. Despite these high rates, workers are rarely held accountable for their absences. *Corresponding author. Email: ppandey@worldbank.org Interventions that provide physical resources alone will not be enough to improve outcomes if workers delivering the service do not perform as expected. Monitoring by the local community can play a key role in improving public service delivery (World Development Report 2004). To increase service providers' accountability to the local community, a number of developing countries including India are decentralizing control over local public services to local communities. However, anecdotal reports and survey-based evidence indicate that communities and members of local governments are often uninformed of what services they are entitled to and what state mandated controls they have over these services (CAG Report 2001 ; Bardhan and Mukherjee 2005;Banerjee et al. 2008).
Providing information that empowers stakeholders may therefore be one way to stimulate public demand for quality services. Informed communities can exert pressure on teacher and schools to perform. A newspaper campaign in Uganda, for example, found that publicizing the diversion of primary school funds reduced capture of funds from 80% to 20% (Reinikka and Svensson 2004). But information campaigns that involve a consistent and replicable intervention, random assignment to intervention, and rigorous evaluation of outcomes are lacking.
The evidence from randomized evaluations on this topic is so far mixed. A recent randomized study in the Indian state of Uttar Pradesh (UP) with a five to six month follow-up period found no impact of information on learning outcomes (Banerjee et al. 2008). Information was given on oversight of communities over public schools, and was given by the village head in village meetings. Another study in India, also in UP, with a one year follow-up period, finds a positive impact of information on receipt of health and other social services, based on self reported-outcomes ). The information was on oversight of communities with respect to health services, schools and village governance. The standardized campaign was done through repeat village meetings over two months and was structured to disseminate the information uniformly via tools such as audiotape and posters. The results from these two studies are not strictly comparable due to differences in campaign structure and outcome measurement. For this research agenda to inform policy dialogue, more research is needed based on replicable ways of information dissemination, objective outcome measurement and follow up surveys over longer periods.
This study is an evaluation of a community-based randomized controlled trial to determine the impact of an information campaign on learning and other school outcomes. The campaign disseminated information to the community on its oversight roles in schools and the education services to which they are entitled. Our hypothesis was that school outcomes may improve if communities have detailed information about their oversight and about the services parents are entitled to from schools. We conducted the study in three Indian states, Karnataka, Madhya Pradesh (MP) and UP. Even though all three states have devolved oversight roles to the community, they differ in the extent to which such devolution has actually taken place. UP and MP are two large states in north and central India lagging in outcomes, while Karnataka is in southern India and ahead in economic and development outcomes.
The magnitude of the impact of such an intervention will depend on several factors. First, changing attitudes and practice patterns require ample time, since they have been a way of life for decades. This is perhaps the greatest limitation of the current study, in which we are measuring outcomes after only two to four months. The exact nature of the intervention, the extent of oversight communities have over schools, and the extent of public action in response to the campaign will also be important. The relative population proportions and interests in public services of the different groups in the community will be one determinant of public action. If local elites do not have a stake in public services, public action may be captured (Bardhan 2002).

Background in Indian context
The 73rd amendment to the Indian Constitution in 1992 made it mandatory for Indian states to devolve control over public services and finances for these services to the local village government (called the gram panchayat [GP] 1 ) (Constitution of India 1992). However it was left up to the states as to how much control to devolve. Various states in India have devolved control over services to different extents. We provide a brief description of the school committees for public schools and their roles and responsibilities in the three study states.

Madhya Pradesh
A parent-teacher association (PTA) is mandatory in every school. The PTA has an Executive Committee consisting of 14 members that runs the PTA, although all parents are supposed to participate in decision-making. The PTA chair and school headmaster jointly operate the school account. The school account receives annual grants for repair and maintenance, school development, teaching learning material, school uniforms, civil works and mid day meals. The PTA is supposed to monitor learning, and to manage and monitor funds coming to the school accounts. It verifies a teacher's attendance in order for his/her monthly salary to be released.

Karnataka
Every school has a mandatory School Development and Monitoring Committee (SDMC). Members of the SDMC include the headteacher of the primary school, elected members of the village government and parents. One of the parents is the chair of the SDMC. The SDMC chair and the headteacher jointly operate the school account, and the types of funds received in the school account are similar to those in Madhya Pradesh. The SDMC is supposed to monitor learning and hold a parent meeting every three months to discuss learning. The SDMC does not have explicit control over teachers, but in all three states these committees can inspect schools and register complaints with district/block education offices.

Uttar Pradesh
A village education committee (VEC) is mandatory in every GP. The VEC typically has five members, chaired by the elected head of the village government. Other members include the senior most teacher and three parents. The VEC chair and the head teacher of the school jointly operate the school account. In addition, the GP account that is cosigned by the VEC chair receives stipend funds and mid-day meal funds. The VEC is to monitor learning, and to manage and monitor school funds coming to the school account and GP account. The VEC selects contract teachers and decides annually whether to renew their contracts.

Setting
This study is a cluster randomized controlled trial of 610 GPs in three states -Karnataka, MP and UP -randomly allocated to receive or not receive the treatment, which is an information campaign. A GP is a cluster of approximately one to three adjacent villages and is the smallest unit of local government, which consists of an elected head and council members.
The trial was conducted from February 2006 to August 2007. In each state, four districts were chosen purposefully, matched across states by literacy rates. The districts in order from low to high literacy rates are presented in Table 1.
Within a district, 50 GPs were selected from two randomly chosen blocks. 2 We used a random number generator to randomly select the blocks and then GPs within the blocks. One-half of the GPs within each block were then randomly assigned to the intervention arm and the remaining half to the control arm. Treatment and control GPs were evenly spread across the two blocks to reduce any potential contamination between intervention and control villages. 3 This gives a total of 100 control GPs and 100 treatment GPs per state.
In Karnataka the design was identical except an additional set of treatment villages was added that received a slightly different treatment called information and advocacy campaign. The number of GPs in each of the three cells in Karnataka -control, Treatment 1 (information campaign), Treatment 2 (information plus advocacy) -is 70. 4 Sample sizes were determined using cluster randomized sample size calculations based on a 5% significance level and 80% power.

Baseline survey.
In each GP, one school was randomly selected from all public schools that had Grades One to Five. Teachers who taught these grades in the school are in the sample. In MP and UP, 45 randomly chosen students in a school (15 each from Grades Two, Three and Four) and in Karnataka 30 randomly chosen students in a school (15 each from Grades four and Five) are in the sample. 6 School enrollment registers were used for random selection of students. Sometimes a given grade had less than the required number of students in which case all students were selected. The surveys were conducted by in-person interviews by a team of trained research assistants with prior experience in administering rural household surveys in the region. Surveys were conducted in local languages: Hindi in MP and UP, and Kannada in Karnataka. Instruments were pilot tested prior to use.

Outcomes measured at baseline
• Teacher attendance and activity. Four unannounced visits were made, one every two or three weeks, to record attendance and activity. Activity is a measure of whether a teacher is actively engaged in teaching when the team arrives. It is scored one if the teacher is teaching, writing on the board, supervising written work, teaching by rote or another method; and scored zero if the teacher is absent, chatting, sitting idle/standing outside classroom, keeping order but not teaching, doing non-teaching work. Teacher attendance and activity variables are constructed as averages over the four visits and interpreted as fraction of visits a teacher was present (or engaged in teaching). Both variables take values between zero and one. • Students were tested in school on competency and curriculum-based language and mathematics tests that lasted approximately 20 minutes. The language test included reading and writing competencies while the mathematics test contained addition, subtraction, multiplication and division. The competencies tested fall within or below the competencies listed for the grade by the minimum level of learning framework recognized by the Government of India. 7 As an example, the minimum level of learning specifies a child in Grade Two to be able to read short paragraphs and write dictated sentences, should recognize numbers up to 100, be able to do two-digit additions with carryover and subtractions with borrowing. In MP and UP the tests took place at the end of the school year, while in Karnataka they were four months later at the beginning of the school year. Consequently, the test given to Grade Four (Three) students at the end of the school year in MP and UP was given to Grade Five (Four) students at the beginning of the school year in Karnataka. • Interviews of parents of sample students on their knowledge about school oversight committees, whether the students had received entitlements for current school year; textbooks, school uniform, stipend, whether the mid-day meal was served daily in the past week and whether parents had raised school-related issues. In all three states, children in primary grades of public schools are entitled to benefits as incentives to attend school. These include a daily mid-day meal and textbooks for all students, uniform and a stipend for certain caste/ gender categories. In MP and UP, female students in educationally backward blocks 8 -and in Karnataka, all students -are entitled to a school uniform annually. In UP, students from disadvantaged social classes, Scheduled Caste (SC), Scheduled Tribe (ST), and other backward class (OBC), are eligible for an annual stipend of Rupees 300. Non-disadvantaged students (i.e. those in the other/general caste category) below the poverty line are also eligible for the same stipend. In MP, female SC/ST students are eligible for an annual stipend of Rupees 150.
• Interviews of oversight committee members about their knowledge and participation in oversight.

Intervention
We collaborated with the Nike Foundation in designing the information campaign. The Foundation provided financial support and creative advice in the development of campaign tools. The tools consisted of a short film of six minutes, a poster, a wall painting, a take-home calendar and a learning assessment booklet. The tools were the same in all states except that the information communicated was state specific. The film, poster and calendar focused on the following information: details of roles and responsibilities of school oversight committees; rules for selection of members of these committees; rules for committee meetings; number of mandatory meetings, minimum attendance requirements for meetings; record keeping of minutes; organization and funding of school accounts; right to information regarding the school including right to obtain copies of any school record; where to complain about any problems; and benefits that students in primary grades are entitled to, such as a cash stipend, textbooks, mid-day meals, school uniforms. The film and poster contained key information while the calendar contained all of the information in detail.
The learning assessment booklet outlined the minimum levels of language and mathematics skills that children are expected to acquire by grade, based on the minimum level of learning framework recognized by the Government of India. Parents could use the assessment booklet on their child to find out whether the child knows the minimum expected for his grade. The information in all the tools was obtained from the basic education departments, and tools were verified and approved by the state governments for their content before the campaign. Campaign teams were blind to baseline and follow-up surveys.
In addition to the information campaign treatment in each of the three states, there was a second treatment carried out only in Karnataka. This was an additional twominute capsule at the end of the film that showed average wages for different levels of schooling to increase awareness about the economic benefits of schooling. It also advocated the audience to become involved in monitoring outcomes in the school. All other tools were the same as in the first treatment.
The information campaign was conducted in three rounds, each round separated by a period of two or three weeks. Each round consisted of two or three meetings in different neighborhoods of the GP as well as the distribution of posters and takehome calendars. Residents were informed in advance about the dates and locations of meetings, and separate meetings were held in low-caste and high-caste neighborhoods. The primary target audiences were parents and members of school oversight committees. A meeting lasted about 30-40 minutes and consisted of a film presentation that was played twice, opportunities to ask questions and for discussion to happen among the audience. People were notified that the information was collected from the government. To ensure that the information campaigns were uniform, research assistants read a scripted introduction and were only allowed to answer questions to which the answers were already written on the calendars. Any other questions or issues were not answered. The team was also not allowed to participate in any discussion that took place among the community members following the film presentation.

Follow-up survey
Baseline survey participants were re-interviewed after 12 months by research assistants who had no knowledge of the intervention. To maintain this blinding, intervention group parents, students and committee members were not asked if they attended an informational meeting.

Focus group
One year after the intervention, we conducted focus group discussions in 10 randomly selected intervention GPs each in MP and UP. We conducted two focus group meetings in each GP, one among residents from SC/ST social classes and the other among residents belonging to non-SC/ST social classes. Each meeting had on average 12 participants who were asked if they remembered the campaign, whether they had discussed the information with anyone else in the village afterwards, whether they had raised school-related issues with teachers or VEC/PTA, why bigger changes were not seen and what could be done to improve the campaign.

Analysis
The unit of analysis for teacher attendance and activity outcomes is the individual teacher. The units of analysis for the student test and whether the student received their entitlements (mid-day meal, textbooks, school uniform and stipend) are individual students. For outcomes from interviews of school committee members, the unit of analysis is the individual member.
To measure the effect of the campaign, for each outcome we conduct a linear 'difference in differences' regression analysis comparing the change in intervention with the change in control group from baseline to follow up after adjustment of standard errors for clustering. Focus groups are analyzed by the percentage of respondents to each question. Responses to open-ended questions put to focus groups are presented as the main themes mentioned.

Baseline characteristics
Treatment and control villages are similar in socio-economic characteristics of the sample ( Table 2). The baseline survey also indicates no significant difference between treatment and control samples with respect to outcomes (Tables 3 and 4). It indicates low teacher attendance and poor student learning. School oversight committees are neither aware of their roles nor participating in oversight (Tables 4; Pandey, Goyal, and Sundararaman 2008). 9 However, there is huge variation across states. Karnataka has better student and teacher outcomes as well as higher levels of community awareness than MP and UP. Barely 16-18% of students in MP and UP at the end of Grade Four can read simple sentences and words compared with 47% in Karnataka (Table 3). Average teacher attendance is 65% in UP, 67% in MP and 88% in Karnataka. The average fraction of teachers actively engaged in teaching is 27% in UP, 31% in MP and 68% in Karnataka (Table 4).
With respect to entitlements, there is greater efficiency in delivery in Karnataka where more than 93% of students received meals, books and uniforms (Table 4). In      MP and UP more than 80% of students received textbooks and uniforms, while a student belonging to the SC or ST category in UP received only 76% of the entitled stipend and in MP a female SC or ST student received only 35% of the entitled stipend.
With regard to community oversight, a large proportion of oversight committee members, especially parent members, did not receive any training from the state regarding their roles and responsibilities. Parent members do not participate much in oversight, particularly in MP and UP (Pandey, Goyal, and Sundararaman 2008).

Impact of the information campaigns
The analysis is a 'difference in differences' linear regression where the change in outcome from baseline to follow-up is the dependent variable and is regressed on a constant term and a treatment dummy variable. For individual i in GP j in block k, the regression equation is as follows: where ∆Y ijk is the change in outcome from baseline to follow-up, and X jk is a dummy variable taking value one if village j in block k belongs to the treatment group and zero if it belongs to the control group. ε ijk is a random error term and a is a constant term in the regression equation. The estimate of the coefficient b is an estimate of the average treatment effect.
Because there were two interventions in Karnataka as opposed to one in MP and UP, we present results from MP and UP first and then those from Karnataka. 5.1.1 Change in process: is there an increase in community participation in school oversight?. In UP, the intervention is associated with a 25 percentage increase in the number of VEC meetings and a 25 percentage increase in member participation in school inspections. Both these effects are significant at a p value below 0.03 (Table 5). 10 In MP, the difference in differences analysis is not possible at the individual member level since new rules for PTA election came into effect and PTA committees were re-elected based on these as the campaign was running. Difference in difference analysis is only done using mean PTA response at GP level and shows no impact on PTA members' participation (Table 5). However, there is a significant increase in percentage of parents who talked to the teacher about the quality of education in campaign villages (Table 5).

Teacher effort.
In UP, the intervention is associated with seven percentage points (11%) higher teacher attendance (p = 0.04) ( Table 5). There is no effect on teacher activity. In MP, there is no effect on teachers' attendance but the intervention is associated with nine percentage points (30%) higher teacher activity (p = 0.02). It is interesting to note that even though contract teachers have different appointment terms than regular teachers, in both states the impact of the campaign is similar regardless of the contract type.
The difference in impact can be due to differences in the extent of oversight over teachers. In MP, school committees are obligated to verify teacher presence for the ∆Y a bX ijk jk ijk = + +ε Table 5. Ordinary least squares regression: difference in differences in other outcomes. teacher to receive her salary, while in UP committees control the tenure of contract teachers. Absence of impact on teacher activity in UP may have been due the running of state elections during the follow-up survey and the participation of teachers in election-related tasks.

Entitlements received by students.
Because the entitlements can differ across caste categories and by gender, results are presented separately for these subgroups wherever applicable (Table 5). These analyses are valid since the sample selection was stratified by caste categories and by gender. Oversight committees in both states are responsible for monitoring delivery of entitlements. The impacts occur on stipends and school uniforms.
In UP there is an increase of 33% in stipend received by high-caste students (p = 0.04) while there is no effect for SC/ST and OBC category students. In MP an increase of 26% occurred in stipend received by SC/ST girls (p = 0.02).
In UP there is an increase of 18% in the percentage of girls who received school uniform (p = 0.05). In MP there is a no overall impact but an increase of 14% in the percentage of girls from high castes who received school uniform (p = 0.02).

Learning achievement.
The impacts on learning are few and modest (Table 6). There is positive impact on reading in Grade Three in both states (p < 0.05). In UP the impact is on the percentage of children who can read sentences and words, while in MP there is an impact on percentage of children who can read words. The magnitude of the impact is three percentage points (27%) in UP and four percentage points (14%) in MP. There is no impact on other competencies or in Grades Two and Four.

Focus groups
Thirty-five percent of participants in the 40 focus groups stated discussing teacher attendance issues with either the teacher or the VEC/PTA. Forty-one percent said they had approached the school or VEC/PTA about learning issues. More than 70% of participants who raised school issues talked to teachers and 30% or less talked to VEC. More than 85% stated going with other parents, rather than alone, to talk to teachers or VEC. Seventy-three percent of focus group participants had discussed the disseminated information with others in the village, and 59% said the discussion went on for several weeks. All of the groups mentioned student learning as a main theme discussed. Other topics of discussion were teacher attendance, quality of teaching and entitlements. When asked why bigger changes did not occur in learning or teacher effort, prominent themes were that 'teacher is dominating', 'it is difficult to talk to the teacher' and 'teacher does not care'. When asked how the information campaign can be made more effective, the dominant response was to have more frequent meetings. Seventy-three percent of the groups mentioned this response. Another suggestion was to focus the information campaign towards illiterate parents.

Impact in Karnataka
Since the two treatments differ in one dimension and are otherwise identical, we report the average impact of the two treatments pooled compared with the control group and the additional impact of the second treatment compared with the first.

Change in process.
As 53% of SDMC members were newly elected between baseline and follow-up surveys, difference in difference analysis is done using the mean SDMC response at GP level. Results show no impact on member participation, although this can be due to new member elections that did not leave enough time for participation change. However, there is a significant increase in the percentage of parents who approached the SDMC about school quality (Table 7). 11

Teacher effort.
There is no impact on teacher attendance and activity as could be expected because teacher effort was high at baseline; nearly 90% of the teachers were present, and 80% of those present were teaching. However, we observed at follow-up that teachers with low baseline attendance were more likely to have been transferred from the school in treatment villages compared with control villages, most probably as a result of complaints filed by oversight committees to the education office.

Entitlements.
There is an increase of five percentage points (7%) in the percentage of children reporting satisfactory quality of the mid-day meal (p = 0.03). Although the school oversight committee is responsible for managing delivery of all entitlements, there is no impact on delivery of textbooks and uniforms as these were 97% at baseline.

Learning achievement.
The impacts on learning are few. There is an increase of six percentage points (15%) in the percentage of children in Grade Four who can write words (p < 0.02). In Grade Five, there is an increase of eight percentage points (27%) in the percentage of children who can do multiplication and lower competencies in mathematics (p < 0.03). There is no additional impact of the second treatment (Table 8).

Discussion
We find that providing information through a structured campaign to communities about their oversight roles in schools could be an effective policy tool to improve school outcomes. Positive impacts were observed on process outcomes, entitlements to students and teacher effort, while impacts observed on learning were modest. There are differences across states in where the impact occurs. This is not surprising for at least two reasons. The states had different starting points; baseline outcomes were higher in Karnataka, suggesting greater efficiency in delivery. Additionally the institutions created by decentralization differ across states in structure, roles and responsibilities.
First, on process and behavior outcomes, some improvements were observed in community participation. Gains are larger in UP and more modest in the other two states. In UP an increase is observed in the number of meetings and school visits reported by school committees. There is an increase in the percentage of parents talking to teachers in MP and in school oversight committees in Karnataka. Differences in the impact on participation across states can be due to several factors. Besides the differences in the structure and oversight of institutions created by decentralization, the time between the intervention and the follow-up survey may matter for change in behavior to be observed. In MP and Karnataka, there may not have been enough time for committee members' participation to have changed. In both states, school committees were re-elected and resulted in a majority of new members since baseline. Another factor can be that there are multiple channels of change in behavior, from parents to teachers or from parents to oversight committees. In MP, the channel may be directly from parents to teachers as there is an increase in the percentage of parents talking to them while in Karnataka, the channel may be from parents to schools via oversight committees. This interpretation is supported by focus group discussions.
Second, on entitlements, cash stipends and uniform entitlements improved in MP and UP, and the quality of mid-day meals improved in Karnataka. Oversight committees in all three states are responsible for overseeing delivery of entitlements. These outcomes are also easier to influence over a short time period. It may be easier for parents to demand the entitlements than influence the quality of teaching. The impact in MP and UP varies by student caste, suggesting that caste may be an important factor in influencing individual or collective action. In Karnataka there is no impact on other entitlements because their reach was already high at baseline. Third, with respect to teacher effort, in MP there is positive impact on teachers' engagement in teaching and in UP the impact is on teachers' attendance. No impact is observed in Karnataka where teacher attendance and engagement in teaching were much higher at baseline. The differential impact is probably also due to differences in the extent of oversight over teachers. In MP, school committees are obligated to verify teacher presence for the teacher to receive his/her salary; while in UP, committees control the tenure of contract teachers. In Karnataka, there is no explicit control other than filing a complaint in the education office.
Fourth, on learning outcomes, the gains are few and modest and do not occur uniformly across grades or competencies as test scores are unlikely to be influenced much in a short time. Assessing impact may require more time and a sustained intervention.
What are the mechanisms through which outcomes changed? Focus groups shed light on possible channels of change. Change seems to have occurred through discussions among villagers and villagers bringing up issues with teachers and school oversight committees.
Our findings clearly suggest that, firstly, decentralization to communities is meaningless unless it is made real (i.e. communities at least need to know what oversight roles they have). Secondly, providing information to communities that are unaware, through a structured campaign, can play a useful role in changing behavior and learning outcomes, particularly in lagging states. Previous work on this topic (Banerjee et al. 2008) had not found a positive impact of information dissemination. The nature of our intervention, in particular the structure, intensity and uniformity of the campaign, may explain the differences in results observed here. All information campaigns are not the same just because they are giving information. They differ based on the type and detail of information given, how it is communicated (the medium used) and whether those delivering it are perceived to be credible and reliable sources. 12 The strengths of our study include using a structured intervention that is easily replicable, a rigorous cluster randomized controlled trial design and broad geographic coverage. However, the greatest limitation of the findings reported here is that outcomes are measured soon after the intervention. Changing behavior to change school outcomes requires time. Barriers to collective action are apparent from focus group discussions and may take time to be overcome, especially in light of the recent studies that document institutional inertia (Engerman and Sokoloff 2000;Banerjee and Iyer 2005). Future research is needed to examine whether behavioral changes translate into learning and whether a campaign sustained over longer time generates greater impact and whether there are sustained differences in impacts across states.