Evaluation of textbook distribution
Project Title: Evaluation of nationwide textbook distribution on learning in South Sudan Location: South Sudan Client: DFID Date: September 2013 - September 2016
Montrose was engaged to evaluate the effect that textbook distribution in South Sudan has had on learning outcomes, management and maintenance of textbooks.
With only 16% of teachers having a teaching qualification and with a predominantly non-English speaking teaching workforce despite English being the main medium of instruction, there was a critical need for textbooks to support and enhance the teaching and learning process in South Sudan. Furthermore, prior to the textbook distribution, most schools in South Sudan had never received any textbooks and for the few that had, these were often outdated and not reflective of the revised curriculum following the formation of the new nation. This situation was worsened as the average life of a textbook in South Sudan’s harsh climate was estimated at a maximum of three years meaning that for any schools that had books, these deteriorated quickly. It is for this reason that DFID stepped in to meet this need and offer a funding line for the procurement of textbooks nationwide based on the revised curriculum and produced to withstand the harsh climate.
Montrose was contracted to conduct a longitudinal impact evaluation of the textbook distribution process to assess impact on teaching and learning outcomes among teachers and primary school children across six states.
This was conducted through mixed-methods annual baseline, midline and endline evaluation studies that aimed to measure a number of aspects of the programme including:
- The impact of the textbook distribution on access, equity and retention of learning (i.e. the relevance and effectiveness of the programmes). Multiple indicators were assessed including changes in enrolment, attendance and drop out, changes in learning outcomes, and any gender differences or variations between populations.
- The efficiency and equity of the textbook distribution process itself, and how efficiently their use has been applied in classrooms, assessed through multiple factors including the capacity of teachers to use them and their level of training, and the ease of use by students.
- The progress achieved towards the development of a national textbook strategy, including coherence, coordination and sustainability of the DFID textbooks distribution efforts.
Early Grade Reading Assessments (EGRA) and Early Grade Mathematics Assessments (EGMA) as well as lesson observation and school management surveys using tablet data collection were undertaken in 150 schools at 12 months, 24 months and 36 months after the textbook distribution.
The evaluation was carried out as three annual reviews with each review involving mixed methods in country field data collection in schools and subsequent publication of evaluation reports for each phase.
- The first evaluation phase aimed at assessing the supply chain management practices within the assessed schools along with baseline surveys of teaching and learning outcomes in order to measure impact of textbook distribution at mid and endline.
- The second phase measured changes from Year 1 in the status of the textbooks, the teaching and learning outcomes and findings related to the Value for Money (VfM) proposition.
- This final phase measured changes from the previous two years in the status of the textbooks in the sample schools, the teaching and learning outcomes in the sample schools, and findings related to the VfM proposition. The final (Year 3) review also provided a comparison of findings over the three years and presents final evaluation conclusions.
To view more projects click on the links below.