In recent years, national and international assessments have received a lot of attention, largely due to concerns over the quality of education systems and their outcomes. Tests such as the Programme for International Student Assessment (PISA) and the Trends in International Mathematics and Science Study (TIMSS) have been publicised in the hope of pushing countries to competitively reform their education systems. However, these tests critically lack information regarding ‘how‘ to reform education systems—especially in diverse contexts such as India, where a large majority of children lack the appropriate cultural capital to succeed within mainstream schooling.
India also produces two periodic large scale assessment surveys—Annual Status of Education Report (ASER) by Pratham and National Achievement Survey (NAS) by NCERT. Both make note of the disheartening levels of learning among India’s children, but also lack information regarding pathways to improve learning outcomes for marginalised children. Socio-demographic information collected as part of these surveys is not meaningfully utilised to understand how and why certain factors become barriers for learning, but rather focuses on quantitative evaluations of the number of students unable to perform at age/grade levels.
A common argument put forward to this critique is that qualitative questions about ‘why’ or ‘how’ students learn/fail cannot be addressed through large scale surveys. We argue that this is easily possible, as evident from our study (commissioned and funded by UNICEF Maharashtra) on 2,783 students from grades 2, 5 and 9, from tribal districts of Maharashtra, India. However, this required thinking through assessment very differently and adopting certain principles:
- First, assuming that higher grade skills build on lower grade skills, we used items from one grade lower and from the same grade to understand where children get left behind in the transition from one grade to another.
- For grade 2, we used oral assessments, though grade 2 students in India are expected to take written tests. We chose oral tests, arguing that with just 2 years of schooling, children would not have sufficiently acquired school language, especially in tribal areas that have very different linguistic and cultural patterns. Tests for grades 5 and 9 were written.
- Consultations with local stakeholders revealed that certain items in our test might be unfamiliar to tribal children, even though they appear in school textbooks. Thus, items were contextualised based on these consultations.
- Finally, while time limits were set for completion, we instructed our field researchers to be liberal in following these, as our focus was not on the outcome but the process, with a view to understanding difficulties children faced in test-taking.
Results showed that while grade 2 predictably performed the best, grade 9 performed better than grade 5. Findings also showed that grade 5 students performed poorly in language (Marathi, the local language of instruction in Maharashtra), but comparatively better in mathematics. This was in contradiction with the overall trend, as well as trends for grades 2 and 9, where students had the most difficulty with mathematics and performed best in language. Thus, we hypothesised that language affected grade 5 scores significantly. Furthermore, we also hypothesised that, even by grade 5, school demands of formal, written language affect performance (which is not seen for grade 2 due to oral tests; and by grade 9 familiarity with formal, written language is stabilised).
Another finding from the study was that social category was the variable most likely to affect these results, with students from tribal communities being the least likely to score pass marks. Interestingly, other marginalised groups, such as the Scheduled Castes (SC, formerly considered ‘untouchables’), were actually likely to catch up with students from non-marginalised communities from grade 5 onwards. In language, they performed similar to non-marginalised groups in grades 5 and 9. In maths, they were likely to catch up with other children by grade 5, and surpass them by grade 9.
The combined results suggest that language becomes a significant pathway affecting learning outcomes overall. For tribal children belonging to different linguistic structures and oral cultures, learning at school is additionally difficult when compared to other marginalised groups. This is also perhaps borne out by the fact that for SC children, with not so much variation between home and school language, performance improves; but performance actually improves in maths which relies less on language.
Thus, in conclusion, we argue that changes made to the nature of assessments help to understand the pathways that affect learning outcomes. Large data sets can also be harnessed for this, if planned appropriately.
This blog was written by R. Maithreyi, who completed her PhD in childhood studies at the National Institute of Advanced Studies, and currently works as Research Adviser with the Centre for Budget and Policy Studies , Bangalore, India. She is also a practicing child psychologist, drawing much of her insights on children’s learning and cognitive processes from her clinical work at Parijma Medical Centre. For the 2017 UKFIET conference, 23 individuals were provided with bursaries to assist them to participate and present at the conference. The researchers were asked to write a short piece about their experience.
This blog has been published on the UKFIET website on 19 January 2018 and can be viewed here.
[Disclaimer: Views presented above are those of the author and do not necessarily reflect those of CBPS]