Learning Gains Study

The Learning Gains studies investigate the impact of teachers’ participation in professional development on learner attainment. In particular we have focused on the impact of the Transition Maths 1 course which has been completed by more than 150 teachers from approximately 80 secondary schools in Gauteng.

Learning Gains 1 Study (2013)

The Learning Gains I study, conducted in 5 project schools in 2013, showed that Grade 10 learners taught by teachers who had participated in TM1 outperformed learners, in the same schools, taught by teachers who had not participated in the course. The gains were statistically significant with an effect size of d=0.21 which translates to approximately 3 months of additional progress by learners.

The sample size was relatively small, and the comparison groups were drawn from the same schools so the results were treated as “evidence of promise” that the TM1 intervention can make an impact at the level of the learner. Further details can be found here. (ref: Pournara, Hodgen et al)

Learning Gains II Study (2016-2019)

The Learning Gains II study has built on the previous study, using a more rigorous test instrument and a much larger sample of teachers and learners. The new test instrument was designed and piloted in 3 cycles in 2016.

In 2017, 11 teachers who had completed the TM1 course in 2016 participated in the study. Data was collected with 991 of their Grade 9 learners. We also collected data with 988 Grade 9 learners of 15 teachers in the same 9 schools who had not done the course. There was strong evidence to show that the TM1 course had impacted the mathematical knowledge for teaching of the 11 teachers. However, the results of the learner testing were disappointing. There was no statistically significant difference in the gains in the learner test scores (pre-test to post-test) between the comparison group and the TM1 group. When we looked more carefully at the teachers’ profiles, we saw that the TM1 group and their colleagues had, on average, been teaching mathematics for a similar number of years. While the TM1 group had more experience teaching at Grade 8 and 9 levels, their colleagues had considerably more experience teaching in Grades 11 and 12. This suggests that participation in the TM1 course did not make up for years of teaching experience at senior secondary level in the first year after doing the course. A different interpretation of the results might argue that, based on the learner results, participation in TM1 enabled the teachers to do “as good a job” in teaching Grade 9 Mathematics as their colleagues who have more experience in teaching higher grades. We also need to bear in mind the claims in the research literature that the impact of PD on learner attainment is not immediate. To read more (ref: Pournara, Barmby 2019 - Saarmste)

In 2018 we included teachers from the 2016 and 2017 TM1 cohorts. We tested their learners in Grades 9 and 10. We also tested learners in comparison schools where teachers had no previous connection with the WMCS project.

There was strong evidence of the impact of the TM1 course on teachers’ mathematical knowledge for teaching in both the 2016 and 2017 groups. Initial analysis of learner performance shows that the Grade 9 and Grade 10 learners taught by TM1 2016 teachers made statistically significant gains over the other two groups, with effect sizes for Grade 9 and Grade 10 groups of d=0.68 and

d=0.50 respectively. However, the gains of the group taught by TM1 2017 teachers were not statistically significant in relation to the comparison group. Once again this supports the international findings that the impact of PD on learner attainment is delayed. To read more (ref: Pournara, Barmby 2019 - PME)

Further analysis is in progress and results will be made available soon.