The gains are real, but some better definitions are in order. Last year's held back kids do not in fact show up in this year's fourth grade results, but they do the following year. If the improvement were just a statistical artifact then there would be a one time jump in performance for the first year (that the 4th grade did not include the bottom 10% of the entering class). Subsequent years might show a decline as the poor students graduate into the next 4th grade. That is not what the data shows, which is a slow but significant and steady improvement over a decade. Over that time period the one-time effect would disappear.
The article Ritzik cited was not an academic study but a back-of-the-envelope thing by a journalist who didnt disclose his statistical methods of adjusting the data. So i cant really condemn the analysis; i can just raise suspicions. I suspect what he did was in effect to put every held-back third grader back into the fourth grade score which would of course eliminate the effect of the reform. But the whole point of the reform was to make sure that fourth graders could "read to learn" and not still be trying to "learn to read". Kevin Drum (the author) could argue that the fourth graders who were held back had an extra year to learn and therefore shouldn't be compared to the nation's fourth graders that were not held back. Hence the definition quibble. But this is an obvious point about kids spending am extra year in school.
Had it been the case that Mississippi implemented the retention policy in 2021, just before the 2022 NAEP administration, then his claim MIGHT be the case (see below about other interventions taking place simultaneously). However, Mississippi started holding third graders back in 2013. Both the 2017 and 2022 NAEP showed improvement in reading. In both of these tests would be students who were held back but then went on to take the 4th grade NAEP exam. This study out of Boston University showed that the retention policy had an effect on reading through grade 6: https://excelined.org/wp-content/uploads/2023/02/WEPC_MSRetentionPolicyBrief_2023.pdf
The gains are real, but some better definitions are in order. Last year's held back kids do not in fact show up in this year's fourth grade results, but they do the following year. If the improvement were just a statistical artifact then there would be a one time jump in performance for the first year (that the 4th grade did not include the bottom 10% of the entering class). Subsequent years might show a decline as the poor students graduate into the next 4th grade. That is not what the data shows, which is a slow but significant and steady improvement over a decade. Over that time period the one-time effect would disappear.
The article Ritzik cited was not an academic study but a back-of-the-envelope thing by a journalist who didnt disclose his statistical methods of adjusting the data. So i cant really condemn the analysis; i can just raise suspicions. I suspect what he did was in effect to put every held-back third grader back into the fourth grade score which would of course eliminate the effect of the reform. But the whole point of the reform was to make sure that fourth graders could "read to learn" and not still be trying to "learn to read". Kevin Drum (the author) could argue that the fourth graders who were held back had an extra year to learn and therefore shouldn't be compared to the nation's fourth graders that were not held back. Hence the definition quibble. But this is an obvious point about kids spending am extra year in school.
Had it been the case that Mississippi implemented the retention policy in 2021, just before the 2022 NAEP administration, then his claim MIGHT be the case (see below about other interventions taking place simultaneously). However, Mississippi started holding third graders back in 2013. Both the 2017 and 2022 NAEP showed improvement in reading. In both of these tests would be students who were held back but then went on to take the 4th grade NAEP exam. This study out of Boston University showed that the retention policy had an effect on reading through grade 6: https://excelined.org/wp-content/uploads/2023/02/WEPC_MSRetentionPolicyBrief_2023.pdf
Further, his analysis assumes that retention was the primary mechanism for improvement on the test scores. But that isn't the case, in 2013, Mississippi implemented a number of other interventions to improve reading some of which are described in this Ed Week piece: https://www.edweek.org/teaching-learning/mississippi-students-surged-in-reading-over-the-last-decade-heres-how-schools-got-them-there/2023/06
There is no way to isolate the impacts of any of these on test score improvement given they were implemented simultaneously..
This also means there’s a percentage of the third-graders in the “miracle” class who have now mastered the content who were held back.
Learning is more important than looking good on a test score at a point in time.
Passing young folks on without mastery has proven its failure.