A few months back, there appeared an article in The Conversation titled,
It’s common to ‘stream’ maths classes. But grouping students by ability can lead to ‘massive disadvantage’
The article is by Elena Prieto-Rodriguez, a professor of mathematics education at the University of Newcastle. As is typically the case with Conversation pieces on education, Prieto-Rodriguez’s article is poorly written, self-indulgent and thin. The article is easily forgettable and we haven’t been able to forget it. It has been bugging us.
Prieto-Rodriguez’s article is on streaming, separating kids into different classes based upon (perceived) ability: what Americans call “tracking” and the English call “setting”. As the title suggests, Prieto-Rodriguez is not a fan. Prieto-Rodriguez discusses her own small-time surveying, which is not indicative of much of anything, but she then refers to more researchy research. On the basis of this research, Prieto-Rodriguez voices two major criticisms of streaming, the first being on the issue of equity:
International research has also found streaming students is inequitable.
As a 2018 UK study showed, students from disadvantaged backgrounds are more likely to be put in lower streamed classes.
The UK study to which Prieto-Rodriguez links is titled,
The symbolic violence of setting: A Bourdieusian analysis of mixed methods data on secondary students’ views about setting
Yes, we’ve bumped into Bourdieusian analysis before, and the Bourdieusian analysis here is even worse. Written by an octet of nitwits, the UK study is unreadable. There is also no need to read the study, because of course Prieto-Rodriguez is correct.
Streaming, when it is done seriously and formally, typically begins in Australia in early secondary school, but the reality is that the kids are effectively streamed long, long before that. They are streamed by whether or not they have attended a primary school that had any interest in teaching proper arithmetic, and they are streamed by whether or not their parents have the concern and/or the time and/or the money to make up for the primary school’s failings. Kids are thus channeled, by culture and by wealth and by dumb luck, years before the secondary schools have had any say over the matter.
This is a disgraceful state of affairs, which can only be addressed by the nuclear bomb reform of schooling, requiring reform of the curriculum and resources and teacher training and testing. Until all that is done – roughly when hell freezes over – the secondary schools are left with two options for their younger kids: a school can ignore the inequity and stream according to who they’ve got; or, a school can avoid streaming for a few years, until the kids “stream” into the (relatively) higher maths and the middle maths and the veggie maths. The first option, as indicated by Prieto-Rodriquez and by any sense of reality, pretty much dooms the lower streamed kids to the lower streams forever. The second option, however, involves the school lying through its teeth, and with the end result pretty much the same.
A solid fraction of Year 7 students have no hope of getting to higher level mathematics, some because of the lack of innate ability, and most because of having never been given a shot and it is too late or damn close to too late for this to be rectified. Secondary schools know this and they routinely lie about this. Is there then nothing secondary schools can do? Are they really so powerless? Close, but no.
Secondary schools can, and should, do three things. First, they should stop lying to parents. All schools do it and they should all be thoroughly ashamed for doing it. Secondly, the secondary schools should recognise Year 7 as the platform or the cliff: the overwhelming majority of secondary students who can be saved, can be saved in Year 7 or never. This then requires telling ACARA and VCAA to go to hell if and when required, and instilling in students the proper arithmetic and algebraic knowledge and skills, reinforced with proper, solid practice and proper, solid testing. Thirdly, schools should marshal their strongest resources for Year 7. The school’s very best teachers should be an integral part of Year 7 mathematics, including that the school’s very best teachers should be an integral part of trying to bring the weaker students up to whatever speed they are capable of attaining.
Is any of this an argument against streaming? Yes, at least in practice. As long as lower streamed students are provided crappy, “this’ll do” subjects, taught by crappy, “this’ll do” teachers, there is a very strong fairness argument for revolting against a Let Them Eat Dreck mentality. But in principle, and even with secondary schools having been given the hospital handpass of woefully under-prepared Year 7 students, streaming could and should be a good thing, particularly for strong and/or keen students. There is nothing in principle about streaming that precludes all students being treated with proper respect and concern. Streaming should simply permit a teacher to focus on like-ability students, to guide them as is needed and to challenge them as is suitable, and to not drive herself nuts with the façade of “differentiation”. Prieto-Rodriquez, however, disagrees even on the principle.
Prieto-Rodriguez’s second argument against streaming is that it doesn’t do what is intended:
A 2009 review of research studies found that not streaming students was better for low-ability student achievement and had no effect on average and high-ability student achievement.
So, a meta-analysis determined that streaming is bad for the lower kids, something that no one requires a meta-analysis to understand; streaming needn’t be bad for the lower kids but, with the way schools currently work, it would be a shock if it were not. But, the meta-analysis also found that streaming failed to benefit the average and above-average kids. Which has to be utter nonsense.
There is not a snowflake’s chance in hell that this latter conclusion is true, and it is why Prieto-Rodriguez’s article has been bugging us. One can appreciate that it’s not all one way, that non-streaming can have its benefits, even for above-average kids; the possibility of peer teaching and the like is a very good thing and should be considered in any calculations. But still, all in all, and whether or not the real-world inequity is a strong enough counter-argument, streaming will benefit the stronger kids. However it happened, the meta-analysis is wrong.
The meta-analysis is by a guy called Ning Rui and is titled,
Four decades of research on the effects of detracking reform: Where do we stand?—A systematic review of the evidence
Rui concludes (p 181),
The main finding from this review is that detracking, or heterogeneous-ability grouping, was beneficial to low ability students in terms of enhancing their academic achievement without being detrimental to the high- and average- ability students. Therefore, a major implication is that heterogeneous grouping should be encouraged and promoted, especially in schools where the lower-track classes have been traditionally assigned fewer resources and less qualified teachers. This review does not support the competing claims that the performance of higher achieving students would decrease as a result of detracking.
Greg Ashman, who also gave the Bourdieusian analysis a slap, simply notes that a second meta-analysis, conducted in 1988 by Chen-Lin Kulik and James Kulik, comes to a very different conclusion from Rui’s. Sure. Contradictory conclusioning is par for the course in education research and everyone believes the “evidence” they wish to believe. We knew that there was a snowball’s chance in hell that Rui’s analysis proved what he claimed and so, as did Ashman, of course we were happy to go along with Kulik and Kulik’s meta-analysis. Based on fifty-two studies, the most telling conclusion of Kulik and Kulik’s meta-analysis was (p 425),
Although grouping had only a small overall effect on student achievement, one subgroup of studies produced especially clear effects. In this type of study students of high ability, or “gifted” students, were put into a special honors class for enriched instruction in their secondary school subjects. Studies of this type usually reported significant results, and they usually reported effects on achievement that were medium in size. High-ability students apparently benefited from the stimulation provided by other high- aptitude students and from the special curricula that grouping made possible.
Exactly as one would expect. Still, we wanted to know how Rui arrives at his absurd conclusion. So, foolishly, we looked.
Rui’s 1996 paper is very detailed and very careful with his discussion and his statistical analysis. He appears to have taken great care in considering the type of studies on offer, and in analysing and weighting them accordingly. We’re not the person to judge, but if Rui has performed a snow job with the stats, it’s a very impressive job. The underlying question is, what has Rui chosen to analyse.
Rui’s analysis is based upon fifteen studies. That would appear small in comparison to the Kuliks’ fifty-two studies, but it is possible that Rui selected more carefully (p 167):
Preference was given to experimental and quasi-experimental studies that compared detracked groups (or heterogeneously grouped classes) with tracked groups (or ability-grouped classes) … In this review, an experimental study is defined as a study where each subject is randomly assigned to an intervention group or a control group before the implementation of the intervention. A quasi-experimental study shares many characteristics of a true experimental study, except for the lack of random assignment.
So, the concern was to focus upon studies for which a proper comparison of outcomes could be made. Fair enough. Except, the region where comparisons are most difficult or are absent entirely may be precisely where the benefits of streaming have their strongest effect, where the stronger kids are going far beyond or far deeper than the other kids. The very first of Rui’s “a priori criteria” for a study to be included in his analysis reads (p 167),
Only studies of comprehensive detracking that incorporated most or all students were included in the review. In other words, studies on programs for gifted children, special education programs, and remedial programs were excluded due to their lack of generalization.
It thus appears that Rui threw aside exactly the types of studies that the Kuliks found to have exhibited the strongest effects of streaming. Can one be more sure? With some work, yes.
In discussing his findings in relation to “average or high-achieving students”, Rui refers to ten of his fifteen studies (pp 170-173). We dug up these ten studies, which was far from easy, and we looked. We didn’t read the studies with great care, but we think we read enough. Two of the studies, an experimental study on social science and a quasi-experimental study on reading, appear to not have involved anything close to mathematics teaching and we did not investigate them further. On the other eight studies, in summary:
- Slavin and Karweit is a 1985 experimental study, on primary mathematics teaching, which compared a within-class ability-grouped teaching program to both a teacher-assisted individualised teaching program and, in a second experiment, to a whole-class teaching control group. Slavin and Karwein found the ability-group teaching and individualised teaching performed similarly, and significantly better than the control group (p 362). In a very thoughtful discussion section, Sklavin and Karwein consider some of the surprising outcomes of their study. In any case, their study points, if anywhere at all, to the benefits of ability grouping.
- Kissoon-Singh is a 1996 experimental study, which considered “cooperative computer-based instruction” in a Year 7 science unit. Students were randomly placed in three groups: homogeneous high ability; homogeneous average ability; and heterogeneous. Amongst other findings, Kissoon-Singh found that the high ability students in the heterogenous group slightly, but significantly, outperformed the high ability students in the homogeneous group (p 94). This is perhaps a surprise until one considers that the “teaching” here constituted computers and group work. It would appear that the heterogenous group provided strongly the potential benefits of heterogeneity, including the opportunity for peer teaching, while the homogeneous group provided almost none of the benefits of ability grouping.
- Burris, Heubert and Levine is 2006 a quasi-experimental longitudinal study, comparing streamed students to heterogenous-and-accelerated students, in regard to the later participation in and achievement in advanced mathematics classes. They found that the latter group performed significantly better (p 117), which seems to prove nothing beyond that it’s a good idea to challenge all students.
- Hawkins is a 1999 “observational study”, which considered the grades of students who were streamed by reading ability in Year 7 and then were detracked for Year 8. Hawkins found that average grades in most subjects remained about the same, but that mathematics grades improved dramatically (p 36). Of the hundred possible explanations, God only knows which is the actual one.
- Mulkey, Catsambis, Steelman and Crane is a very complicated 2005 longitudinal analysis, based upon a large body of US national data. There are a zillion variables, and any conclusion should be taken with a large dose of salt. Overall, however, the authors conclude a strong positive correlation of Year 8 mathematics tracking with Year 12 mathematics performance, albeit with some negatives as well (p 164).
- Argys, Rees and Brewer is a 1996 longitudinal study based upon the same data as the previous study, and comes with the same caveats. In any case, the authors suggested there was a negative correlation of Year 10 mathematics tests scores with detracking, particularly for stronger students (pp 636-638).
- Angrist and Lang is a 2004 study of a program sending lower achieving, minority students to more affluent, predominantly white schools. The authors found no evidence that the presence of the minority students had a negative effect on the (maths or other) grades of the other students (pp 1631-1632). Which is great. What Rui imagines this tells us about detracking, however, God only knows.
- Hallinan is an observational study of five secondary schools published in 2003, but for which Rui refers to a 2000 draft. Hallinan considers the effect of ability placing in five secondary schools; her main concern and her main conclusion, à la Burris, is that on average all students do better in higher ability groups, that it is worth challenging everyone (pp 118-120). In the 2000 draft, however, Hallinan makes a passing comment, which Rui picks up on, that “Students from schools that did not ability group in eighth grade do considerably better than other students [in 9th grade]” (p 15). Hallinan seemingly places zero weight on this observation, and the final, 2003 version states why: the non-tracked students were mostly from Catholic schools, which tend to score higher on the relevant tests (p 116).
That is it. We may have missed a nuance or two, but as near as we can tell that is all the evidence for Rui’s conclusion that the cessation of streaming would not be detrimental to high-achieving students. Which, of course, was a conclusion that Prieto-Rodriguez was happy to go with. Because that’s what maths ed research is all about: finding the evidence for whatever one wishes to believe, no matter how absurd, no matter how contrary to own-eyes Chico reality that belief may be. And which is why we have no intention of repeating this long and painful exercise.
I have never disagreed with your arguments and congratulate you on your efforts to bring attention to the nonsense that passes for mathematics education, but please get your spelling and grammar right. “Homogenous” is an incorrect spelling of “homogeneous”.
Whoops. Slap deserved and corrections (3/4) made.
I am always cautious about claims that are “evidence based”.
What do you trust if not evidence?
I think it is fair to say some people have started using selected evidence to make a case that should not be made. You will find this in any anti vax or flat earth treatise.
What you want is a conclusion based on the best available evidence and that addresses the best counter arguments for the conclusion.
The BS detector becomes a proof by example- it’s bullshit if they avoid easy to find counter evidence or alternative conclusions supported by the evidence.
We see that a lot with counters to Greg Ashman’s conclusions. They generally fail to address what he actually writes or the evidence he presents.
My reasons for being cautious are as follows.
First, sometimes the evidence is not convincing when you look at it more closely. The experiment is poorly designed, or the statistical results are merely correlations.
Second, the philosophical problem of induction makes me cautious. There is no logical reason for believing that the future will resemble the past. We do so simply because it seems like a good idea.
Third, there is a tendency to reduce problems to one explanatory variable. If you change X then you will change Y. But what about A,B,C, etc?
I did not say that I don’t trust evidence. I said that I was cautious about claims that are “evidence based”.
Your second reason is kinda nuts, but otherwise, sure.
I guess the question is, why is the use of the phrase “evidence based” such a reliable indicator of nonsense.
The reason ‘evidence-based’ is something of an indicator is that it implies the user may not really understand the scientific method and its context. ‘Evidence-informed’ implies that the writer may understand the sheer complexity of education and the difficulty of RCTs etc etc in this context. Humility and nuance go a long way in education (as in much of life). Don’t get me started on ‘Evidence-based policy’.
Streaming bad, differentiation good. Got it!
My only experience of differentiation working is when the textbook sets three levels of questions and the kids can choose their level (subject of course to monitoring).
I find it hard to understand how you can teach multiple lessons at once.
Has anyone successfully done it – or is it yet another buzzword where the system puts responsibility for catering for multiple levels at once firmly on the teacher?
I’d argue that streaming, done well, is peak differentiation: different content, different environment, different outcomes, different teaching… it hits all the criteria.
Of course. Prieto-Rodriguez is claiming, on the basis of Rui’s meta-analysis, that streaming-differentiation doesn’t work, even for the good kids, which was the main subject of the post. But ignoring that, of course streaming is differentiation.
But the main argument against streaming is that it is inequitable, and I have some commie sympathy with that position. As I argued, I don’t think streaming need be inequitable, but in practice it seem clearly to be so.
Differentiation is pushed by the department as a way to further exploit the labor of its workforce. Don’t provide the material resources that would enable the education system to support all students in practical ways, instead just say: it’s your job to do more with less (and if you can’t manage it, you’re bad at your job or you don’t care enough about the students).
I agree SRK – to me it’s a way of managing the rhetoric of dealing with increased expectations (eg meet individual student needs) in an industrial system (which it should be – we don’t have the money to meet ‘individual needs’ – that’s a parent’s job). Particularly in a mainstream system which is somehow supposed to cater for a much more diverse student population without the resources needed to do that. It’s perhaps most evident in maths.
So much of the modern world like OHS and compliance ‘training’ often protect the organisation rather than the employee.