1. Rosenshine’s Success Principle
There has been a discussion of late, here and at Greg Ashman’s place and in Elon’s sewer, of whether Rosenshine’s principles are simply general principles – principles which guys like me consider basically good by being basically common sense – or whether Rosenshine’s principles amount to more, um, explicit instructions, which are thereby more open to specific critique. The answer would appear to be either, or neither, depending upon which of the three will sell better in a given context.1
This is a *long* post. It was not intended to be but there just seemed to be a lot to say about, in the end, very little. In any case, if the reader is not sufficiently wise to turn away now and go back to the cat videos, then they should pour themself a large mug of coffee or a stiff drink and settle in.
We’ll consider, as has Ashman, one specific Rosenshine principle, on classroom “success rate”:
7. Obtain a high success rate: It is important for students to achieve a high success rate during classroom instruction.
Stop the presses.
This is a good and important principle, of course, but it is also difficult to see how it is more than common sense. True, as Ashman points out,2 not everybody advocates or follows this principle, but that does not negate the principle’s truismness. That common sense is not now so common does not alchemise the sense into precious wisdom. In any case, Ashman’s straw man has been soundly defeated: Rosenshine cannot mean anything, and we can all go home.
There is supposedly more to Rosenshine’s success principle than the coffee mug slogan above, however, and this is where things get tangled. Rosenshine’s success principle is fleshed out with two sections: Research findings; and In the classroom. We have to consider what this all means, if anything, as well as what people might imagine it means.
1. There is a parallel question of what precisely is being advocated by Donovan and Carroll and the other bullhorns for “explicit teaching”, but that very important question will have to keep for another post.
2. Ashman makes his point by referring to a discussion of “productive struggle” in an NCTM booklet (p 48), but it is not a great example and Ashman overplays his hand. Firstly, the NCTM booklet is from 2014; Ashman should be making the point with some more recent indicator of egregious practice. Secondly, and without getting into whether there are non-horrible versions of “productive struggle”, the NCTM discussion is concerned with how a teacher should react initially to a student’s “lack of immediate success” (emphasis added). There is no suggestion that the student should not eventually succeed, just a question of how and how quickly that success should be achieved, as well as what that “success” can and should look like. As such, Ashman’s claim that “we cannot be orchestrating productive struggle whilst also adhering to Principle 7” may be true for some advocate’s definition of orchestrating struggling but the claim does not follow from anything Ashman points to in the NCTM passage. Still, for whatever it is worth, Ashman’s point holds; there is little doubt that too many teachers pay too little attention to Rosenshine’s success principle.
2. In the Classroom
In the classroom consists of some (reasonable) argument for the success principle, an anecdote and some more specific advocacy, including some semi-advocacy. The advocacy consists of noting that the “most effective teachers” attain a desired “success level” by chopping the content of their lessons into more easily consumable chunks, and ensuring success – through explanation and sufficient practice – chunk by chuck. This is all clear and sensible, at least as a default,3 and fits in with Rosenshine’s second principle, which is specifically on the consumable chunks thing. More interesting and less clear is Rosenshine’s semi-advocacy for a teaching technique known as “mastery learning”:
Unless all students have mastered the first set of lessons, there is a danger that the slower students will fall further behind when the next set of lessons is taught. So there is a need for a high success rate for all students. “Mastery learning” is a form of instruction where lessons are organized into short units and all students are required to master one set of lessons before they proceed to the next set. In mastery learning, tutoring by other students or by teachers is provided to help students master each unit. Variations of this approach, particularly the tutoring, might be useful in many classroom settings.
There is some obvious sense in mastery learning, even if there are some less obvious issues; we’ll consider mastery learning a bit more below. But, is mastery learning part of Rosenshine’s Principles™? It doesn’t matter whatsoever for teaching, of course. But, given there are inspectors around checking teachers’ passports for the Rosenshine visa, it’d be helpful to know.
3. Broadly stated principles can can be fundamentally sound but end up looking pretty silly when confronted with the reality of specific topics in specific disciplines. While comfortable with the principle, if this chunk principle were to be instituted – or, much worse, mandated – in too inflexible a manner, I think some of the teaching outcomes may be pretty horrendous. Not everything is so easily or so naturally chunkable.
3. Research Findings
The Research findings section is much more problematic,4 which Ashman acknowledges, but I believe the problems are more fundamental that Ashman suggests. Making some sense of this section involved a lot of chasing stuff down, and it is still very muddy, but I can quickly summarise. It now feels to me that Rosenshine’s success principle, to the extent that it is intended to be anything more than broad common sense, is built on quicksand.
We’ll consider the Research findings bit by bit.5 It begins,
In two of the major studies on the impact of teachers, the investigators found that students in classrooms with more effective teachers had a higher success rate, as judged by the quality of their oral responses during guided practice and their individual work.
Even before hunting for the “two major studies”, one has to decipher Rosenshine’s very clumsy sentence. It would seem that “higher success rate” is being defined as the “quality” of in-class “individual work” and “oral responses”, but how are the “more effective teachers” determined? Are they being defined as those with this “higher success rate”, or is there some external criterion? If so, what is it, and is the suggestion that there is simply a correlation between the two or also some causal connection? It turns out to be the latter (with a strong suggestion of causation of the “effective teaching” by the in-class success), but it is far from obvious.6
Now to the “two major studies”. Rosenshine provides two general references for his success principle, and we’ll get to them, but pretty clearly they are not the major studies referred to here.7 it appears likely that the major studies are those referred to in a 1983 paper by Rosenshine (p 341):8
Not only is the frequency of teacher questions important, but the percentage of correct student responses is also important. One of the major findings of the BTES study (Fisher, Berliner, Filby, Marliave, Cahen, & Dishaw 1980) was that a high percentage of correct answers (both during guided practice and when working alone) was positively correlated with achievement gain. Similarly, Anderson et al. (1979) found that the percentage of academic interactions where the student gave the correct answer was positively related (r = .49) to achievement gain.
The Beginning Teacher Evaluation Study was a correlational study conducted on forty-six second year and fifth year reading and mathematics classes, with a range of teacher and student behaviours observed. A key element of the study was what the authors refer as Academic Learning Time, defined to be “the amount of time [in the classroom] a student spends engaged in an academic task that s/he can perform with high success”. The estimate of “student learning” was determined by subsequent (mostly multiple choice) tests. Among their findings, the authors report that “the amount of time a student spends engaged in an academic task that s/he can perform with high success is positively associated with student learning” (p 13). The authors explain and discuss this finding, although no quantification or supporting data is provided.
The paper by Anderson et al. concerned a control study of first-year reading.9 The purpose of the study was to test the effectiveness of a reading program based upon twenty principles of instruction, including a number of principles on calling out children and real-time evaluation of the children’s success (pp 196-198). The program was undertaken by seventeen classes, with a control group of ten classes. As measured by pre- and post-study tests, the study group outperformed the control group (Table 1). The study observed many classroom behaviours, including “error rates”, being “[the] percent of all academic interactions that included correct answers” (p 204). As noted by Rosenshine, this (lack) of error rate was found to strongly correlate with student achievement (p 205).
So, with some tiny, inconsequential, hardly-took-any-time-at-all hunting, Rosenshine’s first sentence has been substantiated. One might query how supportive is this evidence, and perhaps there are other studies out there that are less supportive,10 but I for one am very happy to move on (and never needed convincing in the first place).
On to the second sentence:11
In a study of fourth-grade mathematics, it was found that 82 per cent of students’ answers were correct in the classrooms of the most successful teachers, but the least successful teachers had a success rate of only 73 per cent. A high success rate during guided practice also leads to a higher success rate when students are working on problems on their own.
Two sentences in one shot. We’re really zipping along.
Once again, Rosenshine’s meaning is far from clear, although, now being practiced in Rosenshinian, we can take a decent guess. And, once again, we have to hunt for Rosenshine’s reference, which, once again, is not either of the references he has provided. But, at least this time Rosenshine has provided some clues.
It is possible that Rosenshine is taking his percentages from a 1979 control study by Good and Grouws.12 The Key Instructional Behaviors for the study group included a focus on “seatwork”, with a clear directive to “provide uninterrupted successful practice” (p 357), but the published article does not appear to provide any indication of success rates. Another possibility is that Rosenshine is referring to a 1977 correlational study by Good and Grouws; in this study, a comparison of the “top and bottom nine” out of forty-one teachers, yielded correct response rates of 83% and 76% (Table 1).13 Whatever. Close enough, I guess.
Two sentences to go.
The research also suggests that the optimal success rate for student achievement appears to be about 80%.
God knows where this one is from. I simply have no clue.14
And, except for the final sentence, intended to support the penultimate sentence, and which we’ll address later, that is it for the Research findings.15 What did we learn?
4. The Research findings sections in the 2010 and 2012 versions of Rosenshine’s principles are identical except for some grammatical tidying.
5. This first sentence took me a good two hours to semi-decipher. One might ask why education academics cannot provide proper references, with pinpoints when obviously helpful. The cynical answer is that it is deliberate, that education research evidence is so intrinsically muddy and inconsistent and challengeable, the academics are consciously or unconsciously reluctant to make any evidence easily accessible for scrutiny. That may not be true of Rosenshine but, whatever the reason, his referencing here is laughable.
6. A version of this sentence appears in a 1986 article (p 67) by Rosenshine (and a legible transcript here), an earlier attempt to formalise his principles. This earlier version of the sentence is even more confusing, naturally reading as if attaining a high in-class success rate provides the definition of effective teachers. But, at least the 1986 article includes (unpinpointed) references (one of which is inaccurate), so the intended meaning of the sentence can (eventually) be determined.
7. Neither of the two references provide either of the probable major studies as a further reference.
8. For a non-paywalled copy, see here (p 7), or a 1986 updated version of the paper by Rosenshine and Stevens (p12).
9. I could not find a non-paywalled link to the paper, but a 1976 outline of the research can be accessed here.
10. Rosenshine’s 1986 article, which contains a version of the 1983 passage quoted here (p 67), also includes reference to a third study, by Gersten, Carnine and Williams (1982). This study is based upon the Direct Instruction Follow Through Model. The authors note some indication of statistical unreliability of the “student accuracy” data (p 71), but I did not try to decipher it; the authors suggest the unreliability is due to a “design flaw”.
11. Yes, I am aware that this is ridiculous.
12. Christian Bokhove, a professor of mathematics education who likes to needle Ashman (and vice versa), has previously done the detective work on this, and more successfully than me. Rosenshine’s 1986 article gives the percentages as 82% and 73% (p 67), referencing the 1979 study by Good and Grouws.
13. Rosenshine’s 1983 paper has percentages of 82% and 76% (p 342), referencing the 1977 study by Good and Grows; see also the Rosenshine and Stevens version, which quotes the same percentages (p 12), referencing an earlier, statistical report for the same 1977 study. The Rosenshine paper also notes that these percentages were not replicated in a 1980 secondary school study.
14. Rosenshine’s 1986 article suggests an optimal success rate of “around 75-80 per cent” but also provides no reference.
15. Noting the above problems with Rosenshine, Ashman recommends relying upon “other summaries of the same research base”. Perhaps. But, for the evaluation of Rosenshine’s principles, this is not so much moving the goal posts as it is changing the soccer field.
4. Rampaging Robert Slavin
Before summing up, it is worth taking a brief detour and considering the two references Rosenshine provides in support of his Success Principle. The references may be pretty useless for the suggested purpose but they turn out to be very interesting.
The first article is a long and very interesting 1980 survey by Norman Frederiksen on the teaching of “problem solving”. That is a dangerous and oft-perverted term, of course, but Frederiksen states early on the different meanings (pp 336-337). He first distinguishes between “ill-structured” and “well-structured” problems; the former he puts in with “the fuzzy problems that are frequently encountered in real life”, and which we would describe as “that ACARA and VCAA junk”. The latter are the mathematically sound problems. Frederiksen then distinguishes the well-structured problems that are essentially straight-forward from harder problems, which require “productive thinking”: essentially the distinction between exercises and (multi-step) problems, with of course a spectrum in between and with the classification of a “problem” very much dependent upon the intended solver. I have not read Frederiksen’s paper closely but it would appear to be the worth the effort. I will just note that the Rosenshine Principle literature appears to have very little to say on the teaching of harder problems, making Rosenshine’s inclusion of the Frederiksen reference more than a little ironic.
Rosenshine’s second reference, a 1987 article by Anderson and Burns, is plain weird. The article is a missile fired in a rather fierce battle over mastery learning and makes little sense in isolation. The battle as a whole, however, is fascinating.
The mastery learning battle was launched in 1987 with Mastery Learning Reconsidered, an analysis of mastery learning by Robert Slavin, a prominent educational psychologist for many decades: that is a photograph of Slavin at the top of the post. While sympathetic to the goals and the methods of mastery learning, Slavin was sceptical of both the research evidence for mastery learning and the underlying principle.
There is no way to do justice here to Slavin’s arguments and those of his many combatants. In brief, Slavin was unconvinced that the purposely designed post-tests typically employed in mastery learning studies provided an accurate evaluation of the learning; and, Slavin was not convinced that the time-cost of ensuring mastery for all was properly compared to that of alternative programs (including his own).
The battle was long and complicated, and I do not know who was right. But, at minimum, I think Slavin required a good answer and I am not convinced that the mastery learning defenders provided one. If I were a betting man, my money would be on Slavin. In any case, the battle is well worth a read.16
Mastery learning is not the same as Rosenshine’s principles, of course. But for the sake of Slavin’s questions, it seems a reasonable proxy. i believe that there are comparable questions that the Rosenshine guys should rightly answer.
16. The missiles are mostly paywalled, alas, but here they are: (i) The 1987 launch by Slavin; (ii) a 1987 response in the same journal issue by Guskey; (iii) a 1987 response in the same journal issue by Anderson and Burns (the article Rosenshine references); (iv) a 1987 re-response in the same journal issue by Slavin; (v) a 1988 interview with Slavin, with responses from Guskey, Bloom, Block, Walberg and Hunter; (vi) a 1989 article by Slavin, responding to Guskey, Bloom, Block, Walberg and Hunter; (vii) a 1990 meta-analysis of mastery learning by Kulik, Kulik and Bangert-Drowns including responses to Slavin; a 1990 article by Slavin to Kulik, Kulik and Bangert-Drowns.
5. Nothing Succeeds Like Fake Success
Anyone still around? We’re almost done, and then we (maybe just I) can go to bed.
It’s time consider the final sentence of the Research findings. Recall, the penultimate line:
The research also suggests that the optimal success rate for student achievement appears to be about 80%.
The suggestion seems reasonable enough, even if the supporting research is nowhere to be found. But, then comes the final line in support:
A success rate of 80 per cent shows that students are learning the material, and it also shows that the students are challenged.
No, a success rate of 80% cannot show any such a thing. It is this claim that fundamentally nails why I do not trust Rosenshine’s principles beyond a general sensibleness.
I am happy with the idea that students should be generally successful but not 100% successful in classroom response. It seems farcical to fuss over an optimal percentage, or even how to imagine a percentage could be so precise, but I have no problems with 80% as a ballpark rate. There are two fundamentally different ways, however, that a teacher might achieve that 80-ish per cent.
First, a teacher might explain not much at all, and then not ask much at all of the students. Or, a teacher might consider the material in significantly greater depth and expect the students to master the material to a significantly greater depth. This second teacher can still teach mostly chunky, they can still explain in bits and require mostly correct answers from the students, while simply expecting and encouraging a significantly greater effort from the students. The two groups could have the exact same 80%, but with one group significantly more challenged. The 80% itself tells us nothing about proper challenge.
This is the fundamental problem with Rosenshine as a program, and with its advocates. It, they, simply do not evidence any concern for content, for whether the mathematics is to be learned to any decent depth. By far, the greatest problem in Australian mathematics is the poverty of the curriculum and the consequent under-challenging of students. But, rather than seeking to address this poverty in some serious manner, they simply ignore it; Australia’s Rosenshinians are happy to wallow in this inch-deep pool. They play a game that means they benefit from it.
Rosenshine cannot be the be all and end all, certainly not in the mechanised form it is taking on in Australia. The deeper the water, the more one has to be flexible with Rosenshine and to allow sensible flow. But flexibility, to say the least, is not what these guys are about. And they are scared to death of deep water.
6. It’s the Vibe, and it’s Not the Vibe
Finally, one last word on Rosenshine as principle or program. Anderson and Burns criticise Slavin for not appreciating the spirit underlying mastery learning. Quoting another paper on mastery learning, they remark (p 215),
Slavin seems unaware of the values inherent in mastery learning and, ultimately, the conduct and interpretation of mastery learning research. Anderson and Block (1977) have defined mastery learning as “both a philosophy of school learning and an associated set of specific instructional practices” (p. 163). In delineating the characteristics of mastery practices, however, Slavin seems unwilling to examine the underlying belief systems critical to the implementation of mastery learning in a school or district (Block, 1985).
Anderson and Block and Burns see no problem with this dual attitude but I do, and I think Slavin did. I don’t think Slavin had any trouble appreciating the “underlying belief systems” of mastery learning, I think he simply refused to conflate this belief system(s?) with “an associated set of specific instructional practices”, and that he was perfectly capable of being very sympathetic to one while being very critical of the other. I think the mastery learning guys had to make up their minds what “mastery learning” properly was. And, very similarly, I think the Rosenshine guys have to make up their minds.
Plenty of us are happy with and very much appreciate the Rosenshinian vibe. It’s when Rosenshine becomes an associated set of specific instructional practices that we think the Rosenshinians may have lost the plot, transforming good principle into absurd, handcuffing fetish. So, maybe decide which it is: principles or specifics. Then the rest of us can decide whether we’re on the same team. Currently, it seems not.
So, without defining “success” and by defining a “high rate” to be “about 80%” and then giving a couple of academic papers from 50 years ago (OK, so not quite that far back) an argument is somehow made?
Sorry, I’m not buying it.
Even if I agree with parts of it – there is too much “trust me, this is true” here. Although, to be fair, I always found Hattie much the same.
They define “success” on a given question or exercise as being correct.
I’m not sure I consider that a definition. I get that they do. I’m not a fan.
I think they’re definition of “success” is clear enough and serves their purpose, although I agree the connotations are a little congratulatory. What I think is much more concerning is their use and sense of the word “challenge”.
Yes, well… that is also a bit self-referential.
None of it fills me with any desire to read more of their work.
(Or even ponder the implications, really).
Having spent a week of my life reading this stuff, I *certainly* having no desire to read any more.
I am not sure I fully understand what Rosenshine is actually offering here
‘The most successful teachers have students who achieve roughly 80% on an assessment’ means nothing without trying to define ‘most successful’ or what a solid assessment would actually do, nor is it a good way to compare students unless all students are doing the same assessment. Then, as you mention Marty, what the hell does he mean by ‘challenged’? And how does a score of 80% define that challenge adequately? Would a student getting 30% not also be ‘challenged’, albeit differently to a student getting 80%? He also seems to just ignore mentioning what one does about students who don’t master previous content and/or don’t achieve 80% on an assessment (such as the aforementioned student getting 30%)
One brief side note: I *hate* writing academic papers for the fact that lines such as ‘a high percentage of correct answers… was positively correlated with achievement gain’ have to be backed up by a citation. I don’t know how many times I wrote an essay and got a note saying ‘where is your citation?’ for something I thought to just be common sense. I still struggle to identify when the hell I am actually supposed to cite something. I guess I’ll never be a true academic (thank God)
Thanks, Fox. Rosenshine’s writing is crappy, although this does not imply his principles are useless (I don’t think they are), or even that the more specific Rosenshinisms are useless (I fear they may be).
I don’t know where your “roughly 80%” quote comes from, and I’d prefer to deal with precise quotes, or at least clear paraphrasing.
The term “most successful teachers” in the quote I provided seems to mean the the teachers whose classes did better on post-study tests and/or the classes in the control groups (having done better than classes in the control groups). The wording is far from clear, but I think what is being described is possibly kosher, keeping in mind Slavin’s strong reservations on the post-study evaluations.
The “challenge” thing is a real issue. These guys simply have no interest in or appreciation for content, for what is actually being taught. The underlying idea of the 80% is fine: be clear in the classroom on what students are to learn, and have them practise at a suitably mostly achievable level, and check. But they then pretend the meaning of this 80% challenge thing is something that it not remotely is.
One of the things I noticed when reading all this forty year old stuff – and I read a *lot* – is how much better their writing was than the current nitwittery.
Yes, my apologies regarding the ‘roughly 80%’ quote – I appear to have conflated two separate ideas from Rosenshine there. I do not think my point regarding that would survive scrutiny anyway
I do get what Rosenshine meant by ‘most successful teachers’, but I don’t appreciate the vagueness of it and how that ultimately relates to high-achieving students. I suppose I have reservations with basing the success of a teacher solely upon their student cohort’s ability – realistically (at least with the students being referred to by Rosenshine and the studies he references) a successful teacher would be one that demonstrates student *growth* in their learning, at least to me
Ultimately, I do agree that reading older research papers and the like tends to be a lot more interesting and clear than a lot of the stuff being churned out currently. In saying that, I can guarantee you have far more experience in that regard than I do!
I don’t think Rosenshine is being vague in his thought. I think he is simply a poor, lazy writer.
Commenting on Fox: I have told this story before. Forgive me if you have read it previously.
When I was a graduate student, I recall one course which went for a year. Of the 15 members of the class, 13 thought that the lecturer should be sacked because he was so bad. Another student and I thought that he should teach every course because he was so good. And I am confident that we were the best students in the class. Was he a good teacher?
I don’t think you’ve told it here before. Grad school is very different from school school, but your story is nonetheless very good food for thought.
Rampaging Robert Slavin. Well I know how my next couple of hours are going to be spent.
With Robert, or Roy?
Roy. IF I had a Robert itch that I didn’t know needed to be scratched, you have scratched it and I can get on with more important holidaying things. I think maybe Bradbury to start with, then gymnastics and synchronised swimming.
No argument against Roy. But Robert is also worth a look. He was *very* good at pissing off people (and people with a hell of a lot more sense than the current ideologues).
I was actually tempted to look into Robert. But, it’s not going to be a start of holidays thing. In a week or so I’ll come back and begin clicking on links. For now, Roy is where I’m at and very grateful to be there.
Happy to have reminded you of Roy. Also, great to hear from you blogwise: long time, no comment.
Here’s my 2 cents ….
While I know you may disapprove, Claude says of critiques of Rosenshine’s principles “In summary, the strongest critiques cluster around three things: the evidence base being treated as more definitive than it is, the principles being deployed as a surveillance or compliance tool rather than a reflective one, and the framework’s relative silence on creativity, dialogue, and context-dependent pedagogy. Most critics aren’t against the principles per se, but against the uncritical way they’re sometimes adopted.” Which isn’t too far from your analysis.
I do wonder whether any educational research stands up to generalisations. Education is so complex – can it be summarised in a rigorous way that is appropriately cited? Many people certainly seem to think that the “Science of Maths” (and its proponents do acknowledge the limitations of the term) is something new, rather than basically returning to what centuries of refinement have led to.
They are very useful principles (and the choice of term ‘principles’ is appropriately vague) – the sort of thing that should be taught in teacher training courses (but obviously isn’t). They are particularly useful in the current climate.
Thanks, JJ. Claude didn’t do that bad a job, although I think to the extent that I disagree with Rosenshine’s principles, it is pretty different. What I really disagree with is Ashman and co sanctifying everything they do by chanting “Rosenshine”.
I have not read or considered carefully Rosenshine’s other nine principles, nor the seventeen associated semi-principles. The one thing that really pissed me off about Principle #7 was Rosenshine’s casual presumption that an 80% “success rate” in class exercises/questions is necessarily indicative of suitable “challenge”. This seems to me fundamental nonsense, and a nonsense that fits in with Ashmanism. I don’t believe Ashman has the slightest interest in, or even sense of, challenging school mathematics.
But the real point is that I don’t see what Ashman and his kin are doing follows from what Rosenshine wrote, or from the studies Rosenshine relied upon. Of course I could be wrong. Perhaps if I visited Ashman’s school, I’d think differently.