First, enjoy some great Kraftwerk, because, and just because:
Regular readers may recall Australian reporter Natasha Bita. Natasha did some really excellent stenographic work for ACARA. Natasha also played right along with AMSI’s most recent Chicken Little crusade. It turns out that Natasha is an excellent stenographer even when there’s nothing to stenograph. Continue reading “We Are the Robots”
A few days ago Greg Ashman handballed an article to us, suggesting we might enjoy it, although clearly he meant “enjoy” it. The (paywalled) article, just published in the journal Research in Mathematics Education, is titled
Yeah, you don’t have to read the article. We did.
Continue reading “PoSWW 28: The Intersection of Feminism With PISA”
The 2019 TIMSS results are just about to be released, and the question is should we care? The answer is “Hell yes”.
TIMSS is an international maths and science test, given at the end of year 4 and year 8 (in October in the Southern Hemisphere). Unlike PISA, which, as we have noted, is a Pisa crap, TIMSS tests mathematics. TIMSS has some wordy scenario problems, but TIMSS also tests straight arithmetic and algebra, in a manner that PISA smugly and idiotically rejects.
The best guide to what TIMSS is testing, and to what Australian students don’t know and can’t do, are the released 2011 test items and country-by-country results, here and here. We’ll leave it for now for others to explore and to comment. Later, we’ll update the post with sample items, and once the 2019 results have appeared.
UPDATE (08/12/20)
The report is out, with the ACER summary here, and the full report can be downloaded from here. The suggestion is that Australia’s year 8 (but not year 4) maths results have improved significantly from the (appalling) results of 2015 and earlier. If so, that is good, and very surprising.
For now, we’ll take the results at face value. We’ll update if (an attempt at) reading the report sheds any light.
FURTHER UPDATE (08/12/20)
OK, it starts to become clear. Table 9.5 on page 19 of the Australian Highlights indicates that year 8 maths in NSW improved dramatically from 2015, while the rest of the country stood still. This is consistent with our view of NSW as an educational Switzerland, to which everyone should flee. We’re not sure why NSW improved, and there’s plenty to try to figure out, but the mystery of “Australia’s” dramatic improvement in year 8 maths appears to be solved.
UPDATE (09/12/20)
OK, no one is biting on the questions, so we’ll add a couple teasers. Here are the first two released mathematics questions from the 2011 year 8 TIMSS test:
1. Ann and Jenny divide 560 zeds between them. If Jenny gets 3/8 of the money, how many zeds will Ann get?
2. 
(The second question is multiple choice, with options 0.043, 0.1043, 0.403 and 0.43.)
To see the percentage of finishing year 8 students from each country who got these questions correct, you’ll have to go the document (pp 1-3).
There’s much we could write about Matthew Bach, who recently gave up teaching and deputying to become a full-time Liberal clown. But, with great restraint, we’ll keep to ourselves the colourful opinions of Bach’s former school colleagues; we’ll ignore Bach’s sophomoric sense of class and his cartoon-American cry for “freedom”; we’ll just let sit there Bach’s memory of “the sense of optimism in Maggie Thatcher’s Britain”.
Yesterday, Bach had an op-ed in the official organ of the Liberal Party (paywalled, thank God). Titled We must raise our grades on teacher quality, Bach’s piece was the predictable mix of obvious truth and poisonous nonsense, promoting the testing of “numeracy” and so forth. One line, however, stood out as a beacon of Bachism:
“But, as in any profession, a small number of teachers is not up to the mark.”
We is thinking that is very, very true.
A few days ago we received an email from Aaron, a primary school teacher in South Australia. Apparently motivated by some of our posts, and our recent thumping of PISA in particular, Aaron wrote on his confusion on what type of mathematics teaching was valuable and asked for our opinion. Though we are less familiar with primary teaching, of course we intend to respond to Aaron. (As readers of this blog should know by now, we’re happy to give our opinion on any topic, at any time, whether or not there has been a request to do so, and whether or not we have a clue about the topic. We’re generous that way.) It seemed to us, however, that some of the commenters on this blog may be better placed to respond, and also that any resulting discussion may be of general interest.
With Aaron’s permission, we have reprinted his email, below, and readers are invited to comment. Note that Aaron’s query is on primary school teaching, and commenters may wish to keep that in mind, but the issues are clearly broader and all relevant discussion is welcome.
Good afternoon, my name is Aaron and I am a primary teacher based in South Australia. I have both suffered at the hands of terrible maths teachers in my life and had to line manage awful maths teachers in the past. I have returned to the classroom and am now responsible for turning students who loathe maths and have big challenges with it, into stimulated, curious and adventure seeking mathematicians.
Upon commencing following your blog some time ago I have become increasingly concerned I may not know what it is students need to do in maths after all!
I am a believer that desperately seeking to make maths “contextual and relevant” is a waste, and that learning maths for the sake of advancing intellectual curiosity and a capacity to analyse and solve problems should be reason enough to do maths. I had not recognised the dumbing-down affect of renaming maths as numeracy, and its attendant repurposing of school as a job-skills training ground (similarly with STEM!) until I started reading your work. Your recent post on PISA crap highlighting how the questions were only testing low level mathematics but disguising that with lots of words was also really important in terms of helping me assess my readiness to teach. I have to admit I thought having students uncover the maths in word problems was important and have done a lot of work around that in the past.
I would like to know what practices you believe constitutes great practice for teaching in the primary classroom. I get the sense it involves not much word-problem work, but rather operating from the gradual release of responsibility (I do – we do – you do) explicit teaching model.
I would really value your thoughts around this.
Warm regards,
Aaron
We’ve posted on the general nature of PISA’s mathematics questions here and here, and the main point is the sheer awfulness of what is being tested. One question, however, seemed worthy of special note. The following is the first of the PISA 2012 test questions included in this document of past questions, followed by a guide to its grading.
Below are two “units” (scenarios) used in the PISA 2012 testing of mathematics. The units appeared in this collection of test questions and sample questions, and appear to be the most recent questions publicly available. Our intention is for the units to be read in conjunction with this post, and see also here, but of course readers are free to comment here as well. The two units below are, in our estimation, the most difficult or conceptually involved of the PISA 2012 units publicly available; most questions in most other units are significantly more straight-forward.
Here’s an interesting tidbit: PISA‘s mathematics testing doesn’t test mathematics. Weird, huh? Who knew?
Well, we kinda knew. Trustworthy colleagues had suggested to us that PISA was slanted, but finding out the extent of that slant, like lying-on-the-ground slant, was genuinely surprising. (We’re clearly just too optimistic about the world of education.) Not that we had any excuse for being surprised; there were clues of mathematical crime in plain sight, and it was easy enough to locate the bodies.
The first clues are on PISA’s summary page on “Mathematics Performance“. The title is already a concern; qualifications and elaborations of “mathematics” usually indicate some kind of dilution, and “performance” sounds like a pretty weird elaboration. Perhaps “mathematics performance” might be dismissed as an eccentricity, but what follows cannot be so dismissed. Here is PISA’s summary of the meaning of “mathematical performance”:
Mathematical performance, for PISA, measures the mathematical literacy of a 15 year-old to formulate, employ and interpret mathematics in a variety of contexts to describe, predict and explain phenomena, recognising the role that mathematics plays in the world. The mean score is the measure. A mathematically literate student recognises the role that mathematics plays in the world in order to make well-founded judgments and decisions needed by constructive, engaged and reflective citizens.
The alarms are set off by “mathematical literacy”, a pompous expression that promises more than, while signalling we’ll be getting much less than, straight mathematics. All doubt is then ended with the phrase “the role that mathematics plays in the world”, which is so fundamental that it is repeated verbatim.
What this sums to, of course, is numeracy, the noxious weed that inevitably chokes everything whenever there’s an opportunity to discuss the teaching of mathematics. What this promises is, akin to NAPLAN, PISA’s test of “mathematical performance” will centre on shallow and contrived scenarios, presented with triple the required words, and demanding little more than simple arithmetic. Before investigating PISA’s profound new world, however, there’s another aspect of PISA that really could do with a whack.
We have been told that the worldly mathematics that PISA tests is needed by “constructive, engaged and reflective citizens”. Well, there’s nothing like irrelevant and garishly manipulative salesmanship to undermine what you’re selling. The puffing up of PISA’s “world” mathematics has no place in what should be a clear and dispassionate description of the nature of the testing. Moreover, even on its own terms, the puffery is silly. The whole point of mathematics is that it is abstract and transferrable, that the formulas and techniques illustrated with one setting can be applied in countless others. Whatever the benefits of PISA’s real world mathematics for constructive, engaged and reflective citizens, there will be the exact same benefits for destructive, disengaged psychopaths. PISA imagines Florence Nightingale calculating drip rates? We imagine a CIA torturer calculating drip rates.
PISA’s flamboyent self-promotion seems part and parcel of its reporting. Insights and Inpretations, PISA’s summary of the 2018 test results, comes served with many flavours of Kool-Aid. It includes endless fussing about “the digital world” which, we’re told, “is becoming a sizeable part of the real world”. Reading has changed, since it is apparently “no longer mainly about extracting information”. And teaching has changed, because there’s “the race with technology”. The document wallows in the growth mindset swamp, and on and on. But not to fear, because PISA, marvellous PISA, is on top of it, and has “evolved to better capture these demands”. More accurately, PISA has evolved to better market itself clothed in modern educational fetishism.
Now, to the promised crimes. The PISA test is administered to 15 year old students (typically Year 9 or, more often, Year 10 in Australia). What mathematics, then, does PISA consider worth asking these fifteen year olds? PISA’s tests questions page directs to a document containing questions from the PISA 2012 test, as well as sample questions and questions from earlier PISAs; these appear to be the most recent questions made publicly available, and are presumably representative of PISA 2018. In total, the document provides eleven scenarios or “units” from the PISA 2012 test, comprising twenty-six questions.
To illustrate what is offered in those twenty-six questions from PISA 2012, we have posted two of the units here, and a third unit here. It is also not difficult, however, to indicate the general nature of the questions. First, as evidenced by the posted units, and the reason for posting them elsewhere, the questions are long and boring; the main challenge of these units is to suppress the gag reflex long enough to digest them. As for the mathematical content, as we flagged, there is very little; indeed, there is less mathematics than there appears, since students are permitted to use a calculator. Predictably, every unit is a “context” scenario, without a single straight mathematics question. Then, for about half of the twenty-six questions, we would categorise the mathematics required to be somewhere between easy and trivial, involving a very simple arithmetic step (with calculator) or simple geometric idea, or less. About a quarter of the questions are computationally longer, involving a number of arithmetic steps (with calculator), but contain no greater conceptual depth. The remaining questions are in some sense more conceptual, though that “more” should be thought of as “not much more”. None of the questions could be considered deep, or remotely interesting. Shallowness aside, the breadth of mathematics covered is remarkably small. These are fifteen year old students being tested, but no geometry is required beyond the area of a rectangle, Pythagoras’s theorem and very simple fractions of a circle; there is no trigonometry or similarity; there is no probability; there are no primes or powers or factorisation; there are no explicit functions, and the only implicit functional behaviour is linear.
D = dv/(60n) .
(The meaning of the variables and the formula needn’t concern us here, although we’ll note that it takes a special type of clown to employ an upper case D and a lower case d in the same formula.)
There are two questions on this equation, the first asking for the change in D if n is doubled. (There is some WitCHlike idiocy in the suggested grading for the question, but we’ll leave that as a puzzle for the reader.) For the second question (labelled “Question 3” for God knows what reason), students are given specific, simple values of D, d and n, and they are required to calculate v (with a calculator). That’s it. That is the sum total of the algebra on the twenty-six questions, and that is disgraceful.
Algebra is everything in mathematics. Algebra is how we name the quantity we’re after, setting the stage for its capture. Algebra is how we signify pattern, allowing us to hunt for deeper pattern. Algebra is how we indicate the relationship between quantities. Algebra is how Descartes captured geometry, and how Newton and Leibniz captured calculus.
It is not difficult to guess why PISA sidelines algebra, since it is standard, particularly from numeracy fanatics, to stereotype algebra as abstract, as something only within mathematics. But of course, even from PISA’s blinkered numeracy perspective, this is nonsense. You want to think about mathematics in the world? Then the discovery and the analysis of patterns, and the analysis of relationships, of functions is the heart of it. And what makes the heart beat is algebra.
Does PISA offer anything of value? Well, yeah, a little. It is a non-trivial and worthwhile skill to be able to extract intrinsically simple mathematics from a busy and wordy scenario. But it’s not that important, and it’s hardly the profound “higher order” thinking that some claim PISA offers. It is a shrivelled pea of an offering, which completely ignores vast fields of mathematics and mathematical thought.
PISA’s disregard of algebra is ridiculous and shameful, the final stake in PISA’s thoroughly nailed coffin. It demonstrates that PISA isn’t “higher” or “real”, it is just other, and it is an other we would all be much better off without.
At current count, there have been two thousand, one hundred and seventy-three reports and opinion pieces on Australia’s terrific PISA results. We’ve heard from a journalist, a former PISA director, the Grattan Institute, the Gonski Institute, the Mitchell Institute, ACER, the Innovative Research University Group, The Centre for Independent Studies, the AMSI Schools Project Manager, the Australian Association of Mathematics Teachers, the Australian Science Teachers Association, Learning First, an education journalist, an education editor, an education lecturer, a psychometrician, an education research fellow, a lecturer in educational assessment, an emeritus professor of education, a plethora of education academics, a shock jock, a shock writer, a federal education minister, a state education minister, another state education minister, a shadow education minister, an economist, a teacher and a writer.
So, that’s just about everyone, right?
A few days ago the Sydney Morning Herald published yet another opinion piece on Australia’s terrific PISA results. The piece was by Richard Holden, a professor of economics at UNSW, and Adrian Piccoli, formerly a state Minster for Education and now director of the Gonski Institute at UNSW. Holden’s and Piccoli’s piece was titled
‘Back to basics’ is not our education cure – it’s where we’ve gone wrong
Oh, really? And what’s the evidence for that? The piece begins,
A “back to basics” response to the latest PISA results is wrong and ignores the other data Australia has spent more than 10 years obsessing about – NAPLAN. The National Assessment Program – Literacy and Numeracy is all about going back to basics ...
The piece goes on, arguing that the years of emphasis on NAPLAN demonstrate that Australia has concentrated upon and is doing fine with “the basics”, and at the expense of the “broader, higher-order skills tested by PISA”.
So, here’s our message:
Dear Professors Holden and Piccoli, if you are so ignorant as to believe NAPLAN and numeracy is about “the basics”, and if you can exhibit no awareness that the Australian Curriculum has continued the trashing of “the basics”, and if you are so stuck in the higher-order clouds to be unaware of the lack of and critical need for properly solid lower-order foundations, and if you can write an entire piece on PISA without a single use of the words “arithmetic” and “mathematics” then please, please just shut the hell up and go away.