Shuffling NAPLAN’s Deckchairs

We’re late to this, but it’s gotta be done.

Some State education ministers, unhappy with NAPLAN, commissioned a review, which appeared a couple weeks ago. The Review considers many contentious aspects of NAPLAN, but we’ll focus upon “numeracy”, NAPLAN’s homeopathic proxy for mathematics. We’ll leave others to debate “literacy” and the writing tests, and the timing and reporting and so forth.

So, what might the Review entail for the Son-of-NAPLAN testing of mathematics? Bugger all.

Which was always going to happen. For all the endless public and pundit whining about NAPLAN, which is what prompted this latest Review, none of the criticism has been aimed at the two elephants: the Australian Curriculum, which underpins NAPLAN, is a meatless mass of gristle and fat; and “numeracy” is not mathematics, is not arithmetic, and is barely anything. The inevitable consequence is that NAPLAN amounts to the aimless testing of untestable fuzz. As Gertrude Stein would have put it, there is no there there to test.

This misdirection of the Review was locked in by the terms of reference. No mention is made of “mathematics” or “arithmetic”. The single reference in the Terms to “numeracy” is a deadpan call for “the most efficient and effective system for assessing key literacy and numeracy outcomes”, as if this were a clear and unproblematic and worthy goal. It is no surprise, therefore, that the Review gives almost no attention to arithmetic and mathematics, and the meaning(lessness) of numeracy, and indeed works actively to avoid it.

The Review includes a capsule summary of the Numeracy tests, a superficial comparison to PISA and TIMSS, and Australia’s relative performance over time on these tests (pp 34-42). There is no proper exposition, however, of the nature of the tests. There is nothing reflecting the hard fact that NAPLAN and PISA are pseudomathematical garbage. TIMSS, on the other hand, is decidedly not garbage, so what does the Review do with that? That is interesting.

In what could have been a beacon paragraph, the Review compares the Australian Curriculum with expectations on TIMSS:

“… The Australian Curriculum emphasis on knowing and applying is similar to TIMSS but the Australian Curriculum does not appear to cover some of the complexity that is described in the TIMSS framework under reasoning. It seems likely, too, that a substantial number of TIMSS mathematics items are beyond Australian Curriculum expectations for achievement, especially at the Year 4 level.”

In summary, the emphasis on “knowing and applying” mathematics in the Australian Curriculum is just like TIMSS, as long as you don’t really care how much students know, or how deeply they can apply it, or how successful you “expect” them to be at it. Yep, two peas in a pod.

What does the Review then do with this critical paragraph? Nothing. They just drone ahead. Here is the indication that their entire Review is doomed to idiot trivialities, but they can’t see it, or won’t admit it. They see the smoke, note the smoke, but it doesn’t occur to them, or they just can’t be bothered, or it wasn’t in their idiot Terms, to look for the damn gun.

Finally, what of the recommendations proposed by the Review? There are two that concern the testing of numeracy and/or mathematics. The first, Recommendation 2.2, is that authorities

“Rename the numeracy test as mathematics …”

Huh. And what would be the purpose of that? Well, supposedly it would “clarify that [the test] assesses the content and proficiency strands of the Australian Curriculum: Mathematics”. Except, of course, and as the Review itself acknowledges, the Numeracy test doesn’t do anything of the sort. And, even to the minimal extent that it does, it just points back to Elephant Number One, that the Australian Curriculum is not a properly sound basis for anything.

The isolated suggestion to rename a test is of course a distracting triviality. Alas, not all of the Review’s recommendations are so trivial. Recommendation 2.3 proposes a new test, for

“… [the] assessment of critical and creative thinking in science, technology, engineering and mathematics (STEM) …”

Ah, Yes. Let’s test whether ten-year-old Tommy is the new Einstein.

This is a monumentally stupid recommendation. Is Jenny the next Newton? Maybe. But can she manipulate numbers and expressions with sufficient speed and accuracy to hold, let alone mould, a substantial mathematical thought in her head? Just maybe you might want to test for that first? Is Carol the new Capote? Then perhaps first teach her the basics of grammar, first teach her how to construct a clear and correct sentence. Then you can think to tease out all the great works inside her. Is Fritz another Mozart? Gee, I dunno. How are his scales? And on and on.

This constant, idiot call for the teaching of and, worse, the testing of “higher order” thinking, this mindless genuflection to reasoning and creativity, is maddening. It ignores the stubborn fact that deeper thought and creativity in any discipline can only be built upon the craft, upon the basic knowledge and skills of that discipline. The Review’s call is even worse for that, since STEM isn’t a discipline, it’s just a foggy con job.

This Godzilla versus Mothra battle is never likely to end, nor likely to end well. On the one side are the numeracy nuts, who can’t see the value of skills independent of some ridiculous application. On the other side are the creativity clowns, who ludicrously denigrate “the basics”, and ludicrously paint NAPLAN as the basics they’re denigrating. Neither side exhibits any understanding of what the basics are, or their critical importance. Neither side has a clue. Which means, unless and until these two monsters somehow destroy each other, we’re all doomed.

Should Schools Now Be Opening?

Just for a change, we’re asking a non-rhetorical question. So, ignoring the fact that ScoMoFo is a thug, and ignoring the fact that “Dumbo” Dan Tehan is a thug, is it time for schools to reopen?

Our suspicion is that, at least in Victoria, the answer is “no”. We haven’t thought hard about it, however. So, while we (try to find time to) think some more, we’d be interested in what others have to say.

Very quickly, here are the arguments we see for opening schools in Victoria:

  • Federal health officials suggest schools are safe.
  • The NSW study (not yet peer-reviewed) suggests schools are safe.
  • Childminding.
  • Year 12 students are getting seriously dicked around.

Here are the arguments we see for keeping schools in Victoria closed:

  • ScoMoFo is an idiot.
  • Dumbo Dan makes ScoMoFo look smart.
  • Daniel Andrews and his Chief Medical Officer are not idiots.
  • No one has a real sense of what will happen when restrictions ease.
  • The kids (P-11) miss a term of school? Big deal.

DIY Teaching Degrees

Dan Tehan, the Federal minister for screwing up education, has announced a rescue package for Australia’s universities. This was clearly necessary, since the universities are no longer in a position to fleece international students. The package guarantees funding for the universities, and introduces a range of cheap six-month courses in “areas considered national priorities”.

The government’s package is “unashamedly focused on domestic students”. That was inevitable since:

a) the government, and Tehan in particular, doesn’t give a stuff about international students;*

b) Tehan is a born to rule asshole, entirely unfamiliar with the notion of shame.

And, what of these “priority” courses? According to the ABC,

The Government said prices would be slashed for six-month, remotely delivered diplomas and graduate certificates in nursing, teaching, health, IT and science provided by universities and private tertiary educators.

OK, so ignoring all the other nonsense, we have a few questions about those six-month online teaching diplomas:

  • Will such a diploma entitle the bearer to teach?
  • If not, then what is it good for?
  • If so, then what is a school to do with the mix of 6-month diploma-qualified applicants and the standard 24-month Masters-qualified applicants?
  • And, if so, what does that tell us of the intrinsic worth of those standard 24-month Masters?

To be clear, we have no doubt that six months is plenty sufficient for the initial training of a teacher, and indeed is at least five months too many. We also have no doubt that a diploma-trained teacher has the same chance to be a good teacher as someone who has suffered a Masters. They have a better chance, in fact, since there will have been less time to pervert natural instincts and feelings and techniques with poisonous edu-babble.

But, good or bad, who is going to give these diploma teachers a shot? Then, if the teachers should be and are given a shot, who is going to address the contradiction, the expensive and idiotic orthodoxy of demanding two year post-grad teaching degrees?


*) Or anyone, but international students are near the bottom.

Getting Schooled by the Coronavirus

I’ve been busy the last couple of days, and will be for the foreseeable future, since my girlfriend and I have taken our two young children out of school.

I have informed my parent friends of this decision, but I am not advocating that they, or anyone, follow our lead. My girlfriend and I are lucky in that we are financially secure (for now), and are currently freer of work than we might otherwise be.* It is easy for us to bring the kids home, and we could see no good argument against it. Other parents are much less fortunate, and may have a very difficult decision ahead, very soon. I really feel for them, and for everyone dealing with this mess.

This brings up a general and hugely important question: should schools stay open? Honestly, I have no idea. It is an aspect of Australian discussion that I have been trying, and failing, to get my head around. It seems that the main argument for keeping schools open is simply as a childminding service, so that the oldies don’t do the minding and the doctors and the nurses can get on with running themselves ragged. Is that a sufficient argument? I’m sceptical, but I don’t feel confident to say “no”.

Here are two links to articles discussing the matter (in Australia), neither of which I either vouch for or reject:

Why Australia is not shutting schools (The Guardian)

No, Australia is not putting teachers in the coronavirus firing line (The Conversation)

I’m open to people’s thoughts. But, I’ll just add one thing. Prime Scott Morrison has threatened private schools that might close, and he has expressed his confidence in the decision to keep kids at school:

I’m telling you that, as a father, I’m happy for my kids to go to school. There’s only one reason your kids shouldn’t be going to school and that is if they are unwell.

I wonder if all of Morrison’s Liberal colleagues agree.

*) Thank you, to a very stupid university and a very stupid school. I will be forever grateful.

Two PISA Crap

Below are two “units” (scenarios) used in the PISA 2012 testing of mathematics. The units appeared in this collection of test questions and sample questions, and appear to be the most recent questions publicly available. Our intention is for the units to be read in conjunction with this post, and see also here, but of course readers are free to comment here as well. The two units below are, in our estimation, the most difficult or conceptually involved of the PISA 2012 units publicly available; most questions in most other units are significantly more straight-forward.




The Slanted Tower of PISA

Here’s an interesting tidbit: PISA‘s mathematics testing doesn’t test mathematics. Weird, huh? Who knew?

Well, we kinda knew. Trustworthy colleagues had suggested to us that PISA was slanted, but finding out the extent of that slant, like lying-on-the-ground slant, was genuinely surprising. (We’re clearly just too optimistic about the world of education.) Not that we had any excuse for being surprised; there were clues of mathematical crime in plain sight, and it was easy enough to locate the bodies.

The first clues are on PISA’s summary page on “Mathematics Performance“. The title is already a concern; qualifications and elaborations of “mathematics” usually indicate some kind of dilution, and “performance” sounds like a pretty weird elaboration. Perhaps “mathematics performance” might be dismissed as an eccentricity, but what follows cannot be so dismissed. Here is PISA’s summary of the meaning of “mathematical performance”:

Mathematical performance, for PISA, measures the mathematical literacy of a 15 year-old to formulate, employ and interpret mathematics in a variety of contexts to describe, predict and explain phenomena, recognising the role that mathematics plays in the world. The mean score is the measure. A mathematically literate student recognises the role that mathematics plays in the world in order to make well-founded judgments and decisions needed by constructive, engaged and reflective citizens.

The alarms are set off by “mathematical literacy”, a pompous expression that promises more than, while signalling we’ll be getting much less than, straight mathematics. All doubt is then ended with the phrase “the role that mathematics plays in the world”, which is so fundamental that it is repeated verbatim.

What this sums to, of course, is numeracy, the noxious weed that inevitably chokes everything whenever there’s an opportunity to discuss the teaching of mathematics. What this promises is, akin to NAPLAN, PISA’s test of “mathematical performance” will centre on shallow and contrived scenarios, presented with triple the required words, and demanding little more than simple arithmetic. Before investigating PISA’s profound new world, however, there’s another aspect of PISA that really could do with a whack.

We have been told that the worldly mathematics that PISA tests is needed by “constructive, engaged and reflective citizens”. Well, there’s nothing like irrelevant and garishly manipulative salesmanship to undermine what you’re selling. The puffing up of PISA’s “world” mathematics has no place in what should be a clear and dispassionate description of the nature of the testing. Moreover, even on its own terms, the puffery is silly. The whole point of mathematics is that it is abstract and transferrable, that the formulas and techniques illustrated with one setting can be applied in countless others. Whatever the benefits of PISA’s real world mathematics for constructive, engaged and reflective citizens, there will be the exact same benefits for destructive, disengaged psychopaths. PISA imagines Florence Nightingale calculating drip rates? We imagine a CIA torturer calculating drip rates.

PISA’s flamboyent self-promotion seems part and parcel of its reporting. Insights and Inpretations, PISA’s summary of the 2018 test results, comes served with many flavours of Kool-Aid. It includes endless fussing about “the digital world” which, we’re told, “is becoming a sizeable part of the real world”. Reading has changed, since it is apparently “no longer mainly about extracting information”. And teaching has changed, because there’s “the race with technology”. The document wallows in the growth mindset swamp, and on and on. But not to fear, because PISA, marvellous PISA, is on top of it, and has “evolved to better capture these demands”. More accurately, PISA has evolved to better market itself clothed in modern educational fetishism.

Now, to the promised crimes. The PISA test is administered to 15 year old students (typically Year 9 or, more often, Year 10 in Australia). What mathematics, then, does PISA consider worth asking these fifteen year olds? PISA’s tests questions page directs to a document containing questions from the PISA 2012 test, as well as sample questions and questions from earlier PISAs; these appear to be the most recent questions made publicly available, and are presumably representative of PISA 2018. In total, the document provides eleven scenarios or “units” from the PISA 2012 test, comprising twenty-six questions.

To illustrate what is offered in those twenty-six questions from PISA 2012, we have posted two of the units here, and a third unit here. It is also not difficult, however, to indicate the general nature of the questions. First, as evidenced by the posted units, and the reason for posting them elsewhere, the questions are long and boring; the main challenge of these units is to suppress the gag reflex long enough to digest them. As for the mathematical content, as we flagged, there is very little; indeed, there is less mathematics than there appears, since students are permitted to use a calculator. Predictably, every unit is a “context” scenario, without a single straight mathematics question. Then, for about half of the twenty-six questions, we would categorise the mathematics required to be somewhere between easy and trivial, involving a very simple arithmetic step (with calculator) or simple geometric idea, or less. About a quarter of the questions are computationally longer, involving a number of arithmetic steps (with calculator), but contain no greater conceptual depth. The remaining questions are in some sense more conceptual, though that “more” should be thought of as “not much more”. None of the questions could be considered deep, or remotely interesting. Shallowness aside, the breadth of mathematics covered is remarkably small. These are fifteen year old students being tested, but no geometry is required beyond the area of a rectangle, Pythagoras’s theorem and very simple fractions of a circle; there is no trigonometry or similarity; there is no probability; there are no primes or powers or factorisation; there are no explicit functions, and the only implicit functional behaviour is linear.

Worst of all, PISA’s testing of algebra is evidently close to non-existent. There is just one unit, comprising two questions, requiring any algebra whatsoever. That unit concerns a nurse (possibly a CIA torturer) calculating drip rates. Minus the tedious framing and the pointless illustration, the scenario boils down to consideration of the formula

D = dv/(60n) .

(The meaning of the variables and the formula needn’t concern us here, although we’ll note that it takes a special type of clown to employ an upper case D and a lower case d in the same formula.)

There are two questions on this equation, the first asking for the change in D if n is doubled. (There is some WitCHlike idiocy in the suggested grading for the question, but we’ll leave that as a puzzle for the reader.) For the second question (labelled “Question 3” for God knows what reason), students are given specific, simple values of D, d and n, and they are required to calculate v (with a calculator). That’s it. That is the sum total of the algebra on the twenty-six questions, and that is disgraceful.

Algebra is everything in mathematics. Algebra is how we name the quantity we’re after, setting the stage for its capture. Algebra is how we signify pattern, allowing us to hunt for deeper pattern. Algebra is how we indicate the relationship between quantities. Algebra is how Descartes captured geometry, and how Newton and Leibniz captured calculus.

It is not difficult to guess why PISA sidelines algebra, since it is standard, particularly from numeracy fanatics, to stereotype algebra as abstract, as something only within mathematics. But of course, even from PISA’s blinkered numeracy perspective, this is nonsense. You want to think about mathematics in the world? Then the discovery and the analysis of patterns, and the analysis of relationships, of functions is the heart of it. And what makes the heart beat is algebra.

Does PISA offer anything of value? Well, yeah, a little. It is a non-trivial and worthwhile skill to be able to extract intrinsically simple mathematics from a busy and wordy scenario. But it’s not that important, and it’s hardly the profound “higher order” thinking that some claim PISA offers. It is a shrivelled pea of an offering, which completely ignores vast fields of mathematics and mathematical thought.

PISA’s disregard of algebra is ridiculous and shameful, the final stake in PISA’s thoroughly nailed coffin. It demonstrates that PISA isn’t “higher” or “real”, it is just other, and it is an other we would all be much better off without.

Foundation Stoned

The VCAA is reportedly planning to introduce Foundation Mathematics, a new, lower-level year 12 mathematics subject. According to Age reporter Madeleine Heffernan, “It is hoped that the new subject will attract students who would not otherwise choose a maths subject for year 12 …”. Which is good, why?

Predictably, the VCAA is hell-bent on not solving the wrong problem. It simply doesn’t matter that not more students continue with mathematics in Year 12. What matters is that so many students learn bugger all mathematics in the previous twelve years. And why should anyone believe that, at that final stage of schooling, one more year of Maths-Lite will make any significant difference?

The problem with Year 12 that the VCAA should be attempting to solve is that so few students are choosing the more advanced mathematics subjects. Heffernan appears to have interviewed AMSI Director Tim Brown, who noted the obvious, that introducing the new subject “would not arrest the worrying decline of students studying higher level maths – specialist maths – in year 12.” (Tim could have added that Year 12 Specialist Mathematics is also a second rate subject, but one can expect only so much from AMSI.)

It is not clear that anybody other than the VCAA sees any wisdom in their plan. Professor Brown’s extended response to Heffernan is one of quiet exasperation. The comments that follow Heffernan’s report are less quiet and are appropriately scathing. So who, if anyone, did the VCAA find to endorse this distracting silliness?

But, is it worse than silly? VCAA’s new subject won’t offer significant improvement, but could it make matters worse? According to Heffernan, there’s nothing to worry about:

“The new subject will be carefully designed to discourage students from downgrading their maths study.”

Maybe. We doubt it.

Ms. Heffernan appears to be a younger reporter, so we’ll be so forward as to offer her a word of advice: if you’re going to transcribe tendentious and self-serving claims provided by the primary source for and the subject of your report, it is accurate, and prudent, to avoid reporting those claims as if they were established fact.

The NAPLAN Numeracy Test Test

The NAPLAN Numeracy Test Test is intended for education academics and education reporters. The test consists of three questions:

Q1. Are you aware that “numeracy”, to the extent that it is anything, is different from arithmetic and much less than solid school mathematics?

Q2. Do you regard it important to note and to clarify these distinctions?

Q3. Are you aware of the poverty in NAPLAN testing numeracy rather than mathematics?

The test is simple, and the test is routinely failed. NAPLAN is routinely represented as testing the “basics”, which is simply false. As a consequence, the interminable conflict between “inquiry” and “basics” has been distorted beyond sense. (A related and similarly distorting falsity is the representation of current school mathematics texts as “traditional”.) This framing of NAPLAN leaves no room for the plague-on-both-houses disdain which, we’d argue, is the only reasonable position.

Most recently this test was failed, and dismally so, by the writers of the Interim Report on NAPLAN, which was prepared for the state NSW government and was released last week. The Interim Report is short, its purpose being to prepare the foundations for the final report to come, to “set out the major concerns about NAPLAN that we have heard or already knew about from our own work and [to] offer some preliminary thinking”. The writers may have set out to do this, but either they haven’t been hearing or they haven’t been listening.

The Interim Report considers a number of familiar and contentious aspects of NAPLAN: delays in reporting, teaching to the test, misuse of test results, and so on. Mostly reasonable concerns, but what about the tests themselves, what about concerns over what the tests are testing? Surely the tests’ content is central? On this, however, at least before limited correction, the Report implies that there are no concerns whatsoever.

The main section of the Report is titled Current concerns about NAPLAN, which begins with a subsection titled Deficiencies in tests. This subsection contains just two paragraphs. The first paragraph raises the issue that a test such as NAPLAN “will” contain questions that are so easy or so difficult that little information is gained by including them. However, “Prior experimental work by ACARA [the implementers of NAPLAN] showed that this should be so.” In other words, the writers are saying “If you think ACARA got it wrong then you’re wrong, because ACARA told us they got it right”. That’s just the way one wishes a review to begin, with a bunch of yes men parroting the organisation whose work they are supposed to be reviewing. But, let’s not dwell on it; the second paragraph is worse.

The second “deficiencies” paragraph is concerned with the writing tests. Except it isn’t; it is merely concerned with the effect of moving NAPLAN online to the analysis of students’ tests. There’s not a word on the content of the tests. True, in a later, “Initial thinking” section the writers have an extended discussion about issues with the writing tests. But why are these issues not front and centre? Still, it is not our area and so we’ll leave it, comfortable in our belief that ACARA is mucking up literacy testing and will continue to do so.

And that’s it for “deficiencies in tests”, without a single word about suggested or actual deficiencies of the numeracy tests. Anywhere. Moreover, the term “arithmetic” never appears in the Report, and the word “mathematics” appears just once, as a semi-synonym for numeracy: the writers echo a suggested deficiency of NAPLAN, that one effect of the tests may be to “reduce the curriculum, particularly in primary schools, to a focus on literacy/English and numeracy/mathematics …”. One can only wish it were true.

How did this happen? The writers boast of having held about thirty meetings in a four-day period and having met with about sixty individuals. Could it possibly be the case that not one of those sixty individuals raised the issue that numeracy might be an educational fraud? Not a single person?

The short answer is “yes”. It is possible that the Report writers were warned that “numeracy” is snake oil and that testing it is a foolish distraction, with the writers then, consciously or unconsciously, simply filtering out that opinion. But it is also entirely possible that the writers heard no dissenting voice. Who did the writers choose to meet? How were those people chosen? Was the selection dominated by the predictable maths ed clowns and government hacks? Was there consultation with a single competent and attuned mathematician? It is not difficult to guess the answers.

The writers have failed the test, and the result of that failure is clear. The Interim Report is nonsense, setting the stage for a woefully misguided review that in all probability will leave the ridiculous NAPLAN numeracy tests still firmly in place and still just as ridiculous.