The General examfghtrefcwetiofe …
Let’s try again:
Sparked by the report in the Herald-Sun, the matrix error (and the who cares “of-of” error) on the General Mathematics Exam 2 has obviously received a lot of attention. This has included apologies from VCAA, from the CEO of VCAA, and from the Minister of Education. It has also led to VCAA announcing that all students would receive the mark for the question.
We haven’t looked at the exam question (or the exam), and so know nothing of the question beyond what commenters have suggested. Even without knowing the details, however, there seem some obvious things to note, or at least they will be obvious for the regulars at this blog and for the similarly disenchanted.
First of all, for VCE mathematics teachers who have put up with untold years of obfuscation and outright denial, VCAA’s apologies were truly astonishing. This is clear evidence that something is happening behind the scenes. We have no idea if it relates to this critique and the surrounding events, but it is reasonable to contemplate.
Secondly, for Methods and Specialist teachers the incredible fuss over the General matrix error is absolutely hilarious, in a black humour sort of way. We’ll assume that the error is significant enough that something had to be done, but there’s no way that the General error is worse than many past screw-ups that were simply ignored. Forever. How, for instance, could the General matrix screw-up be worse than a 2022 Specialist multiple choice question with no correct answer and with the exam report simply pretending otherwise? Again, the dramatic change in VCAA attitude with the General matrix error indicates something is happening behind the scenes.
Finally, a quick note about VCAA’s reassurance to students. In their statement announcing that all students would be awarded the matrix mark, VCAA went on to say,
“This was the most effective and appropriate way to ensure no student will be disadvantaged and that the assessment process is fair, valid and reliable.”
Similarly, on ABC radio, VCAA’s CEO was at pains to reassure students that all would be put right:
“We’ll make sure that no students are disadvantaged through that error.”
To which the only response is: bullshit.
As the Age report makes admirably clear, it is simply impossible for VCAA to ensure that the grading of the matrix question will be “fair” or “valid”, that no one will be disadvantaged. That is the nature of garbled exam questions: it is almost always impossible to properly repair the damage by manipulating the grading.
So, sure, VCAA can give everybody the mark. But that will include the kids who sweated over the garbled question and lost unknown time and composure, as well as the kids who ploughed through without blinking an eye, as well as the kids who simply ignored the question entirely. To treat all these kids the same is not remotely fair or valid. And of course, in an ATAR ranking competition, to less-deservedly award a mark to some kids is to disadvantage the kids who more deserved the mark.
None of this is to suggest that VCAA made the wrong decision, that they could have made a better choice. But VCAA should stop pretending they’ve accomplished more than they have.
There are some signs that VCAA might have gotten significantly better at telling the truth. But, they have a ways to go.
My daughter sat that exam. She tried to explain to me how she approached that matrix problem, having noticed the catastrophe of the question. I didn’t understand the problem or the approach she described. I sat ‘Maths A’ in 1987, and wept bitter tears knowing that that was the end of my formal maths education
Sigh.
As someone who enjoys black/dark humour, I’m cackling at this whole ordeal 😂
I’m too invested to cackle, but it certainly is fascinating.
VCAA apologized to the schools in their letter, but not in their script to be sent out to students.
It is interesting they have formally apologized this time. I have seen errors in all 3 studies and have been an assessor when these have come up in training.
Usually just brushed off as ‘we feel it didn’t impact the students/many students.
“we feel it didn’t impact the students/many students” just doesn’t cut it.
If even a single student is impacted, that is one student too many. We know for a fact from the QCAA experience:
that it is impossible to truthfully claim that no students are impacted. To make such claims is total bullshit and needs to be called out as total bullshit. There have been too many people walking around with their pants on fire for far too long.
The cohort consists of individuals. Each individual has worked hard to try and achieve their hopes, dreams and ambitions. To smooth out the impact on the individual by applying psychometric pseudo-babble to the cohort is unethical, immoral and WRONG. QCAA has shown what is possible.
As for the Gniel interview with Raf Epstein on 774 ABC, I don’t see how any listener could walk away not feeling anger and contempt. I honestly felt that what was said before the interview even started was trying to suggest that VCAA errors helped build resilience in students. What was said towards the end of the interview was beyond the pale and truly insulting and I’m glad Raf called it out.
Yea I totally agree, it’s probably why they’ve kept all the assessor training online. There were always lengthy, heated arguments in the ‘in person’ sessions between the chief team and us over the errors and compensation for the students. Now they can just mute us.
Language, please. I used “bullshit”, once, because it was an instance where no other word worked. It was not and is not license for you or others to do so.
However:
a) Thanks for noting the QCAA event and my post on it. It’s not the point I was making here. (QCAA didn’t properly fix their error either, since that was impossible.) But, it is of course a hugely important point. In particular the “errors” from last year, which, whatever you want to call them, screwed students up and VCAA damn well knows it screwed students up, infuriates me.
b) Yes, Gniel in his interview on ABC was pretty oily. I don’t think it helped him.
Ministers (and the public service) get daily media summaries. Everyone will know about this.
What happened here is different than what happened in Queensland, however. The QCAA question wasn’t wrong, but the answer was, so QCAA had a relatively simple task of just regrading papers. Here, where the question is fundamentally flawed, the problem can’t be fixed just by regrading papers. I wonder what QCAA would have done had this error occurred over there.
I assume the same thing. I think it was dumb luck that the QCAA question wasn’t wrong: it just happened that the actual answer was one of the options. If there had been no correct answer, the only logical choice is to give everyone the same mark.
Yes, the change from brushing off to grovelling apology is incredible.
For what it’s worth, here is what I think:
Unlike the errors on the 2022 Specialist Mathematics exams (and many of the errors on past Maths Methods and Specialist Maths exams), the errors on the 2023 General Mathematics Exam 2 can be clearly understood by the general public (including all the students themselves!). And therefore the errors have resonated with the media and the public and captured the public interest in a way that cannot be ignored or denied. Plus, General Mathematics has the second largest cohort of students (after English) out of all subjects – tens of thousands of students are enrolled in it. That’s a massive number of people directly affected. Add their families, relatives etc. as stakeholders and you’ve got hundreds of thousands of people affected either directly or indirectly.
I think “grovelling apology” was forced upon the VCAA by sheer weight of public opinion (and ‘the optics’). And of course, if “something is happening behind the scenes”, then a fair amount of kindling may already be laid.
I think these errors are one of the most spectacular ‘own goals’ ever.
Really? You think the nature of the error this year was the difference? It’s possible, but there were plenty of Further errors in the past that were way worse. And I haven’t even been looking.
Yes. But after reading JJ’s comment below, I think what he says was also a – very – significant factor.
Hi BiB – I remember early in my public service career seeing an executive hand over an apology letter about a mistake that caused the Minister significant extra work – that’s the sort of humiliation you work very hard to avoid in the public service. Ministers work hard to ensure they have authority over the department (though that’s a lot easier these days as ‘frank and fearless’ is a very distant memory).
My suspicion is that this is a big deal as it’s after last year’s media coverage. Certainly a typo helps a bit, but normally that would be ‘one of those things’. While it’s hard for the general public to understand, having proper mathematicians criticise the mistakes makes a real difference.
In response to previous media coverage, the old Minister would have sought an assurance that all had been fixed. VCAA responded with a typical ‘review’/cover-up by employing tame consultants (that’s how Deloitte get their money). That strategy only works if it doesn’t obviously happen again. It has happened again and got more media coverage.
Add to that a new Minister on the make, keen to set an example to his bureaucrats of who’s in charge and avoid negative media coverage.
I think the technical term we used to use was that the Minister would have ‘put a rocket up VCAA’.
It’s very much a watch this space and for Marty and Burkhard to plan the next move.
Well done Marty and Burkhard for getting this far – you have achieved an enormous amount already from your hard work. How far real change occurs will be interesting to see and will depend on how the war goes from here.
It’s “Burkard”, not “Burkhard”. But, at least you didn’t refer to him as “Poulster”.
Thanks, JJ. I’m less than convinced we’ll win this war, but Burkard and I still have some ammo.
After his efforts, maybe the double typo Wurkhard is more fitting.
My apologies to Burkard – will get it right next time (unlike VCAA).
You’ve achieved a huge amount given there’s only two of you. The real issue is the need for allies and harnessing the maths teachers’ real voice given that the peak groups like MAV seem to be in bed with Government.
I was joking. But there was an in joke there, which will soon be an out joke.
Thanks, JJ. Yes, the need for allies is crucial, and the MAV has been way, way worse than useless. But, there are other routes, other armies. We’ll see.
If it had been Poultry, one might think that the in-joke was that the hens are coming home to roost.
Of course if you were a standard lobby group, you’d commission Deloitte to do a report with analysis (ie simple spreadsheet with outlandish assumptions) proving that VCAA’s errors will cost the economy $x billion over the coming years.
I’ll check our budget and get back to you.
It will be very interesting to see what happens after the SM exams. It’s pretty much guaranteed that VCAA will severely stuff up logic and proof.
Yes, an excellent point. That is a minefield, even for people who know what they’re doing.
Wandering further off-topic, if there is to be a stuff-up I think it’s more likely to be in the solutions/marking scheme than the question. The proof by induction question on the 2023 Exam 1 was routine but decent. The unknown marking scheme (distribution of 4 marks) is where my curiosity lies. I have no doubt that the algebra and calculus ‘mechanics’ for the base case and the
calculation will be fine, it’s what VCAA deems is an acceptable structure of the proof that makes me wonder.
That is an excellent point. The lack of full solutions and a full grading scheme quickly, or ever, is always a problem. But for induction, it’s way way worse than usual.
Considering VCAA’s sample material, I assume they’d allow pretty much any structure for an induction proof, even if they don’t clearly point out the base case, the inductive step, or define a proposition.
And if they don’t, they’ll have to answer to the schools, teachers and students who relied on that VCAA webinar.
VCAA are utterly useless. They have one job- to get these things right, and they can’t do it. Used to know someone who worked there, who said it was all a bureaucratic bludge.
The CEO must go as a minimum. To come out on the ABC and dish up that is inexcusable. Also, I’m thinking that these maths exams were outsourced to some mob here or overseas and no checking of them was done internally when VCAA received them before distributing them. All in all a complete disaster.
Hi FOS. It’s not like that: “outsourcing” is not the issue. VCAA doesn’t have the expertise to write these exams, and shouldn’t be expected to. The (one) problem is that VCAA don’t have the expertise/will/structure to find people with the required expertise.
As for who to blame, I think it is way more problematic than one or two individuals. Sure, the buck stops in VCAA with the CEO, and his apology was not even close to accurate or adequate. But i think the problems are deeply structural and cultural. They have been thirty years in the making, at least.