As I wrote about briefly, in December last year Burkard and I wrote a critique of the VCE mathematics exams, which was to form the basis of VCAA’s external review. As I’ve just written, here, dealing with VCAA was incredibly frustrating. More importantly, the overview VCAA released to us has entirely hidden the substance of the external review, and/but the review seems to be nothing more than distraction and whitewash. In particular, there still seems to be no clear appreciation within VCAA that mathematical truth is a thing, and that properly qualified mathematicians are by far the best judges of that thing.
After VCAA sent us their overview, and after having received no meaningful reply from VCAA’s CEO, Burkard and I decided the next step was to approach Victoria’s Minister of Education. As support for this, Burkard set up an open letter for university mathematicians to sign. The letter was limited in scope, focussing on what Burkard and I referred to as five major errors on the 2022 exams, and on the implications of these errors and VCAA’s subsequent mishandling of these errors.
The open letter and associated documents live on Burkard’s and my shared website, QEDcat.com. The open letter is here, the list of signatories is here, our full exam critique is here, the document just on the “five major errors” (including the vacuous exam report excerpts) is here, and the overview of VCAA’s external review is here.
Burkard and I have been in touch with the Minister’s office, with the open letter in support. Nothing has eventuated yet, but we are hopeful.
The text of the open letter follows.
*****************************************
In December 2022, two mathematicians, Professor Burkard Polster and Dr. Marty Ross, wrote a critique of the 2022 VCE mathematics exams. Polster and Ross’s critique covers many aspects of the exams and their administration but, in particular, identifies and discusses what they refer to as five major mathematical errors on the exams.
We are university mathematicians, we have read the exam questions at issue, and we agree with Polster and Ross. Whatever term one wishes to use, all five questions are unacceptably flawed. Each question exhibits some fundamental misunderstanding or misrepresentation of the underlying mathematics. Each question, at minimum, would have created unnecessary confusion, with a subsequent loss of time for at least some students, and probably many. It is difficult to imagine how the questions could have been graded in a fair and consistent manner, and in any case the flaws are simply, on a mathematical basis, unacceptable.
We have also read the exam reports on these questions, and they are as concerning as the questions themselves. The reports are completely silent on the flaws in the questions. This is professionally unacceptable and it is also exacerbating the damage. Current students look to past exams and reports for guidance; in these instances, the 2022 exams contain significant misguidance.
As university mathematicians, we are acutely aware of how difficult it is to write mathematics exams that are both clear and correct. Errors will inevitably occur from time to time. When errors occur, however, it is critical to be open and honest about it, and to do one’s best to mitigate the damage.
It is also critical, of course, to have the proper procedures in the first place, to do as much as one can to avoid the occurrence of serious errors. For the production of senior school mathematics exams, this proper process requires the intimate involvement of competent and attentive mathematicians. These mathematicians must be properly involved in both the writing and the vetting of the exams. The 2022 exams are evidence that such procedures are not currently in place in Victoria.
We urge the Minister to take this matter seriously. We urge the Minister to consult with the Victorian Curriculum and Assessment Authority, along with senior Victorian mathematicians, with a view to revamping the current processes for the writing and vetting of VCE mathematics exams.
Sincerely,
The list of signatories is here.
Nice work…
I’m sick of trying to clarify their BS solutions and lack of clarity to my students.
I’m sick of explaining to students that the marking schemes change on a yearly basis.
I’m sick of the ambiguity in a subject, that shouldn’t have ambiguity.
I think everyone needs to continue to make noise until things change…
We’re lucky to have people like you guys sticking up for us.
Can we look at getting them to create a clear study design next? Can we actually know exactly what needs to be taught and it’s not hidden under layers of BS?
Thanks. Please watch your language, even in acronym form.
In NSW, there is a HSC exam writing committee that sets exams. There are university mathematicians who serve on this committee (I know some of them personally). Does this not happen in Victoria?
Hi, Cathy. I don’t know that there’s a permanent committee in Victoria, and I don’t know who’s on the committee, permanent or otherwise. But, whatever is being done and whoever is involved, it’s evidently not working.
So your exams are written by eduquacks?
No wonder they are a mess.
Again, I have no idea. All I know are the consequences.
I’m pretty sure many of the exams are written by teachers who are currently in a school but not teaching the subject…
In the past there’s been Further/ Methods teachers writing Specialist Maths Exams…
You still wonder… SURELY there’s someone on the panel who can see the blatant errors… saying they are small “grammatical errors” that affect the whole question is a joke.
I’m pretty sure many of the exams are written by teachers who are currently in a school but not teaching the subject…
In the past there’s been Further/ Methods teachers writing Specialist Maths Exams…
You still wonder… SURELY there’s someone on the panel who can see the blatant errors… saying they are small “grammatical errors” that do not affect the whole question or have a small impact is a joke.
Doing this is good. Fingers crossed.
Thank you, thank you, thank you.
Even if nothing comes of all this, thank you.
Hi Marty – this is excellent – well done – there are some heavy hitters in the maths world there.
I suspect that it may be helpful to prepare a broader contextual case as to why the Minister should do something (could be sent as a letter to him). This may help link university academics’ arguments with ‘real world’ implications – it’s not only high school maths and pedantry. You will find this obvious but people who don’t understand maths may not. In these games, every little bit matters. The Minister is a former lawyer, may have to convince Cabinet colleagues and will be up against an entrenched bureaucracy (and a couple of ‘state maths bodies’).
Points that could be made include: affects the reputation of Victoria as an education destination, Victoria’s economy needs strong maths as part of STEM with maths underpinning genomics etc etc, need for strong maths skills and ecosystem demonstrated by COVID, maths underpinning modern world and importance of STEM – AI etc, comments from this blog about teachers being sick of having to explain problems to students, unfairness to students, hard to strive for excellence if exam is full of errors, unfairness in system tends to disadvantage those most already disadvantaged, innovation relies on really high level maths skills, doesn’t happen in other states, self-referential VCAA no match for international realities, hard to inspire confidence to lift numeracy (NAPLAN, PISA) if can’t even get exams right etc etc.
Hi, JJ. Thanks for the kind words and thanks very much for the advice. Yes, it was really pleasing to have some big time mathematicians sign the open letter. And I definitely take your point about “broader context”. If and when Burkard and I get to meet the Minister (presumably more likely his advisors), we’ll definitely keep this in mind.
This is excellent advice!
The Herald Sun has written about this letter here.
Very good. Thanks, Bugle.
When you meet the minister/minions, could you spare 30 seconds to ask them about private schools systematically cheating in VCE english? I wrote to Merlino about it a couple of years ago, but not very much seems to have changed.
I jest of course – you’d best not cause them to have to think too much – I’m just jealous that you might actually be getting somewhere.
Feel free to email me about such concerns.
https://www.theage.com.au/national/victoria/disappointing-education-minister-orders-inquiry-into-vce-exam-bungles-20231113-p5ejio.html
I believe Marty’s seen this, but interesting to note nonetheless. Specifically “Education Minister Ben Carroll has called for a full investigation into errors in this year’s VCE exams, saying he is disappointed in a series of “stuff-ups” that should never have occurred.” and “Carroll said he had met VCAA representatives since the exam period had begun and would meet the authority again on Monday. “I am disappointed,” he said. “The stuff-ups should not have occurred in the first place. That’s why I’ve asked for a full review.””
Yes. Thanks, Sai. It’s not clear what Carroll has in mind, and if he’ll want to dig deep enough, but it seems a good sign.