Yes, this is Deloitte, Part 2. Last year, while Victoria was in the midst of the 2024 leak debacle, it seems that Tasmania was going through something akin to Victoria’s 2023 admin-typo debacle. A number of “errors” were found on senior exams, leading to Tasmanian Assessment Standards & Certification instigating an independent review by – who would’ve guessed – Dial-a-Leech, Deloitte.
Before getting to the WitCHful part of this post (below the *****), there are a few things to note about Deloitte‘s TASC review:
1) Deloitte completed their review in late May and TASC released the (redacted) review in late September.
This is not exactly speedy and one can ponder the reason for the delay, but TASC seems to have released the review willingly, which is a hell of a lot better than the hide-it-forever strategy attempted by VCAA sleazebags.
2) Deloitte’s TASC review is pretty useless
As with VCAA-Deloitte, and with Blacher, the concern is for process and reliability rather than quality or sanity. So, TASC-Deloitte notes TASC’s underfunding and a prone-to-clunkiness exam production process, and so makes important recommendations, but there is seemingly no analysis of the exam errors or the exams more generally.
These process guys really don’t get it and they can inspire no meaningful reform. But, of course, education authorities such as TASC and VCAA have zero interest in meaningful reform.
To be fair, however:
3) TASC-Deloitte expresses concern for the lack of “curriculum experts” within TASC
The review notes (p 5)
The absence of curriculum experts within TASC results in a reliance on Setting Examiners, Exam Critics and the Assessment Manger, to verify the technical accuracy of exams.
Great. A glimmer of sanity (noting that TASC-Deloitte uses “curriculum experts” to refer to “subject matter experts”). But,
This is a potential gap when compared to jurisdictions like NSW and VIC where curriculum experts sit within their state’s assessment body and are involved throughout the exam development process and act as a point of expertise.
You idiots. You just did a goddam review of VCAA and you are still unaware that there is no one remotely resembling a “curriculum expert” at VCAA? To cement this idiocy, TASC-Deloitte comments on the Department for Education, Children and Young People, which is a separate body and which is responsible for curriculum development (pp 8-9):
DECYP employs curriculum experts across humanities, arts/English and STEM. We recommend that TASC consider establishing an agreement with DECYP to involve their curriculum experts in the exam development process. By integrating these experts into the review process, TASC can align the exams with educational standards and accurately reflect curriculum objectives.
Yeah, that’ll work a treat.
Unfortunately, it seems that DECYP has only a “limited number” of “curriculum experts” (the number quite possibly being zero). This leads TASC-Deloitte to a second glimmer of sanity:
4) TASC-Deloitte gives a shout-out to the Monash mathematicians
Noting that DECYP will be of little use (and much less than they imagine)(p 5),
Based on this finding, we recommend that TASC explores a partnership with a tertiary education body, such as the University of Tasmania, to enhance expertise in the exam development program. The VCAA has a similar initiative with Monash University, involving senior mathematicians in reviewing and providing quality assurance processes for VCE maths exams.
Well, the Monash guys are providing much less “quality assurance” than “barely tolerated damage control” but, whatever. Such a UTAS partnership would obviously help much more than pretty much everything else recommended.
********************************************************************
Now, to the WitCH. As a reminder, TASC-Deloitte was arranged as a response to multiple “errors” on the 2024 exams. At the time, these errors were described as “minor” by TASC. Indeed some errors seemed pretty minor, being spelling errors and whatnot. In other cases, TASC was using “minor” in the Gniel manner of pathologically denying reality; there were significant screw-ups.
We have not been able to locate a clear list of the identified “errors”. ABC reported that altogether there were eleven errors across six exams:
Chemistry, English, Food and Nutrition, Physics, Accounting and Biology.
Readers will now understand exactly why we’re here: no mathematics? We didn’t believe it either, and so we looked.
Now, Dear Readers, it’s your turn to look. See what you find, either in the way of errors or just straight-out crap. (To be clear/fair, there is plenty of non-crap to be found.)
Links to the exams (and calculator-policy and curriculum) are below, and we’ll update the post with screenshots as readers point out items of note. Go for it.
2024 Mathematics Methods – Foundation
******************************************************
UPDATE (04/10/25)
Following up on Banacek’s comment, here is Q11 from MM Exam:
OK, straight away, Specialized Q2 – does not specify n has to be an integer.
There is a lot of clunky wording in the paper, but this stood out immediately.
As did the font changes. Probably not an error as such, but felt a bit… unprofessional?
Yes, Q2 is sloppy. Clear in context but no real excuse for that.
Tasm Methods: Oh boy… badder than Vic.
Wording shows incompetence.
Q4 ” transformations required ( in order) from … to transform into the form…”. Yak.
Q10 Show? Why give the answer?
Q11
“One example of a solution…” ???
a. Use Fig 3? It shows a line (normally looking) passing through the Origin. A MESS!
Q12 Evaluate? Not just ” Find”? Indef integral
Q14 then, “determine” when “evaluate” seems to be the word?
Will do.
Oh dear… yes, agree on the command terms for the integrals. YUK.
Thanks, Banacek. Yes, it was MM that inspired (rather, ensured) the post. Just landed from a long flight, but I’ll do some updating tomorrow.
14a) Calculate a definite integral. Then part b) is “Explain why the answer to a) is negative”. Umm? Because that’s what I got when I calculated the integral?
May be childish, but it’s about interpreting def int as signed area, so area above x-axis is less than the area below…
Not an issue, I think, though childish (but students are typically terrible in phrasing such conclusions – do we require worded answers at the end of the solutions? Not anymore. Used to be standard yesteryears.
Hi Banacek. I’m sure SRK was aware of that, and was attacking the placement of the question rather than its intrinsic merit.
Apart from the howlers, the Methods exam is extraordinarily clunky.
Thanks again, Banacek, and sorry to be slow. I agree with all that (and there are other MM questions that annoyed me). I’ve updated the post with Q11, which really is appalling.
The shock should not be that Tassie has so many mistakes – it’s a very small budget with few resources for such things. The shock is that Victoria with so many resources go it so badly wrong. Of course Tassie got Deloitte – that’s standard practice.
Thanks, JJ. Of course it’s no shock, and I have some sympathy for TASC. And of course it is standard to get leeches like Deloitte to do this kind of thing. But it is all insane. They are missing the point, just like VCAA now is missing the point.
That is the point in my experience Marty. When something goes publicly wrong and they need to demonstrate that ‘something’ has been done, bureaucracies focus on process, not content or culture, because those things are hard to change and even more to the point, especially in the modern era, it’s almost impossible to demonstrate any action on those publicly (even if you care about them).
Yehudi Blacher is a generic executive bureaucrat – there is no education content there as far as I know – this was fairly inevitable. What was more astonishing was that they engaged Bennett who really knew what he was doing, but perhaps they didn’t have much of a choice after Deloitte. And perhaps the failure of Deloitte meant they couldn’t do the reflexive modern practice of engaging PwC or KPMG or EY and they settled for an executive ex-bureaucrat instead.
Best hope now is to keep the pressure up and hope the new CEO and the Minister’s utter dedication to not being embarrassed again (especially in the lead up to the election) will bring about some real change. Anything positive coming out of the modern bureaucracy is a minor miracle – the public service is just such a shadow of what it was.
Nah. The goal of not being embarrassed is not gonna do a damn thing.
It will concentrate VCAA’s minds like nothing on earth (especially the execs). That does not mean good long-term or even short-term outcomes. But it does mean an absolute focus on not being seen to make a mistake. This is a Minister who would like to be Premier. The wrath of a Minister is something to be avoided at all costs.
Nonsense.
Have to agree to disagree. Education is about kids. The public service is about Ministers.
No, you missed my point. I agree that VCAA will be determined to not make “mistakes”, but this has to be understood in terms of the reality that the entire of VCE is a disaster.
Blacher is about not making “mistakes” but this is not remotely enough. What is required is reform, which is the point of Bennett and which is stone dead.
I’m afraid that the initial Bennett review implementation has to be seen as an oddity. Public services across the Western world have been declining for decades. Victoria’s public service was already weaker than NSW (can’t prove it – only based on what I’ve seen) before Daniel Andrews very significantly degraded the Victorian service. His brilliant political skills, especially in communications, covered a lot of it, but the reality is of a seriously dysfunctional organisation that is most departments. Education was never well regarded in any case.
Carroll appears to actually care (very hard to know but he has taken decisions like pushing through phonics), but ultimately the iron law of politics will have to drive his behaviour. He won’t be Minister for very long in any case – they don’t stay long these days to be able to see anything through (Nick Gibb in the UK is a very rare case indeed).
Still it is better than it was – there is clear recognition that VCAA is a disaster. There is a new CEO. There still is the possibility of change. So the work you and others have done has been worthwhile. As Philip Adams was wont to say “The situation is hopeless, we must take the next step”.
Good Adams quote.
Of course things are better, and I’m pleased (and am proud to take some of the credit). But “better” does not mean “remotely good”, and Carroll, although in principle a good guy, is gonna be really really gun shy.
VCAA is still a disaster, and the “make no mistakes” attitude of everyone is only going to further cement the disaster.