VCAA’s Responses to Poorly Conceived 2022 Exam Questions

This continues on from the previous post, where we whacked VCAA’s internal response to Burkard’s and my critique of 2022 exam questions with minor mathematical issues (part 3). This post considers questions Burkard and I identified as being poorly conceived (part 4).

Such questions are not so easily dealt with. As Burkard and I wrote,

Here, we provide examples of poorly conceived and/or poorly constructed and/or shallow questions from the 2022 exams; there are many more. The issues with these questions may not amount to errors, but in some ways these questions are even worse: they fail to test proper mathematical thought and computation in a clear and coherent manner. We shall not provide many details of the questions here: it is up to any reviewer to undertake a proper analysis of these and all questions, and consider our suggested, and all, shortcomings.

Which is pretty much it. It is really up to any reviewer to consider these questions, and the exams in their entirety, in depth. In fairness, it also made it difficult for VCAA to respond in brief. Still, Burkard and I flagged specific questions and VCAA responded (if only for themselves). So, we’ll consider the response. As with the previous post, I’ll provide only the core of VCAA’s responses, leaving out the by-the-numbers denials.

********************************************************

Specialist Mathematics Exam 1 (exam here and report here (Word, idiots))

Q4

A simple partial fractions question, worth four marks, which can easily be separated by inspection.

“Simple” as a categorization is too subjective and cannot apply meaningfully to the whole cohorts [sic] interpretation. The algebraic seperation [sic] technique the authors wish to use is not one demonstrated frequently by students.

The question provided for examination of multiple dot points from KK and KS across multiple AoS. There have frequently been questions of the same calibre on past papers.

Won’t someone please think of the stupid children?

No, the question was simple, even without the shortcut. If students didn’t do better than the average of 2.5/4 perhaps that was in part due to the question being extended (but simple) algebra and part due to VCAA doing its traditional nitpicking of taking off a mark for the absence of the +c and/or the absolute values in the logarithm term. The fact that questions of the same calibre have appeared frequently simply indicates the issue of under-challenging is systemic.

Q6(b)

An awkwardly constructed and needlessly complicated vector proof question, with too much scaffolding.

The question is a standard proof from antiquity, with guidance scaffolded into the question seed to prevent ambiguity for students.

The theorem is from antiquity but not the proof (which uses VCAA’s famous “vector scalar (dot) product”). Once again, VCAA must learn to distinguish scaffolding from mud. The presentation is clumsy, acting as anti-scaffolding, and one can “prevent ambiguity” without holding the students’ hands for every mini-step.

Specialist Mathematics Exam 2 (exam here and report here (Word, idiots))

MCQ8

An ill-considered and inaccurately worded “direction field” (slope field) question, requiring students to look too carefully to determine the correct answer.

The question is worded in a standard fashion with similar questions asked annually.

So?

It is no big deal, but the more accepted term for a field of tiny little gradients is “slope field”, with “direction field” more properly referring to a field of tiny little vectors. The concept, in any year, could also be better tested without requiring the use of a magnifying glass.

MCQ9

A badly executed inverse Euler’s method question. A question of this form could be good, but would need to be much cleaner, with simpler numbers, enabling a focus on the algebraic.

Outcome 3 from the SD requires discriminatory questions be asked around judicious CAS use. The numbers in the question were tested on and handled well on all CAS platforms.

VCAA loves them their CAS. The point is that this question could have been very good if it had been de-CASsified. As is, the question is pretty pointless.

MCQ10

A needlessly fussy and involved implicit differentiation question: poorly worded and poorly constructed, requiring too careful a consideration of signs.

This observation is not shared by the writing panel, ESV, SSV, or others in the Mathematics community.

VCAA’s “Mathematics community” is wrong.

MCQ20

In principle a very good question, but much too long a prompt and much too involved for a multiple choice question.

This observation is not shared by the writing panel, ESV, SSV, or others in the Mathematics community.

I take it back: VCAA’s “Mathematics community” is nuts. The prompt is long. With a diagram. And a table. For 1 mark.

QB3

A very poor differential equations question. Mostly routine, if somewhat meaningless, but the wording for (d) and (e) is so heavy, clumsy and opaque as to be seriously misleading. There is no reason for the first particle to pass through O, rather than beginning at O, and there is no reason for the second particle to begin two seconds later. In principle, (e) is a good question, but the interesting algebra is lost in the mess.

The story context for exam 2 questions in Specialist Mathematics and Mathematical Methods is always contrived to permit examination of KK and KS. This was well facilitated in this instance.

This is not true and it ignores all the flagged issues.

QB4

A contrived kinematics question, testing simple CAS skills rather than decent algebra.

Outcome 3 from the SD requires discriminatory questions be asked around judicious CAS use. The numbers in the question were tested on and handled well on all CAS platforms.

This misses the fundamental point: CAS poisons everything, and this question was thoroughly poisoned by CAS.

Mathematical Methods Exam 1 (exam here and report here (Word, idiots))

Q4

A poorly conceived, repetitive and very badly worded probability question. The mathematics being tested is simple, the question consisting of little more than an invitation for students to trick themselves into attempting to calculate probabilities for sampling without replacement.

The scenario is common in contemporary textbooks and classroom teaching.

The authors’ opinion is not shared by the writing panel, ESV, SSV, or others in the Mathematics community.

Although the question is not formally in error, it is absolutely atrocious: see the WitCH.

Q7

A long, aimless question, testing almost nothing of substance. It would have been almost impossible to have worded clearly, and it was not. The grading for this question should be very closely scrutinised.

Blah blah blah blah blah.

This question is similar in failings to Q4. One either recognises that the thing is a bloated, silly and pointless mess or one does not. The assessment guide (obtained under FOI) indicates that, as was predicted, the grading was anal retentive and nasty, the entirely predictable result of testing nothing but trivia. Appalling stuff.

Mathematical Methods Exam 2 (exam here and report here (Word, idiots))

MCQ6

Too busy a question, made needlessly busier by the function notation. Students have to hunt too carefully for the correct answer.

The question was fit for function. This is an examination and students will need to apply techniques to find answers.

Oh, I see. That explains it then.

MCQ14

A pointless test of pushing buttons.

The solution could be determined only by a well prepared student who understood the correct mathematical sequencing of ideas.

The “correct mathematical sequencing of ideas” for finding the expected value of a given continuous random variable? Yeah, sure. Very involved stuff.

MCQ16

In principle a reasonable question, but made pointless by CAS.

The subject assumes student familiarity with CAS, and examines the application of technology (Outcome 3). This question was fit for function.

There is nothing much to say except to repeat that CAS poisons everything. Hidden inside the technocrap there is a good question, but VCAA simply does not care to ask good questions.

MCQ18

In principle a reasonable question, but made ugly by CAS. Such a question would be significantly improved by using simpler numbers, or being framed algebraically.

The subject assumes student familiarity blah blah blah.

Same response to the same response. This stuff is so, so awful, and VCAA is so, so unable to see it.

QB2

An aimless, pseudo-modelling question, investigating and testing nothing of worth or interest. Part (e) is simply absurd.

The question was deemed fit for function in examining student proficiency in KK and KS across OC1, OC2 and OC3 by the writing panel, CM, ESV and SSV.

Sigh. The reader will just have to look. But, in semi-brief, here is what I wrote when summarising the question for the secret exam business:

A really, really stupid and depressing question. There’s a famous and lovely model of rabbits and foxes, and it ain’t trig. The questions themselves are trivial and pointless and as boring as dirt, and/or it’s just CAS/Magritte idiocy. To have the analysis of Acos(t) + Bsin(t) replaced with mindless button pushing demonstrates as well as anything how low Victorian senior mathematics has descended. Simply criminal. The fox transformation in (d) and (e) is a pointless add-on, absurdly expressed with matrices, and the notation is probably VCAA-illegal, and definitely should be VCAA-illegal.

QB3

A long and contrived pseudo-modelling question, to be answered almost entirely with CAS. The story is pointless and distracting. Although the question is very long and worth 14 marks, nothing is tested to any significant depth.

The context for the question is a novel application opening scope for exmining [sic] student achievement across OC1, OC2 and OC3, across AoS1, AoS2 and AoS3.

No, no, no, no and no. The question, which I have elsewhere hammered to its bloody death, is appalling.