This one comes courtesy of a smart VCE student, the issue having been flagged to them by a fellow student. It is a multiple choice question from the 2009 Mathematical Methods, Exam 2; the Examination Report indicates, without comment, that the correct answer is D.
These matrix transformation questions always felt a little forced in methods – and I guess the exam writers just applied the general idea without thinking about the symmetries of the function they were transforming. D is the answer you get without using any symmetries of sine. Leaving in another correct transformation [I’ll not spoil it for anyone else] is an understandable oversight if it was written for a quiz in a school.
Identifying why it is a WitCH would be a good exercise to give to students…
I wonder if anyone noticed this error before the ATARs were given out. And I wonder if anyone chose the other correct solution on purpose – if you could see the other correct solution you probably know enough to choose the solution the examiner meant.
Thanks, Simon. I agree, although “forced” is quite an understatement. And yes, when this kind of thing is produced by a harried teacher, you typically give them a pass. Still, I wonder if the screw-up here was just dumb (bad) luck, combined with incompetent checking.
As you suggest, it’s hard to imagine many students spotted only the unintended correct answer. But as for the error being spotted before the ATARs appeared, the answer is almost certainly “no”. It’s not like VCAA rushes out their reports. Indeed, the question is why, after 10+ years, the Report is still incorrect?
How does this keep happening?
Damo, you know exactly how.
Indeed. Let me rephrase – When feeling charitable, I understand that setters can make mistakes with questions and that vetters can let one through to the keeper. If these were genuinely isolated instances, I would be inclined to feel more sympathy towards them. When I’m feeling less charitable, I think that setters sometimes write questions that are fundamentally flawed (often because they are trying to be too cute) and that should be vetoed at any early stage. In this instance I have less sympathy for the setters, particularly the more senior members of the panel, and less again for the vetters. You would assume that at least one of the vetters is (and my understanding is that at least one vetter is) a mathematician. So I’m not sure how some of the questions make their way onto the final paper.
The thing is, for a question like the one above, I would be inclined to feel charitable. Yes, the person setting the question fucked up. And it should have been picked up by a number of different people throughout the process – from the other members on the panel, to the vetters, to any other DuLLard who might stick their finger into the pie. But I can see how it slipped through. So, I would be inclined to feel charitable IF the response to this was reasonable. That is, we fucked up here, we better be really careful that we don’t fuck up again. Tighten up the setting and vetting process. Work through all of the questions thoroughly, check and check again all of the options for MC questions, etc… Make sure that it doesn’t happen again. IF this was an isolated fuck up, yep I would feel charitable. If there was more accountability, if someone would take responsibility for and properly own the mistakes, rather than sweep them under the carpet, yep I would feel more charitable.
But it feels that every year, there is a monumental fuck up that ends up getting glossed over at best, ignored at worst. I know how these mistakes happen. And I’ve got some idea of why they keep happening. But I don’t know why they are allowed to keep happening, particularly with such regularity.
Thanks, Damo. What makes you believe that at least one of the vetters is a mathematician, and what do you mean by “mathematician”? (Feel free to email me if it is better communicated that way.
Even as a once-off, I don’t feel remotely charitable on this question; the error isn’t subtle. As to why these errors are permitted to keep happening, is it really so difficult to understand?
According to the VCAA website, the exam panel includes a so-called study specialist, who “confirms the accuracy of the theoretical and technical content of the examination” and “should have considerable expertise and appropriate qualifications in the subject”.
https://www.vcaa.vic.edu.au/About-us/Pages/Be-on-a-VCE-examination-development-panel.aspx
Sigh. Another stupid fucking fish in another stupid fucking barrel.
At least this time it says “a transformation” rather than “the transformation”, so whomever wrote the question understood that multiple correct answers are possible…
I think that almost makes it worse that two of the possible answers were then included.
Actually, yes!
I will make a confident prediction of what VCAA will say if asked about this:
The response will be that ‘the equation
is not the same as the equation 
, they are *equivalent*.’ And then there would be further pontification about equivalence not being the same as equality (cue VCAAspeak stream of conciousness).
People in and associated with VCAA who should know better (that is, people who are supposed to have mathematical competence) are in constant denial of the mathematical errors and blunders that continually arise in exams, reports and documentation. Try asking if
(yes, I know there should also be a modulus and a ‘+c’) is part of the *Maths Methods* Study Design … Try asking why the Bernoulli *Distribution* has been added to the Adjusted Maths Methods Formula Sheet
(https://www.vcaa.vic.edu.au/Documents/exams/2020adjusted/AdjustedMM1-formula-w.pdf) …
This takes us beyond simple errors that vettors should be detecting but aren’t, we’re talking about significant mathematical content NOT in the Maths Methods Study Design that nevertheless is apparently examinable. Where’s the oversight?
John, do you really think VCAA would pretend that B is not correct? Sure, they’re prone to crabbed, legalistic ass-covering, but a claim that the “equations” are different would be absurd.
Any such defence of the question would also fail: the question doesn’t ask for the “equation”, it asks for the “curve”. So, they’re screwed.
But now, that makes me wonder. In my answer above to Simon, I pondered whether including B was just due to oversight and bad luck. It seemed *very* bad luck. So, is it possible that B was deliberately included as a distractor? If so, that would be insane. And, they’re still screwed.
Well, there’s only one way to find out (maybe I’ll try for 5/5 before the year ends) … (Actually I think the response might be more along the lines of “that was 11 years ago, who cares, stop wasting our time with nitpicks, you’re just looking for stuff to complain about.”)
And yes, I really do think they would use this defence, or something similarly absurd. After all, look what they wrote in the 2019 Specialist Maths Examination Report for that vector resolute multiple choice question. Is what they said there any less absurd? … The Defence rests, Your Honour.
And I think you’re right – B *was* meant as a distractor. I can see exactly how it might have come about – “let’s make a small change to the correct answer. Aha! Let’s change the + to a minus. Good job!”.
What a pity symmetry tripped them up. A little knowledge is a dangerous thing – and the writers and vettors are highly dangerous. Again the question – What were the vettors being paid to do? Asleep at the wheel.
I wonder if there’s anyone here that has inside gossip from the Markers Meeting … (yes, I know MCQ is marked by computer, but it might still have come up in conversation)
Hi, John. I wasn’t clear. What I’m suggesting is that perhaps B was intentionally “correct” (i.e. the very same function), but was considered incorrect in their eyes (not the very same equation).
I can believe that. It’s exactly the defence I’d expect VCAA to give.
Except, as I noted, this black letter law-ish defence fails, since the question is asking for the identification of curves, not equations.
Yes, although they didn’t understand how to write in plain English.
I’ve just noticed something else – may be significant, may not be: this question was not on the MM non-CAS paper.
I don’t actually think it is relevant though, because with questions like this where (a) no working has to be shown and (b) there is never much variation in the way these are asked in VCAA exams, students tend to memorize the basic transformations and go from there.
Did the 10% who selected option B multiply out the matrix equation and simplify? I guess we will never know.
Thanks, RF. I noticed that as well, but wasn’t sure what to make of it. The 40% who didn’t answer the question “correctly” were pretty evenly spread, so I’d assume the majority of that 10% just guessed.
The more I think of the question not appearing on the other paper, the more I think it is coincidence and not much else. When looking through the two curriculum documents from back then, the Algebra topic was very different, but the Graphs and Relations (from which this question presumably originates) is not that different.
Is seems like the examiners report only comments on MCQs when less than half the students get it correct.
If I remember rightly (but I probably don’t), transformations using matrices was on the CAS course but not on the non-CAS course.
If the purpose of assessment is to determine where students are in their learning of the subject, what can we infer about the learning of those students who gave B as their answer?
More importantly, what can we infer about the learning of those examiners who gave B as an option?
It is alaways unfair to competent students as they will look for an error when having 2 correct answers wasting their time.
One has to be on guard with horizontal translations, can lead to cosine graph, can be positive /negative. High alert automatically on.
Here: ” matrix must have 1/2 and 3 , possibly one of them negative, but hey, horizontal transl can change it, on main diagonal and y comp of column vector 1.
This leaves B and D. Pi/2 left or right….both work
Oops , must have done something wrong, let’s try again….”
Is this fair for any good student?
How do you folk out there teach students in MM to solve such problems in a very short space of time?
A deep but efficient explanation and practise of the mathematical principles.
Largely by rote, Terry. Since these matrix transformations questions are nearly always only tested as Multiple Choice (and in some cases are bizarrely stupidly difficult or require knowledge outside the curriculum – 2016 Paper 2 MCQ20 is the example that immediately comes to mind…) I used to dedicate maybe 1 lesson on transformations by matrices and largely as a “here is the procedure, here are some examples” lesson. (so pretty much what Marty said!)
I used to spend a lot more time on it in Year 11 Specialist (and do more transformations, such as rotations, sheers, projections) where matrices were a “proper” area of study.
Thanks; I meant “How do you folk out there teach students in MM to solve, in a short space of time, such problems?” What I am driving at is how do you expect the average student to solve these problems in the twinkle of an eye?
OK, fair question – I write 20 examples and then close to the final exam give the class 30 minutes to do them all, since VCAA MM papers have a time to marks ratio of 1.5 to 1.
It is not perfect but it is the best I can do.
I should say “could” since I haven’t taught Year 12 methods in a while.
Two thoughts, one more cynical than the other.
1. While RF is correct about the average time per mark, not all questions take the same amount of time when solved in any reasonably efficient way. So often a student can build up a significant bank of time by answering many of the easier questions relatively quickly.
2. People with more experience with Mathematica might be able to say whether that software is able to answer questions like this very easily with pre-defined commands.
But you are right, that for many of these multiple choice questions (Specialist Maths is the same) there is a ridiculous amount of work involved just to get one mark.
SRK, if you look at Sai’s Mathematica article, you can see he’s created a function Transform which, combined with the in-built function FullSimplify, does the kind of thing you’re suggesting. I’d be surprised if Sai doesn’t also have further tricks in his bag.
In fairness to VCAA, the writers in 2009 didn’t have to consider Mathematica being used on their questions. (On the other hand, 10 years later and it seems that the writers still don’t consider Mathematica being used on their questions).
Yeah, and it shits me. The question students are asking now is not “How do I do this question”, it’s “How do I use Mathematica to do this question”.
VCAA has completely brain-washed students into thinking they should be using Mathematica rather than putting pen to paper and doing the maths. Did I say how much this really shits me. I can laugh at what a bunch of clueless fools Sai is showing VCAA to be, but unfortunately what his bag of tricks is also doing is reinforcing the brain-washing. Sorry, but the latter really shits me and is becoming a bigger and bigger part of the problem.
What I’d much prefer to see Sai and others doing is creating questions that are Mathematica-resistant until the student has done some actual mathematics and got to the point where it’s efficacious to recruit Mathematica to do the remaining heavy lifting. With great power there must come great responsibility (https://www.youtube.com/watch?v=beHkAr1YrIE).
VCAA could do worse than employ someone like Sai to vet their exam questions so that they require doing *mathematics* rather than using a ‘black-box bag of Mathematica tricks’.
I distinctly recall once asking how a question could be made, to which I was told that I wanted to start with the end in mind. I’ve never stopped thinking of trying to make questions since then. Some special features of functions that were conveniently “crafted” for the sake of question and so on. With Mathematica or CAS in general one needs to take care that the concepts you want to test for require much more careful use of Mathematica as an aide, perhaps for getting your bearings as opposed to throwing a question at Mathematica. Some food for thought, mainly for me, but I’m open to thinking about “Mathematica proof” questions.
As for me having a “bag full functions”, that would be true (and it’s growing). I’ve come to notice this is the result of questions that are very simple in nature, asking to solve for this or that, which one of the following…. all of which could be brute-forced or just thrown into Mathematica (god forbid allowing Wolfram Alpha in for any reason). If all you’re asking is “find the stationary point of this…” or the area of that etc, you’re really asking for a CAS to demolish your question (i.e VCE exams). It’s tangentially related but I undertook a pure maths unit (Real Analysis) where the CAS was effectively useless, and Mathematica could do practically nothing to answer these questions. Maybe moving away from “mathematical modelling” that just paints context onto the same dull question could help prevent this?
Now that JF mentions it, I think an interesting idea for a post if Marty is interested is how everyone would write a good question, whether it’s for SM or MM, what they would attempt to get students to do and how they would want them to proceed. I can add a lot of my own thoughts, both in terms of maths and technology and concepts that may force students to use Mathematica as an aide. In fact, I’ve thought of something more fun, that would be “removing” the trivialization from VCE exams in some way, while still sticking to the concepts within the question. One easy fix would be removing the question above since it’s pointlessly trivial…
I just had a discussion with a friend of mine about an MCQ on the exam I did for SM last year and the ridiculousness of the following question came up. I vividly recall each of my friends who had done the exam laugh at the question being trivialized via the Sum function when VCAA would have intended students to notice that past 4, the value of i^(n!) would not change, but alas a quick and brainless solution was more apparent. Make of it what you will, but some questions like this need more “CAS proofing”.
Just need to move it to non calc exam, even out time. That’s it.
If you moved all the questions on Exam 2 that Mathematica trivialised to Exam 1, you’d be left with a 2 hour Exam 1 and a 1 hour Exam 2. Food for thought … Except …
Some idiot decided that technology was more important than mathematics. Until that paradigm changes, your idea is dead in the water, BaSp.
However, the solution is actually simple: the Exam 2 writing panel is completely gormless when it comes to understanding Mathematica and its power (this is not the only area in which the panel lacks gorm, by the way). So sack the whole useless bunch of them and replace them with mathematically competent and Mathematica savvy people. Sai, batter up …
Unfortunately, this would upset too many VCAA toadies and spell an end to nepotism. Plus, I have a theory that this is a subtle way of getting more schools on board with Mathematica and hence VCAA 40.0379° N, 76.3055° W succeeding in pushing through its putrid, fetid mathematically bankrupt agenda …
So my solution (just like some solutions in commercial products and Examination Reports) is dead in the water.
If the IB was not so expensive, the VCAA monopoly on ‘education’ would have been broken long ago.
Thanks, Sai. A nice question, turned to farce by the thoughtless placement.
Thanks very much, Sai, and everyone, for commenting so far.
I’m more than happy to set up a dedicated post/guest-post on the writing of techno-proof MM/SM questions (or on anything – I’m always open to suggestions and offers of contribution). There are two aspects in which I’m not sure of what the focus of such a post/discussion might be, although I’m happy for people to discuss pretty much anything, short of lynchings. (They’re allowed to think about lynchings, just not discuss it.)
The first aspect is whether the intent is to ponder questions for a New, Improved MM/SM. That’s fine, if that would be of interest, but I’m not quite sure of the point. Sure, as JF notes, VCAA is crazy not to be employing Sai full time to pursue such matters. Discussing it here, however, feels to me a little like the People’s Front of Judea.
The second aspect is that, in the sense that this is working towards a New, Improved MM/SM, it would be ignoring a larger, and undermining issue. The undermining issue is the desperate need for the proper teaching, at minimum, of non-techno basics.
Hi there Marty and others,
A lot of discussion of course about MM & SM, rightfully so as they are the ‘algebra-based’ mathematics subjects offered at VCE (in a loose way of course).
Hardly/no discussion of Further Maths – whilst my knowledge on Further as both a student and teacher is non existent, wondering what your thoughts on this are?
I find Further to be a button bashing cacophony of useless and unmathematical shit, plagued by a distinct lack of *mathematics*. Am I being too harsh or is this a shared view?
The FM students that I speak to, as well as the FM teachers, swear by the subject (often saying things like “it’s useful for real life and it’s simpler maths without all the useless algebra and graphing” and I don’t know how to respond).
Also students do rave about it which is something I don’t find is the case in MM and SM.
I wonder what’s going on….
Hi, Steve, my go-to joke whenever Further Mathematics is brought up is to ask “Further than what?”
I don’t look to discuss Further on this blog because I don’t see Further as a mathematics subject. So, I don’t really care what nonsense goes on there. I don’t believe it is a useful subject, and I’m no more impressed in the subject’s positive reviews than I am in MasterChef’s. Once again, I think Neil Postman pretty much says it all.
But, perhaps it would be worth having some posts to discuss the subject, and I’m open to suggestion.
Steve: students rave about it precisely for the reasons you dislike it. Weak students are relieved that their weaknesses with arithmetic / algebra are hidden by the ubiquitous CAS. Strong students are relieved that they can do well with hardly any effort. I’m sick of it, and find it a thoroughly degrading experience, but the ratio of classes to maths teachers at my school almost forces me into teaching it year after year.
Why might a maths teacher like it? Same reason strong students like it – it’s easy, very little preparation and workload.
“The name we may properly give to an education without prerequisites, perplexity, and exposition is entertainment.”
Yes, but it’s also institutionalised cynicism. There’s the expectation (from students, teachers, parents, school leaders, etc.) that a student can be successful (measured by study score) without any significant cognitive achievement.
UPDATE: Surprised nobody saw the ammendment, but VCAA took action to add a comment to Q12, which states “B was also accepted as it leads to an equivalent expression”… Well. glad they took action 10 years after the actual exam. I’m curious as to whether at the marking stage, B was actually marked correct.
The amendment is a blatant lie.