An Offer: Checking For SACking Offences

Pauline Baynes

This is an open offer to review Methods and Specialist SACs. Here are the conditions:

0) The review is free. (You can consider donating to Tenderfeet.)

1) You may email me any Methods or Specialist SAC, by anyone.

2) You should indicate whether or not you are the writer of the SAC.

3) If you are the writer of the SAC, I will be diplomatic.*

4) It’s on your head, in particular for future SACs, if you’re breaking confidentiality rules or conventions. This is not my concern.

5) I will keep all SACs confidential, except to the extent there is explicit agreement otherwise. (See 12-14, below.)

6) Future SACs should, at minimum, be close to a final draft.

7) All SACs should include solutions and a grading scheme.

8) I may decline to review a SAC for being too old, or for other reasons.

9) I will review only for mathematical sense and mathematical correctness.

10) In particular, I will not check for, and do not give a stuff about, VCAA compliance.

11) I will not check all arithmetic and a review should not be taken as a guarantee that the SAC is error-free.

12) Each time I review a SAC I will record so below, with brief and, modulo points 13 and 14, anonymity-preserving comments.

13) I will identify commercial SACs as such, possibly indicating the commercial entity.

14) If you are the author of the SAC and you agree, I will consider making a separate post, to review the SAC in detail and to allow for comment.

 

I will be interested to see who is brave enough to enter (and who is tossed into) the lion’s den.

 

*) Yes, I am capable of diplomacy. I just prefer to do without.

UPDATE (26/7)

We have our first taker: a brave soul has entered the den. I’ll look at the proffered SAC asap. I was also asked what I am after, in making this offer, which is a fair question. The answer is two-fold:

a) (Jekyll) I’m making a genuine offer to provide a critique of a SAC from a mathematical perspective, for any writer who wants it. I’m hoping that by providing such a critique, the writer will become more attuned to any mathematical shortcomings in their (and all) SACs, and in VCE generally. Hopefully then, to the limited extent that VCAA’s idiot curriculum permits it, this will help the writer produce more mathematically coherent and rich SACs in the future.

b) (Hyde) I’m looking to see as much as I can of the nonsense the SAC system is producing. This will allow me to confirm for any teacher or student who has been served swill that they have indeed been served swill. It will also allow me to write upon such SACs, even if in very oblique terms.

UPDATE (27/7)

OK, this post is being steered away from what I intended, but I’m happy to let others steer.

First, a clarification. By “SAC”, I mean any school-based Year 12 assessment that counts towards the final VCE grade. I don’t care if the assessment takes five minutes or five days.

Now, the question is what to do with SACs offered to me by authors? I have two currently. I can either

a) Make the SACs into posts on this blog. The SACs would then be a basis for discussion, and a model for future SACs, but the SACs themselves would presumably not be usable. (Again, I don’t give a stuff about protocol, but obviously teachers must.)

or

b) Keep the SACs off the site, except for brief comments below, and set up a Free SACs to Good Home post. Teachers can then contact me to obtain copies.

Readers can suggest to me what they prefer. They can also suggest how (b) might work in practice.

Letter from a Concerned Student

A few days ago we received an email from “Concerned Student”, someone we don’t know, requesting advice on how to approach VCE mathematics. We have thoughts on this and intend to reply, but the email also seemed generally relevant and of likely interest. The email also raises interesting questions for teachers, and for the writer of this blog. With Concerned Student’s permission, we’ve reproduced their email below. We’ll hold off commenting until others, who actually know what they’re talking about, have had a go. Here is CS’s email:

It seems clear from reading this blog that a significant proportion of the VCE Methods & Specialist curricula are in direct conflict with good mathematical education. As someone entering these subjects next year, what’s the recommended approach to make it through all the content of the study design while also *learning maths*? Should I largely ignore the (… Cambridge) textbook and overall course and focus on self-teaching content along the same lines from better sources, stopping only to learn specifically from the curriculum whatever button mashing is necessary for an exam; or should I instead focus on fighting through the curriculum, and learn some proper maths on the side – I guess the productive question there is “is it easy enough to apply properly learnt maths to the arcane rituals found in VCE course assessments?”

It’s probably worth noting that, as far as I’m aware, the Methods & Specialist teachers at my school are known for being quite good, but they’re obviously still bound by the curriculum they teach.

WitCH 38: A Deep Hole

This one is due to commenter P.N., who raised it on another post, and the glaring issue has been discussed there. Still, for the record it should be WitCHed, and we’ve also decided to expand the WitCHiness slightly (and could have expanded it further).

The following questions appeared on 2019 Specialist Mathematics NHT, Exam 2 (CAS). The questions are followed by sample Mathematica solutions (screenshot corrected, to include final comment) provided by VCAA (presumably in the main for VCE students doing the Mathematica version of Methods). The examination report provides answers, identical to those in the Mathematica solutions, but indicates nothing further.

UPDATE (05/07/20)

The obvious problem here, of course, is that the answer for Part (b), in both the examination report and VCAA’s Mathematica solutions, is flat out wrong: the function fk will also fail to have a stationary point if k = -2 or k = 0. Nearly as bad, and plenty bad, the method in VCAA’s Mathematica solutions to Part (c) is fundamentally incomplete: for a (twice-differentiable) function f to have an inflection point at some a, it is necessary but not sufficient to have f’’(a) = 0.

That’s all pretty awful, but we believe there is worse here. The question is, how did the VCAA get it wrong? Errors can always occur, but why specifically did the error in Part (b) occur, and why, for a year and counting, wasn’t it caught? Why was a half-method suggested for Part (c), and why was this half-method presumably considered reasonable strategy for the exam? Partly, the explanation can go down to this being a question from NHT, about which, as far as we can tell, no one really gives a stuff. This VCAA screw-up, however, points to a deeper, systemic and much more important issue.

The first thing to note is that Mathematica got it wrong: the Solve function did not return the solution to the equation fk‘ = 0. What does that imply for using Mathematica and other CAS software? It implies the user should be aware that the machine is not necessarily doing what the user might reasonably think it is doing. Which is a very, very stupid property of a black box: if Solve doesn’t mean “solve”, then what the hell does it mean? Now, as it happens, Mathematica’s/VCAA’s screw-up could have been avoided by using the function Reduce instead of Solve.* That would have saved VCAA’s solutions from being wrong, but not from being garbage.

Ask yourself, what is missing from VCAA’s solutions? Yes, yes, correct answers, but what else? This is it: there are no functions. There are no equations. There is nothing, nothing at all but an unreliable black box. Here we have a question about the derivatives of a function, but nowhere are those derivatives computed, displayed or contemplated in even the smallest sense.

For the NHT problem above, the massive elephant not in the room is an expression for the derivative function:

    \[\color{red} \boldsymbol{f'_k(x) = -\frac{x^2 + 2(k+1)x +1}{(x^2-1)^2}}\]

What do you see? Yep, if your algebraic sense hasn’t been totally destroyed by CAS, you see immediately that the values k = 0 and k = -2 are special, and that special behaviour is likely to occur. You’re aware of the function, alert to its properties, and you’re led back to the simplification of fk for these special values. Then, either way or both, you are much, much less likely to screw up in the way the VCAA did.

And that always happens. A mathematician always gets a sense of solutions not just from the solution values, but also from the structure of the equations being solved. And all of this is invisible, is impossible, all of it is obliterated by VCAA’s nuclear weapon approach.

And that is insane. To expect, to effectively demand that students “solve” equations without ever seeing those equations, without an iota of concern for what the equations look like, what the equations might tell us, is mathematical and pedagogical insanity.

 

*) Thanks to our ex-student and friend and colleague Sai for explaining some of Mathematica’s subtleties. Readers will be learning more about Sai in the very near future.

WitCH 37: A Foolproof Argument

We’re amazed we didn’t know about this one, which was brought to our attention by commenter P.N.. It comes from the 2013 Specialist Mathematics Exam 2: The sole comment on this question in the Examination Report is:

“All students were awarded [the] mark for this question.”

Yep, the question is plain stuffed. We think, however, there is more here than the simple wrongness, which is why we’ve made it a WitCH rather than a PoSWW. Happy hunting.

UPDATE (11/05) Steve C’s comment below has inspired an addition:

Update (20/05/20)

The third greatest issue with the exam question is that it is wrong: none of the available answers is correct. The second greatest issue is that the wrongness is obvious: if z^3 lies in a sector then the natural guess is that z will lie in one of three equally spaced sectors of a third the width, so God knows why the alarm bells weren’t ringing. The greatest issue is that VCAA didn’t have the guts or the basic integrity to fess up: not a single word of responsibility or remorse. Assholes.

Those are the elephants stomping through the room but, as commenters as have noted, there is plenty more awfulness in this question:

  • “Letting” z = a + bi is sloppy, confusing and pointless;
  • The term “quadrant” is undefined;
  • The use of “principal” is unnecessary;
  • “argument” is better thought as the measure of an angle not the angle itself;
  • Given z is a single complex number, “the complete set of values for Arg(z)” will consist of a single number.
  • The grammar isn’t.

WitCH 36: Sub Standard

This WitCH is a companion to our previous, MitPY post, and is a little different from most of our WitCHes. Typically in a WitCH the sin is unarguable, and it is only the egregiousness of the sin that is up for debate. In this case, however, there is room for disagreement, along with some blatant sinning. It comes, predictably, from Cambridge’s Specialist Mathematics 3 & 4 (2020).

MAV’s Dangerous Inflection

This post concerns a question on the 2019 VCE Specialist Mathematics Exam 2 and, in particular, the solution and commentary for that question available through the Mathematical Association of Victoria. As we document below, a significant part of what MAV has written on this question is confused, self-contradictory and tendentious. Thus, noting the semi-official status of MAV solutions, that these solutions play a significant role in MAV’s Meet the Assessors events, and are quite possibly written by VCE assessors, there are some troubling implications. Question 3, Section B on Exam 2 is a differential equations problem, with two independent parts. Part (a) is a routine (and pretty nice) question on exponential growth and decay.* Part (b), which is our concern, considers the differential equation

    \[\boldsymbol{\color{blue}\frac{{\rm d}Q}{{\rm d}t\ } = e^{t-Q}}\,,\]

for t ≥ 0, along with the initial condition

    \[\boldsymbol{\color{blue}Q(0) =1}\,.\]

The differential equation is separable, and parts (i) and (ii) of the question, worth a total of 3 marks, asks to set up the separation and use this to show the solution of the initial value problem is

    \[\boldsymbol{\color{blue}Q =\log_e\hspace{-1pt} \left(e^t + e -1\right)}\,.\]

Part (iii), worth 2 marks, then asks to show that “the graph of Q as a function of t” has no inflection points.** Question 3(b) is contrived and bitsy and hand-holding, but not incoherent or wrong. So, pretty good by VCE standards. Unfortunately, the MAV solution and commentary to this problem is deeply problematic. The first MAV misstep, in (i), is to invert the derivative, giving

    \[\boldsymbol{\color{red}\frac{{\rm d}t\ }{{\rm d}Q } = e^{Q-t}}\,,\]

prior to separating variables. This is a very weird extra step to include since, not only is the step not required here, it is never required or helpful in solving separable equations. Its appearance here suggests a weak understanding of this standard technique. Worse is to come in (iii). Before considering MAV’s solution, however, it is perhaps worth indicating an approach to (iii) that may be unfamiliar to many teachers and students and, possibly, the assessors. If we are interested in the inflection points of Q,*** then we are interested in the second derivative of Q. The thing to note is we can naturally obtain an expression for Q” directly from the differential equation: we differentiate the equation using the chain rule, giving

    \[\boldsymbol{\color{magenta}Q'' = e^{t-Q}\left(1 - Q'\right)}\,.\]

Now, the exponential is never zero, and so if we can show Q’ < 1 then we’d have Q” > 0, ruling out inflection points. Such conclusions can sometimes be read off easily from the differential equation, but it does not seem to be the case here. However, an easy differentiation of the expression for Q derived in part (ii) gives

    \[\boldsymbol{\color{magenta}Q' =\frac{e^t}{e^t + e -1}}\,.\]

The numerator is clearly smaller than the denominator, proving that Q’ < 1, and we’re done. For a similar but distinct proof, one can use the differential equation to replace the Q’ in the expression for Q”, giving

    \[\boldsymbol{\color{magenta}Q'' = e^{t-Q}\left(1 - e^{t-Q}\right)}\,.\]

Again we want to show the second factor is positive, which amounts to showing Q > t. But that is easy to see from the expression for Q above (because the stuff in the log is greater than \boldsymbol{e^t}), and again we can conclude that Q has no inflection points. One might reasonably consider the details in the above proofs to be overly subtle for many or most VCE students. Nonetheless the approaches are natural, are typically more efficient (and are CAS-free), and any comprehensive solutions to the problem should at least mention the possibility. The MAV solutions make no mention of any such approach, simply making a CAS-driven beeline for Q” as an explicit function of t. Here are the contents of the MAV solution:

Part 1: A restatement of the equation for Q from part (ii), which is then followed by 

.˙.  \boldsymbol{ \color{red}\  \frac{{\rm d}^2Q }{{\rm d}t^2\ } = \frac{e^{t+1} -e^t}{\left(e^t + e -1\right)^2} } 

Part 2: A screenshot of the CAS input-output used to obtain the conclusion of Part 1.

Part 3: The statement   

Solving  .˙.  \boldsymbol{\color{red} \  \frac{{\rm d}^2Q }{{\rm d}t^2\ } = 0} gives no solution  

Part 4: A screenshot of the CAS input-output used to obtain the conclusion of Part 3.

Part 5: The half-sentence

We can see that \boldsymbol{\color{red}\frac{{\rm d}^2Q }{{\rm d}t^2\ } > 0} for all t,

Part 6: A labelled screenshot of a CAS-produced graph of Q”.

Part 7: The second half of the sentence,

so Q(t) has no points of inflection

This is a mess. The ordering of the information is poor and unexplained, making the unpunctuated sentences and part-sentences extremely difficult to read. Part 3 is so clumsy it’s funny. Much more important, the MAV “solution” makes little or no mathematical sense and is utterly useless as a guide to what the VCE might consider acceptable on an exam. True, the MAV solution is followed by a commentary specifically on the acceptability question. As we shall see, however, this commentary makes things worse. But before considering that commentary, let’s itemise the obvious questions raised by the MAV solution:

  • Is using CAS to calculate a second derivative on a “show that” exam question acceptable for VCE purposes?
  • Can a stated use of CAS to “show” there are no solutions to Q” = 0 suffice for VCE purposes? If not, what is the purpose of Parts 3 and 4 of the MAV solutions?
  • Does copying a CAS-produced graph of Q” suffice to “show” that Q” > 0 for VCE purposes?
  • If the answers to the above three questions differ, why do they differ?

Yes, of course these questions are primarily for the VCAA, but first things first. The MAV solution is followed by what is intended to be a clarifying comment:

Note that any reference to CAS producing ‘no solution’ to the second derivative equalling zero would NOT qualify for a mark in this ‘show that’ question. This is not sufficient. A sketch would also be required as would stating \boldsymbol{\color{red}e^t (e - 1) \neq 0} for all t.

These definitive-sounding statements are confusing and interesting, not least for their simple existence. Do these statements purport to be bankable pronouncements of VCAA assessors? If not, what is their status? In any case, given that pretty much every exam question demands that students and teachers read inscrutable VCAA tea leaves, why is it solely the solution to question 3(b) that is followed by such statements? The MAV commentary at least makes clear their answer to our second question above: quoting CAS is not sufficient to “show” that Q” = 0 has no solutions.  Unfortunately, the commentary raises more questions than it answers:

  • Parts 3 and 4 are “not sufficient”, but are they worth anything? If so, what are they worth and, in particular, what is the import of the word “also”? If not, then why not simply declare the parts irrelevant, in which case why include those parts in the solutions at all?
  • If, as claimed, it is “required” to state \boldsymbol{e^t(e-1)\neq 0} (which is indeed the key point of this approach and should be required), then why does the MAV solution not contain any such statement, nor even the factorisation that would naturally precede this statement?
  • Why is a solution “required” to include a sketch of Q”? If, in particular, a statement such as \boldsymbol{e^t(e-1)\neq 0} is “required”, or in any case is included, why would the latter not in and of itself suffice?

We wouldn’t begin to suggest answers to these questions, or our four earlier questions, and they are also not the main point here. The main point is that under no circumstances should such shoddy material be the basis of VCAA assessor presentations. If the material was also written by VCAA assessors, all the worse. Of course the underlying problem is not the quality or accuracy of solutions but, rather, the fundamental idiocy of incorporating CAS into proof questions. And for that the central villain is not the MAV but the VCAA, which has permitted their glorification of technology to completely destroy the appreciation of and the teaching of proof and reason. The MAV is not primarily responsible for this nonsense. The MAV is, however, responsible for publishing it, promoting it and profiting from it, none of which should be considered acceptable. The MAV needs to put serious thought into its unhealthily close relationship with the VCAA.  

*) We might ask, however, who refers to “The growth and decay” of an exponential function?

**) One might simply have referred to Q, but VCAA loves them their words.

***) Or, if preferred, the points of inflection of the graph of Q as a function of t.

Update (26/06/20)

The Examination Report is out and is basically ok; none of the nonsense and non sequiturs of the MAV solutions are included. The solution to (b)(iii) correctly focuses upon the factoring of Q”, although it needlessly worries about the sign of the denominator. There is no mention of the more natural approach to obtaining and analysing Q” but, given the question is treated by the VCAA and pretty much everyone as just another mindless exercise in pushing buttons, this is no surprise.

WitCH 35: Overly Resolute

This WitCH (arguably a PoSWW) comes courtesy of Damien, an occasional commenter and an ex-student of ours from the nineteenth century. It is from the 2019 Specialist Mathematics Exam 2. We’ll confess, we completely overlooked the issue when going through the MAV solutions.

Update (16/02/20)

What a mess. Thanks to Damo for pointing out the problem, and thanks to the commenters for figuring out the nonsense.

In general form, the (intended) scenario of the exam question is

The vector resolute of \boldsymbol{\tilde{a}} in the direction of \boldsymbol{\tilde{b}} is \boldsymbol{\tilde{c}},

which can be pictured as follows: For the exam question, we have \boldsymbol{\tilde{a}} = \boldsymbol{\tilde{i}} + \boldsymbol{\tilde{j}} - \boldsymbol{\tilde{k}}, \boldsymbol{\tilde{b}} = m\boldsymbol{\tilde{i}} + n\boldsymbol{\tilde{j}} + p\boldsymbol{\tilde{k}} and \boldsymbol{\tilde{c}} = 2\boldsymbol{\tilde{i}} - 3\boldsymbol{\tilde{j}} + \boldsymbol{\tilde{k}}.

Of course, given \boldsymbol{\tilde{a}} and \boldsymbol{\tilde{b}} it is standard to find \boldsymbol{\tilde{c}}. After a bit of trig and unit vectors, we have (in must useful form)

\boldsymbol{\tilde{c} = \left(\dfrac{\tilde{a}\cdot \tilde{b}}{\tilde{b}\cdot \tilde{b}}\right)\tilde{b}}

The exam question, however, is different: the question is, given \boldsymbol{\tilde{a}} and \boldsymbol{\tilde{c}}, how to find \boldsymbol{\tilde{b}}.

The problem with that is, unless the vectors \boldsymbol{\tilde{a}} and \boldsymbol{\tilde{c}} are appropriately related, the scenario simply cannot occur, meaning \boldsymbol{\tilde{b}} cannot exist. Most obviously, the length of \boldsymbol{\tilde{c}} must be no greater than the length of \boldsymbol{\tilde{a}}. This requirement is clear from the triangle pictured, and can also be proved algebraically (with the dot product formula or the Cauchy-Schwarz inequality).

This implies, of course, that the exam question is ridiculous: for the vectors in the exam we have |\boldsymbol{\tilde{c}}| > |\boldsymbol{\tilde{a}}|, and that’s the end of that. In fact, the situation is more delicate; given the pictured vectors form a right-angled triangle, we require that \boldsymbol{\tilde{a}} - \boldsymbol{\tilde{c}} be perpendicular to \boldsymbol{\tilde{c}}. Which implies, once again, that the exam question is ridiculous.

Next, suppose we lucked out and began with \boldsymbol{\tilde{a}}- \boldsymbol{\tilde{c}} perpendicular to \boldsymbol{\tilde{c}}. (Of course it is very easy to check whether we’ve lucked out.) How, then, do we find \boldsymbol{\tilde{b}}? The answer is, as is made clear by the picture, “Well, duh”. The possible vectors \boldsymbol{\tilde{b}} are simply the (non-zero) scalar multiples of \boldsymbol{\tilde{c}}, and we’re done. Which shows that the mess in the intended solution, Answer A, is ridiculous.

There is a final question, however: the exam question is clearly ridiculous, but is the question also stuffed? The equations in answer A come from the equation for \boldsymbol{\tilde{c}} above and working backwards. And, these equations correctly return no solutions. Moreover, if the relationship between \boldsymbol{\tilde{a}} and \boldsymbol{\tilde{c}} had been such that there were solutions, then the A equations would have found them. So, completely ridiculous but still ok?

Nope.

The question is framed from start to end around definite, existing objects: we have THE vector resolute, resulting in THE values of m, n and p. If the VCAA had worded the question to find possible values, on the basis of a possible direction for the resolution, then, at least technically, the question would be consistent, with A a valid answer. Still an utterly ridiculous question, but consistent. But the VCAA didn’t do that and so the question isn’t that. The question is stuffed.

Further Update (26/06/20)

As commenters have noted, the Examination Report has finally appeared. And, as predicted, answer A was deemed correct, with the Report noting

Option A gives the set of equations that can be used to obtain the values of m, n and p. Explicit solution would result in a null set as it is not possible for a result of a vector to be of greater magnitude than the vector itself.

Well, it’s something. Presumably “result of a vector” was intended to be “resolute of a vector”, and the set framing is weirdly New Mathy. But, it’s something. Seriously. As John Friend notes, it is at least a small step along the way to indicating the question is not all hunky-dory.

That step, however, is way too small. We’ll close with two comments, reiterating the points made above.

1. The question is wrong

Read the question again, and read the first sentence of the Report’s comment. The question and report justification are fundamentally stuffed by the definite articles, by the language of existence. All answers should have been marked correct.

2. The question is worse than wrong

Even if the vectors \boldsymbol{\tilde{a}} and \boldsymbol{\tilde{c}} had been chosen appropriately, the question is utterly devoid of mathematical sense. It suggests a long and difficult method to solve a problem that, if indeed is solvable, is trivial. 

 

 

Implicit Suggestions

One of the unexpected and rewarding aspects of having started this blog is being contacted out of the blue by students. This included an extended correspondence with one particular VCE student, whom we have never met and of whom we know very little, other than that this year they undertook UMEP mathematics (Melbourne University extension). The student emailed again recently, about the final question on this year’s (calculator-free) Specialist Mathematics Exam 1 (not online). Though perhaps not (but also perhaps yes) a WitCH, the exam question (below), and the student’s comments (belower), seemed worth sharing.

Hi Marty,

Have a peek at Question 10 of Specialist 2019 Exam 1 when you get a chance. It was a 5 mark question, only roughly 2 of which actually assessed relevant Specialist knowledge – the rest was mechanical manipulation of ugly fractions and surds. Whilst I happened to get the right answer, I know of talented others who didn’t.

I saw a comment you made on the blog regarding timing sometime recently, and I couldn’t agree more. I made more stupid mistakes than I would’ve liked on the Specialist exam 2, being under pressure to race against the clock. It seems honestly pathetic to me that VCAA can only seem to differentiate students by time. (Especially when giving 2 1/2 hours for science subjects, with no reason why they can’t do the same for Maths.) It truly seems a pathetic way to assess or distinguish between proper mathematical talent and button-pushing speed writing.

I definitely appreciate the UMEP exams. We have 3 hrs and no CAS! That, coupled with the assignments that expect justification and insight, certainly makes me appreciate maths significantly more than from VCE. My only regret on that note was that I couldn’t do two UMEP subjects 🙂

UPDATE (22/4) The examination report has appeared.