The Treachery of Images

Harry scowled at a picture of a French girl in a bikini. Fred nudged Harry, man-to-man. “Like that, Harry?” he asked.

“Like what?”

“The girl there.”

“That’s not a girl. That’s a piece of paper.”

“Looks like a girl to me.” Fred Rosewater leered.

“Then you’re easily fooled,” said Harry. It’s done with ink on a piece of paper. That girl isn’t lying there on the counter. She’s thousands of miles away, doesn’t even know we’re alive. If this was a real girl, all I’d have to do for a living would be to stay at home and cut out pictures of big fish.”

                       Kurt Vonnegut, God Bless you, Mr. Rosewater

 

It is fundamental to be able to distinguish appearance from reality. That it is very easy to confuse the two is famously illustrated by Magritte’s The Treachery of Images (La Trahison des Images):

The danger of such confusion is all the greater in mathematics. Mathematical images, graphs and the like, have intuitive appeal, but these images are mere illustrations of deep and easily muddied ideas. The danger of focussing upon the image, with the ideas relegated to the shadows, is a fundamental reason why the current emphasis on calculators and graphical software is so misguided and so insidious.

Which brings us, once again, to Mathematical Methods. Question 5 on Section Two of the second 2015 Methods exam is concerned with the function V:[0,5]\rightarrow\Bbb R, where

\phantom{\quad}  V(t) = de^{\frac{t}3} + (10-d)e^{\frac{-2t}3}\,.

Here, d \in (0,10) is a constant, with d=2 initially; students are asked to find the minimum (which occurs at t = \log_e8), and to graph V. All this is par for the course: a reasonable calculus problem thoroughly trivialised by CAS calculators. Predictably, things get worse.

In part (c)(i) of the problem students are asked to find “the set of possible values of d” for which the minimum of V occurs at t=0. (Part (c)(ii) similarly, and thus boringly and pointlessly, asks for which d the minimum occurs at t=5). Arguably, the set of possible values of d is (0,10), which of course is not what was intended; the qualification “possible” is just annoying verbiage, in which the examiners excel.

So, on to considering what the students were expected to have done for (c)(ii), a 2-mark question, equating to three minutes. The Examiners’ Report pointedly remarks that “[a]dequate working must be shown for questions worth more than one mark.” What, then, constituted “adequate working” for 5(c)(i)? The Examiners’ solution consists of first setting V'(0)=0 and solving to give d=20/3, and then … well, nothing. Without further comment, the examiners magically conclude that the answer to (c)(i) is 20/3 \leqslant d< 10.

Only in the Carrollian world of Methods could the examiners’ doodles be regarded as a summary of or a signpost to any adequate solution. In truth, the examiners have offered no more than a mathematical invocation, barely relevant to the question at hand: why should V having a stationary point at t=0 for d=20/3 have any any bearing on V for other values of d? The reader is invited to attempt a proper and substantially complete solution, and to measure how long it takes. Best of luck completing it within three minutes, and feel free to indicate how you went in the comments.

It is evident that the vast majority of students couldn’t make heads or tails of the question, which says more for them than the examiners. Apparently about half the students solved V'(0)=0 and included d = 20/3 in some form in their answer, earning them one mark. Very few students got further; 4% of students received full marks on the question (and similarly on (c)(ii)).

What did the examiners actually hope for? It is pretty clear that what students were expected to do, and the most that students could conceivably do in the allotted time, was: solve V'(0)=0 (i.e. press SOLVE on the machine); then, look at the graphs (on the machine) for two or three values of d; then, simply presume that the graphs of V for all d are sufficiently predictable to “conclude” that 20/3 is the largest value of d for which the (unique) turning point of V lies in [0,5]. If it is not immediately obvious that any such approach is mathematical nonsense, the reader is invited to answer (c)(i) for the function W:[0,5]\rightarrow\Bbb R where W(t) = (6-d)t^2 + (d-2)t.

Once upon a time, Victorian Year 12 students were taught mathematics, were taught to prove things. Now, they’re taught to push buttons and to gaze admiringly at pictures of big fish.

The Median is the Message

Our first post concerns an error in the 2016 Mathematical Methods Exam 2 (year 12 in Victoria, Australia). It is not close to the silliest mathematics we’ve come across, and not even the silliest error to occur in a Methods exam. Indeed, most Methods exams are riddled with nonsense. For several reasons, however, whacking this particular error is a good way to begin: the error occurs in a recent and important exam; the error is pretty dumb; it took a special effort to make the error; and the subsequent handling of the error demonstrates the fundamental (lack of) character of the Victorian Curriculum and Assessment Authority.

The problem, first pointed out to us by teacher and friend John Kermond, is in Section B of the exam and concerns Question 3(h)(ii). This question relates to a probability distribution with “probability density function”

    \[  f(x) =   \left\{\aligned &\frac{(210-x)e^{\frac{x-210}{20}}}{400} \qquad && 0\leqslant x \leqslant 210,\\ &0 && \text{elsewhere.} \endaligned\right.}\]

Now, anyone with a good nose for calculus is going to be thinking “uh-oh”. It is a fundamental property of a PDF that the total integral (underlying area) should equal 1. But how are all those integrated powers of e going to cancel out? Well, they don’t. What has been defined is only approximately a PDF,  with a total area of 1 - 23/2e^{21/2} \approx 0.9997. (It is easy to calculate the area exactly using integration by parts.)

Below we’ll discuss the absurdity of handing students a non-PDF, but back to the exam question. 3(h)(ii) asks the students to find the median of the “probability distribution”, correct to two decimal places. Since the question makes no sense for a non-PDF, of course the VCAA have shot themself in the foot. However, we can still attempt to make some sense of the question, which is when we discover that the VCAA has also shot themself in the other foot.

The median m of a probability distribution is the half-way point. So, in the integration context here we want the m for which

a)      \phantom{\quad}  \int\limits_0^m f(x)\,{\rm d}x = \dfrac12.

As such, this question was intended to be just another CAS exercise, and so both trivial and pointless: push the button, write down the answer and on to the next question. The problem is, the median can also be determined by the equation

b)     \phantom{\quad}  \int\limits_m^{210} f(x)\,{\rm d}x = \dfrac12,

or by the equation

c)     \phantom{\quad} \int\limits_0^m f(x)\,{\rm d}x = \int\limits_m^{210} f(x)\,{\rm d}x.

And, since our function is only approximately a PDF, these three equations necessarily give three different answers: to the demanded two decimal places the answers are respectively 176.45, 176.43 and 176.44. Doh!

What to make of this? There are two obvious questions.

1. How did the VCAA end up with a PDF which isn’t a PDF?

It would be astonishing if all of the exam’s writers and checkers failed to notice the integral was not 1. It is even more astonishing if all the writers-checkers recognised and were comfortable with a non-PDF. Especially since the VCAA can be notoriously, absurdly fussy about the form and precision of answers (see below).

2. How was the error in 3(h)(ii) not detected?

It should have been routine for this mistake to have been detected and corrected with any decent vetting. Yes, we all make mistakes. Mistakes in very important exams, however, should not be so common, and the VCAA seems to make a habit of it.

OK, so the VCAA stuffed up. It happens. What happened next? That’s where the VCAA’s arrogance and cowardice shine bright for all to see. The one and only sentence in the Examiners’ Report that remotely addresses the error is:

“As [the] function f  is a close approximation of the [???] probability density function, answers to the nearest integer were accepted”. 

The wording is clumsy, and no concession has been made that the best (and uniquely correct) answer is “The question is stuffed up”, but it seems that solutions to all of a), b) and c) above were accepted. The problem, however, isn’t with the grading of the question.

It is perhaps too much to expect an insufferably arrogant VCAA to apologise, to express anything approximating regret for yet another error. But how could the VCAA fail to understand the necessity of a clear and explicit acknowledgement of the error? Apart from demonstrating total gutlessness, it is fundamentally unprofessional. How are students and teachers, especially new teachers, supposed to read the exam question and report? How are students and teachers supposed to approach such questions in the future? Are they still expected to employ the precise definitions that they have learned? Or, are they supposed to now presume that near enough is good enough?

For a pompous finale, the Examiners’ Report follows up by snarking that, in writing the integral for the PDF, “The dx was often missing from students’ working”. One would have thought that the examiners might have dispensed with their finely honed prissiness for that one paragraph. But no. For some clowns it’s never the wrong time to whine about a missing dx.

UPDATE (16 June): In the comments below, Terry Mills has made the excellent point that the prior question on the exam is similarly problematic. 3(h)(i) asks students to calculate the mean of the probability distribution, which would normally be calculated as \int xf(x)\,{\rm d}x. For our non-PDF, however, we should should normalise by dividing by \int f(x)\,{\rm d}x. To the demanded two decimal places, that changes the answer from the Examiners’ Report’s 170.01 to 170.06.