The Methods Intelligence Test

Psychologist Daniel Kahneman dedicates his book Thinking Fast and Slow  to the memory of Amos Tversky, his long-time collaborator. Tversky was considered so brilliant by his colleagues that they came up with the Tversky Intelligence Test:

The faster you realise that Amos Tversky is smarter than you, the smarter you are.

It has occurred to us that there is a similar Methods Intelligence Test:

The slower you realise that Methods is stupider than you, the stupider you are.

WitCH 33: Below Average

We’re not actively looking for WitCHes right now, since we have a huge backlog to update. This one, however, came up in another context and, after chatting about it with commenter Red Five, there seemed no choice. The following 1-mark multiple choice question appeared in 2019 Exam 2 (CAS) of VCE’s Mathematical Methods. The problem was to determine Pr(X > 0), the possible answers being

A. 2/3      B. 3/4      C. 4/5      D. 7/9      E. 5/6

Have fun.

Update (04/07/20)

Who writes this crap? Who writes such a problem, who proofreads such a problem, and then says “Yep, that’ll work”? Because it didn’t work, and it was never going to. The examination report indicates that 27% of students gave the correct answer, a tick or two above random guessing.
 
We’ll outline a solution below, but first to the crap. The main awfulness is the double-function nonsense, defining the probability distribution \boldsymbol{f} in terms of pretty make the same function \boldsymbol{p}. What’s the point of that? Well, of course \boldsymbol{f} is defined on all of \boldsymbol{R} and \boldsymbol{p} is only defined on \boldsymbol{[-a,b]}. And, what’s the point of defining \boldsymbol{f} on all of \boldsymbol{R}? There’s absolutely none. It’s completely gratuitous and, here, completely ridiculous. It is all the worse, and all the more ridiculous, since the function \boldsymbol{p} isn’t properly defined or labelled piecewise linear, or anything; it’s just Magritte crap. 
 
To add to the Magritte crap, commenter Oliver Oliver has pointed out the hilarious Dali crap, that the Magritte graph is impossible even on its own terms. Beginning in the first quadrant, the point \boldsymbol{(b,b)} is not quite symmetrically placed to make a 45^{\circ} angle. And, yeah, the axes can be scaled differently, but why would one do it here? But now for the Dali: consider the second quadrant and ask yourself, how are the axes scaled there? Taking a hit of acid may assist in answering that one.
 
Now, finally to the problem. As we indicated, the problem itself is fine, its just weird and tricky and hellishly long. And worth 1 mark. 
 
As commenters have pointed out, the problem doesn’t have a whole lot to do with probability. That’s just a scenario to give rise to the two equations, 
 
1) \boldsymbol{a^2 \ +\ \frac{b}{2}\left(2a+b\right) = 1} \qquad      \mbox{(triangle + trapezium = 1).}
 
and
 
2) \boldsymbol{a + b = \frac43} \qquad           \mbox{( average = 3/4).}
 
The problem is then to evaluate
 
*) \boldsymbol{\frac{b}2(2a + b)} \qquad \mbox{(trapezium).}
 
or, equivalently, 
 
**) \boldsymbol{1 - a^2 \qquad} \mbox{(1 - triangle).}
 
 
The problem is tricky, not least because it feels as if there may be an easy way to avoid the full-blown simultaneous equations. This does not appear to be the case, however. Of course, the VCAA just expects the lobotomised students to push the damn buttons which, one must admit, saves the students from being tricked.
 
Anyway, for the non-lobotomised among us, the simplest approach seems to be that indicated below, by commenter amca01. First multiply equation (1) by 2 and rearrange, to give
 
3) \boldsymbol{a^2 + (a + b)^2 = 2}.
 
Then, plugging in (2), we have 
 
4) \boldsymbol{a^2 = \frac29}.
 
That then plugs into **), giving the answer 7/9. 
 
Very nice. And a whole 90 seconds to complete, not counting the time lost making sense of all the crap. 

WitCH 31: Decomposing

We have a short Specialist post coming, and we’ll have more to write on the 2019 VCE exams once they’re online. But, for now, one more Mathematical Methods WitCH, from the 2019 (calculator-free) Exam 1:

Update (04/07/20)

The main crap here, of course, is part (f): as commenter John Friend puts it, what the hell is this question supposed to be testing? And, sure, the last part of the last question on an exam is allowed to be a little special, but one measly mark? Compared to the triviality of the rest of the question?

Of course, students bombed part (f). The examination report indicates that 19% of student correctly answered that there is one solution to the equation; as suggested by commenter Red Five, it’s also a pretty safe bet that the majority of students who got there did so with a Hail Mary guess. (It should be added, the students didn’t do swimmingly well on the rest of Question 9, the CAS-lobotomising having working its usual magic.)

OK, so what did examiners expect for that one measly mark? We’ll get to a reasonable solution below, but let’s first consider some unreasonable solutions.

Here is the examination report’s entire commentary on Part (f):

g(f(x) + f(g(x)) = 0 has exactly one solution.

This question was not well done. Few students attempted to draw a rough sketch of each equation and use addition of ordinates.

Gee, thanks. Drawing a “rough sketch” of either of these compositions is anything but trivial. For one measly mark. We’ll look at sketching aspects of these graphs below, but let’s get on with another unreasonable solution.

Given the weirdness of part (f), a student might hope that parts (a)-(e) provide some guidance. Let’s see.

Part (b) (for which the examination report contains an error), gets us to conclude that the composition

\boldsymbol{g(f(x)) = e^{\left(3+2x-x^2\right)}} has negative derivative when x > 1.

Part (c) leads us to the composition

\boldsymbol{f(g(x)) = \left(3 - e^x\right) \left(1 + e^x\right)}

having x-intercept when x = log(3).

Finally, Part (e) gives us that the composition f(g(x)) has the sole stationary point (0,4). How does this information help us with Part (f)? Bugger all.

So, what if we include the natural implications of our previous work? That gives us something like the following: Well, um, great. We’re left still hunting for that one measly mark.

OK, the other parts of the question are of little help, and the examiners are of no help, so what do else do we need? There are two further pieces of information we require (plus the Intermediate Value Theorem). First, note that

\boldsymbol{g(f(x)) = e^{\mbox{\bf THING}} > 0}.

Secondly, note that

\boldsymbol{f(g(x)) = \left(3 + e^x\right) \left(1 - e^x\right)} = -}\mbox{\bf HUGE} if x is huge.

Then, given we know the slopes of the compositions, we can finally complete our rough sketches: Now, let’s write S(x) for our sum function g(f(x)) + f(g(x)). We know S(x) > 0 unless one of our compositions is negative. So, the only place we could get S = 0 is if x > log(3). But S(log(3)) > 0, and eventually S is hugely negative. That means S must cross the x-axis (by IVT). But, since S is decreasing for x > 1, S can only cross the axis once, and S = 0 must have exactly one solution. 

We’ve finally earned our one measly mark. Yay?

WitCH 29: Bad Roots

This one is double-barrelled. A strange multiple choice question appeared in the 2019 NHT Mathematical Methods Exam 2 (CAS). We had thought to let it pass, but a similar question appeared in last week’s Methods exam (no link yet, but the Study Design is here). So, here we go. First, the NHT question: The examination report indicates the correct answer, C, and provides a suggested solution:

\Large\color{blue} \boldsymbol{ g(x)=f^{-1}(x)=\frac{x^{\frac15}-b}{a},\ g'(x) = \frac{x^{-\frac45}}{5a},\ g'(1) = \frac1{5a}}

And, here’s last week’s question (with no examination report yet available):

Update (19/06/20)

As commenters have noted, it is very difficult to understand any purpose to these questions. They obviously suggest the inverse function theorem, testing the knowledge of and application of the formula g'(d) = 1/f'(c), where f(c) =d. The trouble is, the inverse function theorem is not part of the curriculum, appearing only implicitly as a dodgy version of the chain rule, and is typically only applied in Leibniz form.

As indicated by the solution in the first examination report, the intent seems to have been for students to have explicitly computed the inverses, although probably with their idiot machines. (The second examination report has now appeared, but is silent on the intended method.) Moreover, as JF noted below, the algebra in the first question makes the IFT approach somewhat fiddly. But, what is the point of pushing a method that is generally cumbersome, and often impossible, to apply?

To add to the nonsense, below is a sample solution for the first question, provided by VCAA to students undertaking the Mathematica version of Methods. So, the VCAA has suggested two approaches, one which is generally ridiculous and another which is outside the curriculum. That makes it all as clear as dumb mud.

WitCH 28: Tone Deaf

We haven’t yet had a chance to go through the 2019 VCE exams, but this question was flagged to me independently by two colleagues: let’s call them Dr. Death and Simon the Likeable. It’s from Mathematical Methods Exam 2 (CAS). (No link yet.)

UPDATE (05/07/20)

Even ignoring the stuff-ups, this question is ugly and pointless; the pseudo-applied framing is ugly and pointless; the CASification is ugly and pointless; the back-to-front integral is ugly and pointless; the matrix equation is ugly and pointless; the transformation is really ugly and really pointless. Part (f) is the pinnacle of ugliness and pointlessness, but the entire question is swill, from beginning to end.

And then there’s Part (e). “This question was not answered well” the examiners solemnly intone. Gee, really? Do you think your question being completely stuffed might have had something to do with it? Do you think maybe having a transformation of x when there’s not an x in sight may have been just a tad confusing? Do you think that the transformation then resulting in a function of t was maybe not the smartest move? Do you think writing an integral backwards was perhaps just a little too cute? Do you think possibly referring to the area of, rather than to the value of, an integral was slightly clunky? And, most importantly, do you think perhaps asking a question for which there is an infinite and impenetrable jungle of answers may have been an exercise in canyon-sized incompetence?

But, sure, those troublesome students didn’t answer your question well.

Part (e) was intended to have students find a transformation of the function f that effectively switches the behaviour on the intervals [0,4] and [4,6] to the intervals [2,6] and [0,2].  Ignoring the fact that the intended question was asked in an absurdly opaque manner, and ignoring the fact that no motivation for the intended question was either provided or is imaginable, the question asked was entirely different, and was ridiculous.

Writing the transformation out,

    \[\boldsymbol{\left\{\aligned &X = ax + c \\ &Y = by + d, \right.\endaligned}\]

we then have

    \[\boldsymbol{\left\{\aligned &x = \frac{X - c}{a} \\ & y = \frac{Y - d}{b}. \right.\endaligned}\]

So, the function y = f(t) y = f(x) can be written

    \[\boldsymbol{\dfrac{Y - d}{b} = f\!\left(\dfrac{X - c}{a}\right).}\]

Solving for Y, that means our transformed function Y = g(X) can be written

    \[\boldsymbol{g(X) = b\, f\!\left(\dfrac{X - c}{a}\right) + d.}\]

Well, this is our function g unless a = 0, in which case g doesn’t exist. Whatever. Back to the swill.

Using the result from Part (d), we have Part (e) asking for a, b, c and d such that

    \[\boldsymbol{\int\limits_2^0 + \int\limits_2^6 \ \left[ b\, f\! \left(\dfrac{X - c}{a}\right) + d\right]  \, {\rm d}X \ = \ \dfrac{15}{\pi}.}\]

What then are the solutions to this equation? The examination report lists a couple of families and then blithely remarks “There are other solutions”. Really? Then why didn’t you list them, you clowns? 

We’ll tell you why. Because the complete solution to this monster is a God Almighty multi-infinite mess. As a starting idea, pick any three of the variables, say a and b and c, to be whatever you want, and then try to adjust the fourth variable, d, to solve the equation. We’ll offer a prize for anyone who can give a complete solution. 

This question is as good an example as there can be of the pointlessness, the ugliness and the monumental klutziness of VCAA’s swamp mathematics.

WitCH 27: Uncomposed

Ah, so much crap …

Tons of nonsense to post on, and the Evil Mathologer is breathing down our neck. We’ll have (at least) three posts on last week’s Mathematical Methods exams. This one is by no means the worst to come, but it fits in with our previous WitCH, so let’s quickly get it going. It is from Exam 1. (No link yet, but the Study Design is here.)

Update (15/06/20)

The examination report (and exam) is out, so it’s time to wade into this swamp. Before doing so, we’ll note the number of students who sank; according to the examination report, the average score on this question was 0.14 + 0.09 + 0.14 ≈ 0.4 marks out of 4. Justified or not, students had absolutely no clue what to do. Now, into the swamp.

The main wrongness is in Part (b), but we’ll begin at the beginning: the very first sentence of Part (a) is a mess. Who on Earth writes

“The function f: R \to R, f(x)  is a polynomial function …”?

It’s like writing

“The Prime Minister Scott Morrison of Australia, Scott Morrison is a crap Prime Minister”.

Yes, you may properly want to emphasise that Scott Morrison is the Prime Minister of Australia, and he is crap, but that’s not the way to do it. This is nitpicking, of course, but there are two reasons to do so. The first reason is there is no reason not to: why forgive the gratuitously muddled wording of the very first sentence of an exam question? From these guys? Forget it. The second reason is that the only possible excuse for this ridiculous wording is to emphasise that the domain of f is all of R, which turns out to be entirely pointless.

Now, to Part (a) proper. This may come as a surprise to the VCAA overlords, but functions do not have “rules”, at least not unique ones.  The functions f(x) = -4x^2\left(x^2 - 1\right) and h(x) = 4x^2-4x^4, for example, are the exact same function. Yes, this is annoying, but we’re sorry, that’s the, um, rule. Again this is nitpicking and, again, we have no sympathy for the overlords. If they insist that a function should be regarded as a suitable set of ordered pairs then they have to live with that choice. Yes, eventually ordered pairs are the precise and useful way to define functions, but in school it’s pretty much just a pedantic pain in the ass.

To be fair, we’re not convinced that the clumsiness in the wording of Part (a) contributed significantly to students doing poorly. That is presumably much more do to with the corruption of students’ arithmetic and algebraic skills, the inevitable consequence of VCAA and ACARA calculatoring the curriculum to death.

On to Part (b), where, having found f(x) = -4x^2\left(x^2 - 1\right) or whatever, we’re told that g is “a function with the same rule as f”. This is ridiculous and meaningless. It is ridiculous because we never did anything with f in the first place, and so it would have been a hell of lot clearer to have simply begun the damn question with g on some unknown domain E. It is meaningless because we cannot determine anything about the domain E from the information provided. The point is, in VCE the composition \log(g(x)) is either defined (if the range g(E) is wholly contained in the positive reals), or it isn’t (otherwise). End of story.  Which means that in VCE the concept of “maximal domain” makes no sense for a composition. Which means Part (b) makes no sense whatsoever. Yes, this is annoying, but we’re sorry, that’s the, um, rule.

Finally, to Part (c). Taking (b) as intended rather than written, Part (c) is ok, just some who-really-cares domain trickery.

In summary, the question is attempting and failing to test little more than a pedantic attention to boring detail, a test that the examiners themselves are demonstrably incapable of passing.

WitCH 26: Imminent Domain

The following WitCH is pretty old, but it came up in a tutorial yesterday, so what the Hell. (It’s also a good warm-up for another WitCH, to appear in the next day or so.) It comes from the 2011 Mathematical Methods Exam 1:

For part (a), the Examination Report indicates that f(g)(x) =([x+2][x+8]), leading to c = 2 and d = 8, or vice versa. The Report indicates that three quarters of students scored 2/2, “However, many [students] did not state a value for c and d”.

For Part (b), the Report indicates that 84% of students scored 0/2. After indicating the intended answer, (-∞,-8) U (-2,∞) (-∞,-8] U [-2,∞) or R\backslash(-8,-2), the Report goes on to comment:

“This question was very poorly done. Common incorrect responses included [-3,3] (the domain of  f(x); x ≥ -2 (as the ‘intersection’ of  x ≥ -8 with x ≥ -2); or x ≥ -8 (as the ‘union’ of x ≥ -8 with x ≥ -2). Those who attempted to use the properties of composite functions tended to get confused. Students needed to look for a domain that would make the square root function work.”

The Report does not indicate how students got “confused”, although the composition of functions is briefly discussed in the Study Design (page 72).

WitCH 24: The Fix is In

We’ve finally found some time to take a look at VCAA’s 2019 NHT exams. They’re generally bad in the predictable ways, and they include some specific and seemingly now standard weirdness that we’ll try to address soon in a more systematic manner. WitCHwise, we were tempted by a number of questions, but we’ve decided to keep it to two or three.

Our first NHT WitCH is from the final question on Exam 2 (CAS) of Mathematical Methods:

As usual, the NHT “Report” indicates nothing of how students went, and little of what was expected. In regard to part f, the Report writes,

p(x) = q(x) = x, p'(x) = q'(x) = 1, k = 1/e

For part g, all that the Report provides is the answer, k = 1.

The VCAA also provides sample Mathematica solutions to schools trialling Methods CBE. For the questions above, these solutions are as follows:

Make of it what you will.

VCAA’s SAC of Roaming


One type of educational horror that we haven’t yet written about are SACs, those internal assignmenty-examinationy things that make every second week of Year 12 studies a living hell. It is a tricky topic since SACs are school-based, often teacher-specific, and our primary goal is to attack inept authority. In that regard, schools and beleaguered teachers are in a weird middle ground, part victim and part villain, and they already have plenty of critics. Nonetheless, SACs are the sea in which students and teachers swim (or sink), and mathematics SACs are typically appalling; the overwhelming majority of mathematics SACs that we see are pointless, anti-mathematical, error-strewn blivits. So, something has to be written about such SACs of shit. And, we have a plan.

Our hand has been forced a little, however, by an email we received from a VCE student. The student is taking Mathematical Methods CBE, the trial version of Methods that uses Mathematica instead of CAS, and the student wrote about a recent Mathematica-based SAC at their school. We then asked a teacher at the school about the SAC, and they confirmed the students’ report.

This then is the students’ story, exactly as written to me.

I’m not sure of how many other students you know that are doing CBE methods but my sac today served pretty well to show how awful things can go, so a new perspective is always welcome right?

Starting from the top, we have school-provided laptops with Mathematica preinstalled. So we go in, and we have to utilise this thing called a palette which takes control of Mathematica (I have many complaints against the palette) and downloads the SAC from some remote server. No problems here right? Well, I’d imagine 200 people simultaneously downloading an item from a server would MAYBE just MAYBE cause some congestion in the network. Hell breaks loose here, a class has their file downloaded and enters reading time while the other 4 or so classes are in utter chaos. The downloader is failing over and over while also saying it succeeded. This goes on for a good half an hour before the teachers collectively decided that the sac would be rescheduled to Thursday. The class that began reading? Oh, they just stop. God knows what they could have done, taken photos of the sac with a snipping tool, copied the files over, the possibilities are endless. This whole thing is almost appalling yet terrifying because this is what I’ll have to do at the end of the year. I’ve got a load of other concerns, among the unending variety of methods to do a question with Mathematica and how an assessors report would assign marks, to which our official VCAA quiz provided pissant “solutions” that were often wrong.

Anyway, my slightly irritated take on the abhorrent state of the CBE system, which I thought may be interesting to you.

To that, the teacher at the school added the following:

[The former Head of mathematics at the school] made a deal with the devil and agreed to the school doing Methods CBE. I don’t feel the consultation process valued the feedback from myself and other teachers – it was always going to happen despite the misgivings of other teachers. I can’t help but think that my feelings on the issue is the reason I’m not teaching Maths Methods this year (for the first time in 9 years). There are many problems with this deal – it makes my blood boil. The current head of maths is a very decent guy and has done a fantastic job dealing with the mess he inherited.

I heard that the SAC was a disaster and actually saw events unfolding from afar – like watching a car crash in slow motion. Blind Freddy could have seen what was going to happen. As I left school, I saw the VCE coordinator and the current Head of Maths running around grim faced.

All of the student’s concerns are legitimate. Furthermore, the SAC was meant to run until 4.45 pm, so many students will have made alternative and inconvenient arrangements to accommodate this and now they have to do it all over again. Not to mention what it’s done to the stress levels of many students. Not to mention the time and resources that had to be expended re-writing the SAC. At every stage VCAA have washed their hands of CBE problems and left the school to do its dirty work, using the students as the guinea pigs.

Further,

1. The palette provided by VCAA had a bug.

2. The VCAA server failed. VCAA are trying to blame the school for both errors and no apology has been given. Re: The server fail. VCAA said that the school should have downloaded from the server prior to the SAC starting (which is not practical). VCAA are saying everything worked fine at the other CBE schools (which all have small student cohorts as opposed to our school’s cohort of over 200, which makes a big difference).

That’s it. Our own point of view is that SACs are all but guaranteed to be awful and Mathematica in the classroom is all but guaranteed to be awful. Here, however, those predictable awfulnesses are beside the point. The point here is VCAA’s Trumplike level of incompetence combined with VCAA’s Trumplike unwillingness to accept responsibility.

WitCH 19: A Powerful Solvent

The following WitCH is from VCE Mathematical Methods Exam 2, 2009. (Yeah, it’s a bit old, but the question was raised recently in a tutorial, so it’s obviously not too old.) It is a multiple choice question: The Examiners’ Report indicates that just over half of the students gave the correct answer of B. The Report also gives a brief indication of how the problem was to be approached:

    \[\mbox{\bf Solve } \boldsymbol{\frac{1}{k-0} \int\limits_0^k \left(\frac1{2x+1}\right)dx = \frac16\log_e(7) \mbox{ \bf for $\boldsymbol k$}.\ k = 3.}\]

Have fun.

Update (02/09/19)

Though undeniably weird and clunky, this question clearly annoys commenters less than me. And, it’s true that I am probably more annoyed by what the question symbolises than the question itself. In any case, the discussion below, and John’s final comment/question in particular, clarified things for me somewhat. So, as a rounding off of the post, here is an extended answer to John’s question.

Underlying my concern with the exam question is the use of “solve” to describe guessing/buttoning the solution to the (transcendental) equation \mathbf {\frac1{2k}{\boldsymbol \log} (2k+1) = \frac16{\boldsymbol \log} 7}.  John then questions whether I would similarly object to the “solving” of a quintic equation that happens to have nice roots. It is a very good question.

First of all, to strengthen John’s point, the same argument can also be made for the school “solving” of cubic and quartic equations. Yes, there are formulae for these (as the Evil Mathologer covered in his latest video), but school students never use these formulae and typically don’t know they exist. So, the existence of these formulae is irrelevant for the issue at hand.

I’m not a fan of polynomial guessing games, but I accept that such games are standard and that  “solve” is used to describe such games. Underlying these games, however, are the integer/rational root theorems (which the EM has also covered), which promise that an integer/rational coefficient polynomial has only finitely many candidate roots, and that these roots are easily enumerated. (Yes, these theorems may be a less or more explicit part of the game, but they are there and they affect the game, if only semi-consciously.) By contrast, there is typically no expectation that a transcendental equation will have somehow simple solutions, nor is there typically any method of determining candidate solutions.

I find something generally unnerving about the exam question and, in particular, the Report. It exemplifies a dilution of language which is at least confusing, and I’d suggest is actively destructive. At its weakest, “solve” means “find the solutions to”, and anything is fair game. This usage, however, loses any connotation of “solve” meaning to somehow figure out the way the equation works, to determine why the solutions are what they are. This is a huge loss.

True, the investigation of equations can continue independent of the cheapening of a particular word, but the reality is that it does not. Of course, in this manner the Solve button on CAS is the nuclear bomb that wipes out all intelligent life. The end result is a double-barrelled destruction of the way students are taught to approach an equation. First, students are taught that all that matters about an equation are the solutions.  They are trained to give the barest lip service to analysing an equation, to investigating if the equation can be attacked in a meaningful mathematical manner. Secondly, the students are taught that that there is no distinction between a precise solution and an approximation, a bunch of meaningless decimals spat out by a machine.

So, yes, the exam question above can be considered just another poorly constructed question. But the weird and “What the Hell” incorporation of a transcendental equation with an exact solution that students were supposedly meant to “solve” is emblematic of a an impoverishment of language and of mathematics that the CAS-infatuated VCAA has turned into an art form.