How to Play VCE with Mathematica

An article titled Mathematica and the Potential Gaming of VCE has just appeared in the MAV’s journal Vinculum (and we have posted it here). By Sai kumar Murali krishnan, who completed VCE last year and who we previously mentioned in this post, the article delivers what the title promises (noting the “Potential” is redundant): Sai demonstrates how Mathematica’s huge library of functions and extremely powerful programming can be used, and has been used, to trivialise VCE maths exams. We believe Sai’s article is very interesting and very important. (For anyone interested to do so, Sai can be contacted by email here.)

Also likely to be of interest, at least to readers of this blog, is the story of the long and weird battle to have Sai’s article appear. Roger Walter, Vinculum’s editor, deserves a hell of a lot of credit for seeing that battle through and ensuring Sai’s article survived, largely unscathed. And a disclaimer: we played a role in Sai choosing to write the article, and we were also involved in the subsequent battle. We intend to write on all of this in the near future.

With Sai’s permission, we’ve posted his article here. In this post we’ll give a few more examples and we’ll provide some concluding paragraphs, which didn’t make it into Sai’s published article. By way of background, Mathematica memory need not be cleared before taking an exam or SAC. Secondly, in computer-based (CBE) Methods, a student enters their answers directly into the Mathematica notebook; this means that Mathematica code and output in and of itself constitutes acceptable working, and is very close to sufficient as answer. 

First, here is a multiple choice question from the 2019 Mathematical Methods exam, which we also discussed here:

The problem is to determine Pr(X > 0). Here is Sai’s solution, utilising standard Mathematica functions:

The point is, of course, that the application of functions such as Area and Polygon requires very little sense of the mathematics involved. For an example requiring no mathematical sense whatsoever, consider the following multiple choice question, which appeared on the 2017 Mathematical Methods exam:

The question is of a standard type, and for these questions Sai created the Mathematica function FTest. The following is Sai’s complete Mathematica working to solve the question above:

A final example, again from the 2019 Mathematical Methods exam:

Here is Sai’s Mathematica working for this question, using two functions he created, FInfo and TangentLine:

Sai’s Vinculum paper contains a number of other examples, and Sai has created a huge library of incredibly sophisticated functions to tackle VCE questions, a library which he shared with his fellow VCE students. Sai’s work raises obvious issues, not least of which is the grossly unfair competition between the majority handheld-CAS students and the few Mathematica-powered students. The original version of Sai’s article ended with two paragraphs, which the MAV Publications Committee demanded be cut:

To the extent VCAA is aware of these issues, there is reason to doubt that they are sufficiently aware, or at least sufficiently concerned. VCAA, after all, has created and continues to maintain this strange and uneven playing field. As further evidence, VCAA provides sample Mathematica solutions, and it is telling that these solutions are clumsy, uninventive and calculator-mimicking, suggesting a limited understanding of Mathematica’s capabilities.

Whatever naivete may exist, I believe it is unlikely to last. Nothing precludes the marketing of Mathematica packages designed specifically for VCE testing and, if Mathematica becomes widely available in VCE, I believe this commercialisation is inevitable. Such a development would turn VCAA’s implementation of Mathematica, which is already very problematic, into an obvious farce. 

Of course the MAV having cut these paragraphs, along with every single reference to the VCAA, doesn’t make their content any less true, any less obvious or any less important.

We intend to write more later in the week.

Guest Post: Mathematica and the Potential Gaming of VCE

What follows is the article Mathematica and the Potential Gaming of VCE, by Sai kumar Murali krishnan, which has just appeared in Vinculum and which we have written about here. Sai’s article is reproduced here with Sai’s permission. Sai can be contacted by email here

 

INTRODUCTION

Last year I completed VCE, including Mathematical Methods (CBE) and Specialist Mathematics. At my school these subjects employed the computer system Mathematica in place of handheld CAS calculators. The CBE (Computer-Based Examination) version of Methods also entailed the direct submission of SACs and the second (tech-active) exam on the Mathematica platform.1

Mathematica is extraordinarily powerful and, as it happens, I consider myself a decent programmer. During VCE, I entertained myself by creating custom functions to automate tedious computations, which I then shared with my fellow students. We were able to store these functions in a paclet (package), ready for use on the SACs and the exam. While handheld CAS calculators can also store (less complex) custom-made functions, Mathematica’s vast in-built library and ease of use moves it into a different class. Mathematica enables the creation of exam-ready functions to perform any computation a student might require.

I have witnessed, and experienced, many problems with the implementation of Mathematica, but in this article I will focus upon the two most glaring and most important issues. First and foremost, Mathematica is so powerful that it can trivialise the testing of the mathematics for which it is purported to be a tool. It enables any student who can program in Mathematica or, more perversely, who has a friend, teacher or tutor who can program in Mathematica, to perform well in VCE mathematics. Secondly, and as an inevitable consequence of this trivialisation, the current partial implementation of Mathematica could create a grossly unfair competition, an unfairness enhanced in Methods CBE by effectively permitting Mathematica code to be submitted as an answer. The students equipped with handheld CAS calculators are the victims. Armed with toys sporting 70s Nintendo displays, they are being outgunned by students deploying full-screen guided missiles.

In this article I will illustrate how Mathematica can trivialise exam questions in Mathematical Methods. In Part 2, I provide an example of the use of Mathematica’s in-built functions. In Part 3, I consider the application of custom-built functions. In Part 4, I summarise, and I indicate why I believe the problems with the implementation of Mathematica are only likely to worsen.

 

IN-BUILT FUNCTIONS

We begin by looking at Question 5, Section B of 2019 Exam 2, which concerns the cubic \boldsymbol{f(x) = 1 - x^3}.

The question first prompts us to find the tangent at x = a, which we perform in one step with the function TangentLine.2 We then find the intersection points Q and P with two applications of the function Solve. Next, the area of the shaded region as a function of a is found by subtracting the area under the cubic from a triangular area: the former is found using the function Integrate, and the latter is found directly from the coordinates using the functions Polygon and Area. Finally, we are required to find the value of a that minimises the area, which is found in one step with the function ArgMin.

What follows is the complete Mathematica code to answer this five-part question:

This solution requires little mathematical understanding beyond being able to make sense of the questions. In particular, the standard CAS approach of setting up integrals and differentiating is entirely circumvented, as is the transcription. In Methods CBE, the above input and output would be considered sufficient answers.

 

CREATED FUNCTIONS

We’ll now venture into the world of custom Mathematica functions, where programmers can really go to town. We’ll first look at the topic of functions and the features of their graphs. Mathematica does not have a built-in function to give all the desired features, so I created the function DetailPlot. To begin, I use a module to gather data about a function, including endpoints, axial intercepts, stationary points, inflection points and, if required, asymptotes. I then turn the module into an image to place over the graph.

Let’s fire this new weapon at Q2(c), Section B of 2016 Exam 2, which concerns the pictured quartic. We are given the equation of the graph and the point A, and we are told that the tangents at A and D are parallel. We are then required to find the point D and the length of AE.

And, here we are:

With very little input, DetailPlot has provided a rich graph, with every feature one might require within easy reach. The intersection points are ‘callouts’, which means that the points are labelled with their coordinates. In particular the coordinates of D and E have been revealed by DetailPlot, without any explicit calculation. We can then press forward and finish by finding the length of AE, a trivial calculation with the in-built function EuclideanDistance.

In the next example I demonstrate that a multi-stage question can still be trivialised by a single piece of pre-arranged code. In the multiple choice question MCQ10 from 2017 Exam 2, the function \boldsymbol{f(x) = 3\sin \left(2\left[x+\frac{\pi}{4}\right]\right)} undergoes the transformation \boldsymbol{T\left(\Big[\begin{smallmatrix}x \\ y\end{smallmatrix}\Big]\right) = \Big[\begin{smallmatrix}2 & 0 \\ 0&\frac13\end{smallmatrix}\Big]\Big[\begin{smallmatrix}x \\ y\end{smallmatrix}\Big]} and we are required to identify the resulting function. For such questions I created the function Transform, and then the in-built function FullSimplify polishes off the question:

My last example is on functional equations, for which I created two functions, FTest and RFTest. I will illustrate the use of the latter function. For MCQ11 on 2016 Exam 2, the equation f(x) – f(y) = (yx) f(xy) is given and it is required to determine which of the given functions satisfies the equation. Here is my entire solution:

 

CONCLUSION

It is impossible to have a proper sense of the power of Mathematica unless one is a programmer familiar with the package. This article presents just a few examples from the vast library of functions I created for Mathematical Methods and I found even more so for Specialist Mathematics. My libraries for both subjects barely scratch the surface of what is possible.

Creating such packages requires skill in both programming and mathematics, but the salient point is that any subsequent application of those programs by another student requires no comparable skill. The programs I have written may improve the performance of mathematically weaker students. Conversely, any student without access to such programs or, worse, is required to use handheld CAS instead of Mathematica, will be at a significant disadvantage.

This demonstrates the potential power of Mathematica to change the focus of VCE mathematics and, consequently, to debase its teaching and its assessment. True, the same issues had already arisen with the introduction of handheld CAS; clever teachers and cleverer students have always engaged in creating and sharing push-a-button CAS programs. Mathematica, however, has massively elevated the seriousness of these issues, all the more so since only a fraction of students have access to the platform.3

Technology, including Mathematica, calculators, spreadsheets and the many online programs, have tremendous potential to assist students with learning, understanding and applying mathematics. What is important for educators is to be careful that students are not using this technology to bypass learning and understanding mathematics.

1. All non-CBE students take the same tech-active exams and are considered in the same cohort for ATAR purposes. The Methods (CBE) exam appears to differ in only a superficial manner, and it appears that CBE students have not been considered a separate cohort since 2016.

2. The examination diagrams have been redrawn for greater clarity.

3. Although the Victorian Government offers Mathematica to all schools, to date many schools have not implemented it.

© Sai kumar Murali krishnan 2020

Letter from a Concerned Student

A few days ago we received an email from “Concerned Student”, someone we don’t know, requesting advice on how to approach VCE mathematics. We have thoughts on this and intend to reply, but the email also seemed generally relevant and of likely interest. The email also raises interesting questions for teachers, and for the writer of this blog. With Concerned Student’s permission, we’ve reproduced their email below. We’ll hold off commenting until others, who actually know what they’re talking about, have had a go. Here is CS’s email:

It seems clear from reading this blog that a significant proportion of the VCE Methods & Specialist curricula are in direct conflict with good mathematical education. As someone entering these subjects next year, what’s the recommended approach to make it through all the content of the study design while also *learning maths*? Should I largely ignore the (… Cambridge) textbook and overall course and focus on self-teaching content along the same lines from better sources, stopping only to learn specifically from the curriculum whatever button mashing is necessary for an exam; or should I instead focus on fighting through the curriculum, and learn some proper maths on the side – I guess the productive question there is “is it easy enough to apply properly learnt maths to the arcane rituals found in VCE course assessments?”

It’s probably worth noting that, as far as I’m aware, the Methods & Specialist teachers at my school are known for being quite good, but they’re obviously still bound by the curriculum they teach.

WitCH 38: A Deep Hole

This one is due to commenter P.N., who raised it on another post, and the glaring issue has been discussed there. Still, for the record it should be WitCHed, and we’ve also decided to expand the WitCHiness slightly (and could have expanded it further).

The following questions appeared on 2019 Specialist Mathematics NHT, Exam 2 (CAS). The questions are followed by sample Mathematica solutions (screenshot corrected, to include final comment) provided by VCAA (presumably in the main for VCE students doing the Mathematica version of Methods). The examination report provides answers, identical to those in the Mathematica solutions, but indicates nothing further.

UPDATE (05/07/20)

The obvious problem here, of course, is that the answer for Part (b), in both the examination report and VCAA’s Mathematica solutions, is flat out wrong: the function fk will also fail to have a stationary point if k = -2 or k = 0. Nearly as bad, and plenty bad, the method in VCAA’s Mathematica solutions to Part (c) is fundamentally incomplete: for a (twice-differentiable) function f to have an inflection point at some a, it is necessary but not sufficient to have f’’(a) = 0.

That’s all pretty awful, but we believe there is worse here. The question is, how did the VCAA get it wrong? Errors can always occur, but why specifically did the error in Part (b) occur, and why, for a year and counting, wasn’t it caught? Why was a half-method suggested for Part (c), and why was this half-method presumably considered reasonable strategy for the exam? Partly, the explanation can go down to this being a question from NHT, about which, as far as we can tell, no one really gives a stuff. This VCAA screw-up, however, points to a deeper, systemic and much more important issue.

The first thing to note is that Mathematica got it wrong: the Solve function did not return the solution to the equation fk‘ = 0. What does that imply for using Mathematica and other CAS software? It implies the user should be aware that the machine is not necessarily doing what the user might reasonably think it is doing. Which is a very, very stupid property of a black box: if Solve doesn’t mean “solve”, then what the hell does it mean? Now, as it happens, Mathematica’s/VCAA’s screw-up could have been avoided by using the function Reduce instead of Solve.* That would have saved VCAA’s solutions from being wrong, but not from being garbage.

Ask yourself, what is missing from VCAA’s solutions? Yes, yes, correct answers, but what else? This is it: there are no functions. There are no equations. There is nothing, nothing at all but an unreliable black box. Here we have a question about the derivatives of a function, but nowhere are those derivatives computed, displayed or contemplated in even the smallest sense.

For the NHT problem above, the massive elephant not in the room is an expression for the derivative function:

    \[\color{red} \boldsymbol{f'_k(x) = -\frac{x^2 + 2(k+1)x +1}{(x^2-1)^2}}\]

What do you see? Yep, if your algebraic sense hasn’t been totally destroyed by CAS, you see immediately that the values k = 0 and k = -2 are special, and that special behaviour is likely to occur. You’re aware of the function, alert to its properties, and you’re led back to the simplification of fk for these special values. Then, either way or both, you are much, much less likely to screw up in the way the VCAA did.

And that always happens. A mathematician always gets a sense of solutions not just from the solution values, but also from the structure of the equations being solved. And all of this is invisible, is impossible, all of it is obliterated by VCAA’s nuclear weapon approach.

And that is insane. To expect, to effectively demand that students “solve” equations without ever seeing those equations, without an iota of concern for what the equations look like, what the equations might tell us, is mathematical and pedagogical insanity.

 

*) Thanks to our ex-student and friend and colleague Sai for explaining some of Mathematica’s subtleties. Readers will be learning more about Sai in the very near future.

WitCH 37: A Foolproof Argument

We’re amazed we didn’t know about this one, which was brought to our attention by commenter P.N.. It comes from the 2013 Specialist Mathematics Exam 2: The sole comment on this question in the Examination Report is:

“All students were awarded [the] mark for this question.”

Yep, the question is plain stuffed. We think, however, there is more here than the simple wrongness, which is why we’ve made it a WitCH rather than a PoSWW. Happy hunting.

UPDATE (11/05) Steve C’s comment below has inspired an addition:

Update (20/05/20)

The third greatest issue with the exam question is that it is wrong: none of the available answers is correct. The second greatest issue is that the wrongness is obvious: if z^3 lies in a sector then the natural guess is that z will lie in one of three equally spaced sectors of a third the width, so God knows why the alarm bells weren’t ringing. The greatest issue is that VCAA didn’t have the guts or the basic integrity to fess up: not a single word of responsibility or remorse. Assholes.

Those are the elephants stomping through the room but, as commenters as have noted, there is plenty more awfulness in this question:

  • “Letting” z = a + bi is sloppy, confusing and pointless;
  • The term “quadrant” is undefined;
  • The use of “principal” is unnecessary;
  • “argument” is better thought as the measure of an angle not the angle itself;
  • Given z is a single complex number, “the complete set of values for Arg(z)” will consist of a single number.
  • The grammar isn’t.

SACs of Shit

SACs may not be the greatest problem with VCE mathematics, but they’re right up there. SACs are torture for teachers and torture for students. They teach nothing. As assessment, they are unnecessary, unreliable and phenomenally inefficient. They are a license for VCAA’s unaswerable auditors applying Kafkaesque rules to act either as favour-givers or as little Hitlers, as the mood takes them. These problems are currently amplified to eleven by VCAA’s “We’ll give you some kind of guidance in, oh, a little while” plan for the plague year.

For all of the awfulness of the above, that’s not the worst of it. The worst is that the majority of SACs are monumentally stupid. Literally. A SAC has the imposing presence of a monument, its towering stupidity casting a shadow over everything.

How are SACs so bad? Many contain errors, often subtle although too often not, but, as irritating as that is, that is not the main problem. The main problem is that they are mathematical nonsense. Typically they will present the student with a ridiculous model of a contrived problem, which is then all redone in greater, brain-bludgeoning generality by throwing in a needless parameter in a randomly chosen location. All of this is undertaken, of course, in the nihilistic world of CAS. Finally, somewhere near the end, the poor beleaguered student, who by this stage just wants to escape with their life, will be required to “comment on the model”, to which the usual response is “It’s really nice, please let me go” and to which the only reasonable response is “It’s fucking insane”.

How do we know SACs are this bad? Because we see them. We see the commercial SACs, and the sample SACs, and the past SACs, and the current SACs. Are they all as bad as we suggest? No, of course not. Specialist SACs are typically nowhere near as bad as Methods SACs, and even many Methods SACs will fall short of truly idiotic torture, rising only to the level of being dumb and painful. Then there are the rare few SACs we see that are good, resulting in an exchange:

“This actually makes sense. Who’s your teacher?”

“Oh, it’s Mr. ….”

“Ah. Yes.”

So, yes, the quality and worth of SACs varies widely, but the average is squarely in the neighbourhood of monumental, tortuous stupidity. Which bring us to the “why”. Why are SACs in general so awful? There are two reasons.

The first and fundamental reason is the VCAA and their view of what they imagine is a curriculum. VCE mathematics subjects are so shallow and so lacking in a foundation of solid reason, that almost any attempt at depth and substance in a SAC is destined to be farce. The VCAA has replaced foundation and depth with CAS, which reaches peak awfulness in SACs. The VCAA promotes the fantasy that CAS magically transforms students into mathematical explorers, clever little Lewises and Clarks skilfully navigating the conceptual wilderness. The reality, of course, is much less Lewis and Clark than it is Burke and Wills. To top it off, SACs must follow guidelines that Terry Gilliam would be proud of, giving us Burke and Wills’ Bogus Brazilian Journey. Or, just Eraserhead. Something like that.

The second reason is the teachers. Sort of. Even if the subjects were coherent, even if they were unpoisoned by CAS and were unconstrained by vague and ridiculous conditions, even then writing a good SAC would be a very difficult and massively time-consuming task. Most teachers just don’t have the mathematical background, or the literary skill, to write a coherent, correct and mathematically rich SAC; many cannot even recognise one. And, that’s writing a good SAC for this imaginary good subject; writing a good SAC for these fundamentally flawed subjects with their ridiculous constraints is close to impossible, even for a strong teacher. And which teachers, particularly weaker teachers, have the time to compose such a good SAC? Why bother trying? And so, with the greatest common sense, most teachers do not. Most teachers stick to the audit-proof and meaningless formulaic SAC bullshit that the VCAA expects and effectively demands.

The VCAA’s SAC system is a crime against mathematical humanity.

UPDATE (15/5)

We received the following from a student acquaintance (who hadn’t read this post):

Hi Marty, given the upcoming math SACs approaching soon, the pressure is on to practice and practice. Attached below is last year’s Methods SAC1 (Unit 3/4) for [the student’s school]. I remember many talented friends of mine who were stumped, and didn’t do very well on this SAC. Personally, I thought this SAC was horrifying. In contrast to Specialist, (I actually quite enjoy Specialist!), Methods seems to be a huge prick because of frustrating, ambiguous SACs containing questions seemingly cooked from the pits of hell itself. Are these sort of SACs common across the state?

The student is, of course, correct. The SAC, which comes from a highly respected school, is a nightmare in all of the ways canvassed above. From start to end it is idiotic CAS-driven pseudo-modelling, complete with Magritte nonsense and a pointlessly prissy grading scheme. And, yes, the SAC contains an error.

Of course we won’t reveal the school, much less any teachers involved, which means that we are also unable to critique the SAC in detail. But that is one of the insidious aspects of the SAC system; an entirely proper concern for privacy means that SAC nonsense, although endemic, fails to be exposed to the public critique that is so very much needed.

UPDATE (26/7)

Once again, this time in response to this post, a student from a “good” school has contacted us in regard to their SAC. This was a Specialist SAC, and the student had contacted us because the teachers/writers had screwed up: some tech aspects of the SAC were a mess, and the subsequent clean-up of the mess was clearly disingenuous and clearly insufficient. But, as always, the situation was much worse that the student suggested.

The student’s SAC was ridiculous. From beginning to end it was pointless, CAS-driven pseudo-modelling. It had the idiotic parameters thrown in. It was poorly written. It displayed poor mathematical understanding, leading to ridiculous own goals. It. Was. Not. Mathematics.

And, we can’t write the details of any of this.

MitPY 6: Integration by Substitution

From frequent commenter, SRK:

A question for commenters: how to explain / teach integration by substitution? To organise discussion, consider the simple case

    \[\boldsymbol{ \int \frac{2x}{1+x^2}dx \,.}\]

Here are some options.

1) Let u = 1 + x^2. This gives \frac{du}{dx} = 2x, hence dx = \frac{du}{2x}. So our integral becomes \int \frac{2x}{u}\times \frac{du}{2x} = \int \frac{1}{u}du. Benefits: the abuse of notation here helps students get their integral in the correct form. Worry: I am uncomfortable with this because students generally just look at this and think “ok, so dy/dx is a fraction cancel top and bottom hey ho away we go”. I’m also unclear on whether, or the extent to which, I should penalise students for using this method in their work.

2)  Let u = 1 + x^2. This gives \frac{du}{dx} = 2x. So our integral becomes \int \frac{1}{u}\times \frac{du}{dx}dx = \int \frac{1}{u}du. Benefit. This last equality can be justified using chain rule. Worry: students find it more difficult to get their integral in the correct form.

3) \frac{2x}{1+x^2} has the form f'(g(x))g'(x) where g(x)=1+x^2 and f'(x) = \frac{1}{x}. Hence, the antiderivative is f(g(x)) = \log (1+x^2). This is just the antidifferentiation version of chain rule.  Benefit. I find this method crystal clear, and – at least conceptually – so do the students. Worry. Students often aren’t able to recognise the correct structure of the functions to make this work.

So I’m curious how other commenters approach this, what they’ve found has been effective / successful, and what other pros / cons there are with various methods.

UPDATE (21/04)

Following on from David’s comment below, and at the risk of splitting the discussion in two, we’ve posted a companion WitCH.