An article titled Mathematica and the Potential Gaming of VCE has just appeared in the MAV’s journal Vinculum (and we have posted it here). By Sai kumar Murali krishnan, who completed VCE last year and who we previously mentioned in this post, the article delivers what the title promises (noting the “Potential” is redundant): Sai demonstrates how Mathematica’s huge library of functions and extremely powerful programming can be used, and has been used, to trivialise VCE maths exams. We believe Sai’s article is very interesting and very important. (For anyone interested to do so, Sai can be contacted by email here.)
Also likely to be of interest, at least to readers of this blog, is the story of the long and weird battle to have Sai’s article appear. Roger Walter, Vinculum’s editor, deserves a hell of a lot of credit for seeing that battle through and ensuring Sai’s article survived, largely unscathed. And a disclaimer: we played a role in Sai choosing to write the article, and we were also involved in the subsequent battle. We intend to write on all of this in the near future.
First, here is a multiple choice question from the 2019 Mathematical Methods exam, which we also discussed here:
The problem is to determine Pr(X > 0). Here is Sai’s solution, utilising standard Mathematica functions:
The point is, of course, that the application of functions such as Area and Polygon requires very little sense of the mathematics involved. For an example requiring no mathematical sense whatsoever, consider the following multiple choice question, which appeared on the 2017 Mathematical Methods exam:
The question is of a standard type, and for these questions Sai created the Mathematica function FTest. The following is Sai’s complete Mathematica working to solve the question above:
A final example, again from the 2019 Mathematical Methods exam:
Here is Sai’s Mathematica working for this question, using two functions he created, FInfo and TangentLine:
Sai’s Vinculum paper contains a number of other examples, and Sai has created a huge library of incredibly sophisticated functions to tackle VCE questions, a library which he shared with his fellow VCE students. Sai’s work raises obvious issues, not least of which is the grossly unfair competition between the majority handheld-CAS students and the few Mathematica-powered students. The original version of Sai’s article ended with two paragraphs, which the MAV Publications Committee demanded be cut:
Whatever naivete may exist, I believe it is unlikely to last. Nothing precludes the marketing of Mathematica packages designed specifically for VCE testing and, if Mathematica becomes widely available in VCE, I believe this commercialisation is inevitable. Such a development would turn VCAA’s implementation of Mathematica, which is already very problematic, into an obvious farce.
Of course the MAV having cut these paragraphs, along with every single reference to the VCAA, doesn’t make their content any less true, any less obvious or any less important.
We intend to write more later in the week.
Yet more evidence (as if more was needed) of the current blatant and fetid cosy arrangement between the VCAA and the MAV under MAV’s current leadership (https://mathematicalcrap.com/2020/02/09/the-troubling-cosiness-of-the-vcaa-and-the-mav/).
As has been asked many times: How can the MAV possibly consider itself a voice for mathematics teachers when VCAA (and many organisations) is clearly such a protected species??
I’ll be writing about the reviewing process very soon. I’m just waiting for the MAV to not reply to questions I sent them.
Sai’s article is good stuff. I might be missing something obvious, but it seems as though the link https://mathematicalcrap.com/2020/07/13/guest-post-mathe%E2%80%A6al-gaming-of-vce/ in the post is broken.
Thanks, Glen. You mean the link to Sai’s article in my first sentence? It seems to work, but perhaps it got abbreviated somewhere.
The main questions here are:
1) Did VCAA conduct a thorough investigation into Mathematica and its potential to create advantage relative to the hand held CAS calculators before launching the Pilot program? Was there a report? If so, where is that report? Where’s the evidence?
I assume such an investigation was conducted and found that there was no advantage. Because how could you ethically proceed with a Pilot program knowing that it was going to give advantage to schools that participated? To do so would be extremely negligent at best and gravely unethical and potentially unlawful at worst. Note that students using Mathematica sat/sit exactly the same exam as those who do not.
So let’s assume that an investigation was conducted and a report written. Where’s the report??
2) Were there subsequent reviews of the Pilot program as the Pilot program proceeded and, I assume, new evidence collected? Not to have done so would be extremely negligent. Did such reviews suggest an advantage? Where are the follow-up reports? Where’s the evidence?
Or did all this happen in a silo with no unaccountability?
However, I am now going to put on my black hat and go down a cynical path of speculation ….
It seems to me that one could make some decent coin by creating a perception (factually true or otherwise) that Mathematica provided a big advantage to students if one was selling Mathematica code that purported to give students this advantage … In other words, the whole “Mathematica gives a big advantage” is simply a marketing ploy that preys on weak and desperate students. I submit (wearing my black hat) that Mathematica creates a DISadvantage for students using it because students – particularly the weaker and desperate ones – get conned into believing that code is king. They start believing that mathematical understanding is not really necessary because the code can do everything. Students start outsourcing their thinking to (one-time use?) Mathematica code written by FOR-profit entities and end up getting badly burnt. And teachers have to work extra hard undoing the damage created by this false belief.
If one was looking for evidence for or against the above theory, one would of course need to talk to the teachers who are using Mathematica with their students.
Some further comments:
Boutique code for boutique questions.
In the case of getting the tangent line, the handheld CAS calculator has a TangentLine command hardwired into it. No coding using the CAS calculator programming language required.
Q13 can be just as easily done on a handheld CAS.
Computer Based Exams (CBE) have been discontinued. So none of the Mathematica output constitutes acceptable working.
The article never makes a comparison with a competent user of a handheld CAS calculator who has also written and obtained programs. How can advantage be claimed when no comparison is ever made?
You honestly don’t see the point of Sai’s article? And just now?
Don’t play games.
I can see the point(s). I just don’t see how some of those points are justified. eg. How do you claim something gives an unfair advantage without making a comparison? Is not one of Sai’s points that Mathematica provides relative advantage because it “could create a grossly unfair competition” …?
No games being played here. But there are games being played somewhere.
We both are aware of the unease this article – eventually – caused in some circles. And if that unease caused Mathematica to be proscribed as a permitted CAS, then that would be a very poor and unjustified knee-jerk outcome in my view.
As for “just now” … I have re-read the article and seen things I didn’t previously see (or perhaps didn’t want to see due to confirmation bias). Delete the hyperbole and superlatives and look for the data and evidence for the assertions being made.
The article makes some big claims, claims that a less informed person (as well as any jealous trouble-maker) could easily misconstrue.
Putting aside the “CAS is poison” mantra (which you know I agree with), my beef is/was with the CBE (which I will claim to my dying breath actually imposes disadvantage on the poor saps it’s inflicted upon). For any given question, some brand/type of CAS will be more advantageous than another. The arguments about fairness of particular exam questions to Casio versus TI users regularly flares up. No doubt, exam questions can be cherry picked that favour Mathematica. Questions can also be cherry picked that favour handheld CAS calculators, and of those, some favour Casio and some favour TI.
As time passes and new evidence emerges, a person might change their mind and see things through a different lens. Time has passed, new evidence has emerged and I now view the article differently from 2 years ago.
Ah. You want it evidence-based.
Yes, evidence would be useful. As I asked earlier:
1) Did VCAA conduct a thorough investigation into Mathematica and its potential to create advantage relative to the hand held CAS calculators before launching the Pilot program? Was there a report? If so, where is that report? Where’s the evidence?
2) Were there subsequent reviews of the Pilot program as the Pilot program proceeded and, I assume, new evidence collected? Where are the follow-up reports? Where’s the evidence?
Maybe it was a one-man show with no accountability at the time?
The statute of limitations never expires on accountability.
You can argue with Sai, if he so cares. I think you’re being ridiculous, and disingenuous.