Guest Post: Mathematica and the Potential Gaming of VCE

What follows is the article Mathematica and the Potential Gaming of VCE, by Sai kumar Murali krishnan, which has just appeared in Vinculum and which we have written about here. Sai’s article is reproduced here with Sai’s permission. Sai can be contacted by email here

 

INTRODUCTION

Last year I completed VCE, including Mathematical Methods (CBE) and Specialist Mathematics. At my school these subjects employed the computer system Mathematica in place of handheld CAS calculators. The CBE (Computer-Based Examination) version of Methods also entailed the direct submission of SACs and the second (tech-active) exam on the Mathematica platform.1

Mathematica is extraordinarily powerful and, as it happens, I consider myself a decent programmer. During VCE, I entertained myself by creating custom functions to automate tedious computations, which I then shared with my fellow students. We were able to store these functions in a paclet (package), ready for use on the SACs and the exam. While handheld CAS calculators can also store (less complex) custom-made functions, Mathematica’s vast in-built library and ease of use moves it into a different class. Mathematica enables the creation of exam-ready functions to perform any computation a student might require.

I have witnessed, and experienced, many problems with the implementation of Mathematica, but in this article I will focus upon the two most glaring and most important issues. First and foremost, Mathematica is so powerful that it can trivialise the testing of the mathematics for which it is purported to be a tool. It enables any student who can program in Mathematica or, more perversely, who has a friend, teacher or tutor who can program in Mathematica, to perform well in VCE mathematics. Secondly, and as an inevitable consequence of this trivialisation, the current partial implementation of Mathematica could create a grossly unfair competition, an unfairness enhanced in Methods CBE by effectively permitting Mathematica code to be submitted as an answer. The students equipped with handheld CAS calculators are the victims. Armed with toys sporting 70s Nintendo displays, they are being outgunned by students deploying full-screen guided missiles.

In this article I will illustrate how Mathematica can trivialise exam questions in Mathematical Methods. In Part 2, I provide an example of the use of Mathematica’s in-built functions. In Part 3, I consider the application of custom-built functions. In Part 4, I summarise, and I indicate why I believe the problems with the implementation of Mathematica are only likely to worsen.

 

IN-BUILT FUNCTIONS

We begin by looking at Question 5, Section B of 2019 Exam 2, which concerns the cubic \boldsymbol{f(x) = 1 - x^3}.

The question first prompts us to find the tangent at x = a, which we perform in one step with the function TangentLine.2 We then find the intersection points Q and P with two applications of the function Solve. Next, the area of the shaded region as a function of a is found by subtracting the area under the cubic from a triangular area: the former is found using the function Integrate, and the latter is found directly from the coordinates using the functions Polygon and Area. Finally, we are required to find the value of a that minimises the area, which is found in one step with the function ArgMin.

What follows is the complete Mathematica code to answer this five-part question:

This solution requires little mathematical understanding beyond being able to make sense of the questions. In particular, the standard CAS approach of setting up integrals and differentiating is entirely circumvented, as is the transcription. In Methods CBE, the above input and output would be considered sufficient answers.

 

CREATED FUNCTIONS

We’ll now venture into the world of custom Mathematica functions, where programmers can really go to town. We’ll first look at the topic of functions and the features of their graphs. Mathematica does not have a built-in function to give all the desired features, so I created the function DetailPlot. To begin, I use a module to gather data about a function, including endpoints, axial intercepts, stationary points, inflection points and, if required, asymptotes. I then turn the module into an image to place over the graph.

Let’s fire this new weapon at Q2(c), Section B of 2016 Exam 2, which concerns the pictured quartic. We are given the equation of the graph and the point A, and we are told that the tangents at A and D are parallel. We are then required to find the point D and the length of AE.

And, here we are:

With very little input, DetailPlot has provided a rich graph, with every feature one might require within easy reach. The intersection points are ‘callouts’, which means that the points are labelled with their coordinates. In particular the coordinates of D and E have been revealed by DetailPlot, without any explicit calculation. We can then press forward and finish by finding the length of AE, a trivial calculation with the in-built function EuclideanDistance.

In the next example I demonstrate that a multi-stage question can still be trivialised by a single piece of pre-arranged code. In the multiple choice question MCQ10 from 2017 Exam 2, the function \boldsymbol{f(x) = 3\sin \left(2\left[x+\frac{\pi}{4}\right]\right)} undergoes the transformation \boldsymbol{T\left(\Big[\begin{smallmatrix}x \\ y\end{smallmatrix}\Big]\right) = \Big[\begin{smallmatrix}2 & 0 \\ 0&\frac13\end{smallmatrix}\Big]\Big[\begin{smallmatrix}x \\ y\end{smallmatrix}\Big]} and we are required to identify the resulting function. For such questions I created the function Transform, and then the in-built function FullSimplify polishes off the question:

My last example is on functional equations, for which I created two functions, FTest and RFTest. I will illustrate the use of the latter function. For MCQ11 on 2016 Exam 2, the equation f(x) – f(y) = (yx) f(xy) is given and it is required to determine which of the given functions satisfies the equation. Here is my entire solution:

 

CONCLUSION

It is impossible to have a proper sense of the power of Mathematica unless one is a programmer familiar with the package. This article presents just a few examples from the vast library of functions I created for Mathematical Methods and I found even more so for Specialist Mathematics. My libraries for both subjects barely scratch the surface of what is possible.

Creating such packages requires skill in both programming and mathematics, but the salient point is that any subsequent application of those programs by another student requires no comparable skill. The programs I have written may improve the performance of mathematically weaker students. Conversely, any student without access to such programs or, worse, is required to use handheld CAS instead of Mathematica, will be at a significant disadvantage.

This demonstrates the potential power of Mathematica to change the focus of VCE mathematics and, consequently, to debase its teaching and its assessment. True, the same issues had already arisen with the introduction of handheld CAS; clever teachers and cleverer students have always engaged in creating and sharing push-a-button CAS programs. Mathematica, however, has massively elevated the seriousness of these issues, all the more so since only a fraction of students have access to the platform.3

Technology, including Mathematica, calculators, spreadsheets and the many online programs, have tremendous potential to assist students with learning, understanding and applying mathematics. What is important for educators is to be careful that students are not using this technology to bypass learning and understanding mathematics.

1. All non-CBE students take the same tech-active exams and are considered in the same cohort for ATAR purposes. The Methods (CBE) exam appears to differ in only a superficial manner, and it appears that CBE students have not been considered a separate cohort since 2016.

2. The examination diagrams have been redrawn for greater clarity.

3. Although the Victorian Government offers Mathematica to all schools, to date many schools have not implemented it.

© Sai kumar Murali krishnan 2020

34 Replies to “Guest Post: Mathematica and the Potential Gaming of VCE”

  1. Firstly Sai, great article. You’ve achieved what I couldn’t with the old TI82 calculators back in the 1990s – so well done.

    A couple of curiosities (and don’t feel obliged to answer – they are just curiosities):

    How did all this start for you? Was it a teacher, parent, older student or purely your own inquiry that led you to the idea that VCE can be gamed in this way?
    Did your school give you the option of doing Methods CBE vs “normal” Methods or did every student at your school sit the CBE assessments?

    1. Hey RF, thank you for the feedback! As for why and how it all started, it’s a complex story involving programming on its own and mathematics as well. To give a background, this school was transitioning to CBE when I joined the school, and as a result, we were taught to use Mathematica as opposed to a CAS calculator. The following year (when I was year 11) was the final year where a CAS calculator could be used in Specialist and Methods and I did Methods 1 2 with Mathematica so in turn, our cohort was what I consider the first major cohort to do the CBE examination (additionally the first cohort at the school).

      I started putting the pieces together since my school offered a technology elective that taught people Python and I really enjoyed programming. At this time, I realized that Mathematica is a programming language of its own. Additionally, I had became really interested mathematics and ended up being able to create and find Mathematica code for Methods and Specialist Matematics. Over time, I spent a lot of time finding connections between programming and mathematics which gave me the ability to create functions that could automate certain tasks of its own. Additionally in 2018 I went to America for the Wolfram Summer Camp which was a fun experience to say the least. Among this, I often saw that many of my fellow students and friends were struggling to use Mathematica, and I thought to try making resources that simplify certain calculations, eventually building up and up like a snowball. However as much as the library seems like a competitive advantage, both my programming and mathematical skills need to mature more before something like this comes to anything that truly could trivialize an exam in its complete capacity. I’m in an odd position where I enjoy both the concepts of maths (since I am still doing maths at university) but also enjoy the challenge of trivializing these concepts.

      As for our school, every student in my cohort or beyond has to do mathematics with Mathematica. In the case of Methods, it is done via the Mathematica notebook whereas SM and Further are done using Mathematica as a CAS. In consequence, the SACs (for Methods) may or may not be a Mathematica notebook, depending on whether VCAA’s system decides to work (See: SAC of Roaming).

      I have a lot of things to say about the CBE (like lack of past exams (as a Mathematica notebook) until the end of the year, no information on scaling (unless you dig deep and do some guess work), terrible/no resources (looking at FuSE), no guidance for answering questions with Mathematica so we were gambling when we used functions and just a lack of support overall. Maybe I’m exaggerating (and if my fellow class-mates want, they can corroborate) but when we transitioned to CBE, the teachers were uncertain with using Mathematica and relevant code, and the students were even less confident in using Mathematica so someone had to step up in the end. A side remark is that from when I speculate the trial began (2013 was the official exam via Mathematica) that no resources have ever been developed in any form by former students. I still wonder why that is…I’m also curious RF since you mentioned that you have worked with Mathematica in the past, what sort of stuff have you done with Mathematica?

      My last remark is that Mathematica has a comprehensive machine learning library (and is a very robust language itself) and I have draft ideas of things I want to someday test for the satisfaction of it. Not that any kid at a VCE level could effectively use those libraries, but I have some interesting ideas, such as extracting the content of an exam notebook and throwing it into some analyzer. (Who knows what else you could do from there!) So far I’ve been able to extract all the text from the MCQ and it could be possible to answer really basic questions such as what’s the period of this function? More food for thought.

  2. Hello Sai,

    All in all, an acknowledgeably insightful article! Well done.

    As a teacher, I must confess that your great talent in combining coding and mathematics really astonished me.
    Besides, this is an opportunity for all of us teachers to refresh our teaching repertoire. We must immediately start learning our coding skills, otherwise getting out-dated!

    We must admit that this is also another shock to our existing curriculum – what is good mathematics? How do we assess mathematics in better measures with the existence of advanced tool – Mathematica, while giving all candidates a fair go?

    Anyway, I hope to see more of your contribution in this area, while informing more teacher and students of the great use of Mathematica. With more students, teachers and schools getting involved with CBE exams, I think your pioneering work will be revolutional and indispensible!

    Kind Regards
    P. N.

    1. P.N., your reaction to Sai’s article is in line with VCAA’s view, and I think this is exactly the wrong conclusion to draw. Sai can clearly speak for himself, but note the censored concluding paragraphs from Sai’s article, reproduced here.

    2. Hi P.N, thank you for the comment! Perhaps I’m too young at this stage to be able to understand the intricacies of the education system but I feel as though there’s a separate answer that doesn’t conform to VCAA’s plan. As for your question on a fair system, it’s incredibly difficult when you have CAS of different levels and of different software because Mathematica has so many capabilities it could potentially emulate a TI Nspire.

      I enjoy maths and programming equally but there are just as many people in programming who claim there is no need for maths in programming. I also know far too many people who believe that a CAS will solve their issues and that they have no need for knowing the theory. I would administer a (theoretical) bludgeoning to both groups as they are complete morons and I have just a sense that if the current system hasn’t already done so, then future maths programs will produce students in the latter group. Both groups are annoying to deal with…Although the first group is the form that produces Algorithmics papers without undertaking a real analysis unit where the O notation is studied or a former algorithms and data structures unit that covers the formal definitions of dominance.

  3. Sai,

    Hopefully your vinculum article prompts a response from dll as Game Theory shouldn’t be needed to excel at VCE. No doubt you will be enjoying data science at Monash given your programming background.

    Have you been able to obtain access to the source code of standard build functions on theWolfram platform through your studies?

    I have been trying to access the source code written in LUA on the Ti nspire CX looking on TI and GitHub with little success. To level the playing field an interface to allow the Tn nspire to import or translate Mathematica code would be one solution but given the low spec OS and clunky keyboard ,the calculator pressers would still be playing catch up

    Obviously other way for VCE to go is to scale Mathematica CBE separately from other CAS exams

    Steve R

    1. As is stands, VCE mathematics subjects are being turned into coding subjects. I predict that VCAA’s response to feedback will be to re-name each subject:

      Computational Specialist Mathematics

      Computational Mathematical Methods

      Computational Further Mathematics

      The Study designs will be amended with phrases such as “Experimental mathematics …” and the whole focus will shift to ‘exploration’ ….

      Re: Various comments such as “We must immediately start learning our coding skills, otherwise getting out-dated”, “the teachers were uncertain with using Mathematica and relevant code” etc.

      The first comment acknowledges that teaching mathematics will no longer be the primary focus of a VCE mathematics teacher – it will be all about teaching coding.

      The second comment acknowledges that Mathematica cannot be used to assist in routine calculations, it is to be used in clever ways to answer specific question types. Teachers must provide this code to students and train them how to use it for that question type. The focus will be on using code, not learning mathematics. At my school I saw no uncertainty in using Mathematica to do routine algebra, calculus etc. The uncertainty was in trouble shooting code used by students and using available code in clever ways to get clever output.

      The DuLL people at VCAA are either too dumb to know, or too arrogant to care, or both.

      The fact that the Mathematica solutions provided by VCAA “are clumsy, uninventive and calculator-mimicking, suggesting a limited understanding of Mathematica’s capabilities” (to quote from the unedited article – MAV demanded any constructive criticism of VCAA be deleted before publication) is very telling. You can also add “wrong in some cases” to the above description of those solutions (and after pointing out some wrong answers nearly 4 weeks ago the Examination Reports remain uncorrected).

      Mathematics at the VCE level is a dead man walking.

      1. JF, it is worse than VCE mathematics being turned into coding subjects. They are being turned into Google subjects.

        1. Re: Being turned into coding subjects

          I have a lot of my own to say about how coding is taught on its own and I shudder to imagine how shoddy the VCAA implementation is. Coding itself is something mathematical in that problem solving techniques can be abstracted to mathematical concepts (see any informatics olympiad problem) but unfortunately something like that will never be taught in consequence. There is no algorithmic thinking, just apply by the rule subjects. It should at least be kept separate in consequence.

      2. Hello J.F.:

        You shared another portion of your valuable pedagogical thoughts.
        However, I would like to clarify my previous point:

        “We must immediately start learning our coding skills, otherwise getting out-dated”

        Well, this is part of (my personal) teaching philosophy.

        It is never harmful for one to see himself as a humble learner of any mathematically relevant knowledge, be it teachers or students. The more characteristics of technologies we know about, the more likely we are to prepare our students and ourself to the unpredictable future. In fact I hardly see the contradiction between sharpening our professional knowledge and teaching students sufficient, rigorous maths.

        I agree with your big idea. We must see any technologies as facilitators in our maths classroom, not putting the priorities the other way around – mathematics (and exam questions) being the facilitator and time/mark-saver. This is the big thing behind. Otherwise, just as you suggested – the fallacy of our curriculum appears – with the phrase “computational” trivializing our mathematical value…

        Fully aware of the current pros/cons and issues brought by technologies, as long as we hold our integrity and do the right things with the kids, whether it be Mathematica, TI or CASIO, it will sooner or later be resolved. But it leaves another side question for us to ponder – who the next would possibly be the person leading these revolutionary changes in our curriculum (hopefully in a good direction…)

        Well, thanks for all your efforts in pushing them rectifying the error…though it seems a trivial matter for them. At least they recognized the “inclusion”, which is better than nothing.

        Mathematica itself is not clumsy at all. The Deterministic factor is who uses it in a Laggy and Lame way and who Dares to avoid many aspects of efficient usages and meaningful mathematics alongside this great Tool.

        1. Yes, the great Tool indeed, Laggy and Lame way and who Dares …. Very DroLL.

          But P, I have to disagree with your optimism.

          The environment Mathematica brings has no integrity, so there’s nothing to hold.

          And it won’t be resolved …. It will be NSolved and DSolved and NoNeedToThinkSolved …. There will be no fairy tale ending. The resolution will be that we get stuck with a bunch of Google subjects with mathematics nothing more than a pitiful background context at best for the coding.

          CAS in VCE is not a facilitator for mathematical thinking. It is a facile-ator. It is a spoiler, inhibitor, obstructor, blocker.

          And it will all be happening on the apocalyptic landscape of CBE (Crap Based Examination)

      3. Re: Solutions. A Report (2019 NHT Specialist Maths Exam 2 Q2(b)) has now been amended by the inclusion of a correct and useful comment. But the incorrect answer did not get changed. So now we have the absurd situation of a report that contains the wrong answer and contains a comment that contradicts the wrong answer.

        VCAA clearly got the work experience kid to amend this report – it is further evidence (as if any more was needed) that VCAA is too distracted with its agenda of turning all of VCE mathematics into a set Google subjects to actually give a damn about mathematical detail or standards.

        1. Update: I’ve just noticed that the Examination Report has been amended. Again. The wrong answer has finally been fixed. Hallelujah!

    2. dll is a very busy man with many more important things to do. He has no time to address genuine and important concerns raised by either teachers or students (I have been told this directly by VCAA). He is far too busy to even ensure that errors in answers in Examination Reports get fixed.

      So give the poor important over-worked guy a break, Steve.

      1. In fact dll is sooo busy with sooo many important things that he doesn’t even have oversight of the marking schemes used for mathematics exams (so we have a mathematics manager with no responsibility for how VCE mathematics exams are marked. So incorrect VCAA marking schemes can be used to incorrectly mark exams such as the NHT 2019 Specialist Maths Exam 2 and the VCAA mathematics manager carries no responsibility for this – NB: I’m not saying this did happen, just that it can happen. We’d never know for sure if it did happen of course, unless a FOI request was successful ….)

        And Steve, you hope this poor important over-worked bloke will have time and/or inclination to respond to the many legitimate and important questions raised in this Vinculum article (and raised by many concerned maths teachers at some time or another). Shame on you, Steve, shame. Have some compassion for the poor miserable wretch. Just because he’s the VCAA mathematics manager doesn’t mean he should waste his valuable time addressing legitimate mathematics concerns raised by us pissant mathematics teachers. Let him crack on with more important things like turning all the VCE maths subjects into Google subjects.

    3. Steve, I’ve requested the grade data for Methods CBE, and will post on it if and when I get it. I suspect that in fact Mathematica students have not performed notably better so far, but I can’t see any way that any parity will continue. As for what to do about it, the answer seems obvious to me. The fact that the same conclusion is not obvious to others here is interesting.

      1. Marti,

        You did very well to get Sai’s edited article published in Vinculum

        So I’m guessing your obvious solutions may be either to remove programmable machines from the non CBE curriculum completely or grade/scale them separately if there is only one exam .

        Alternatively perhaps VCE administrators could consider clearing memory from CAS machines and/or use a non CAS version with a standard build as they do in IB so that everyone candidate has a similar theoretical chance .

        Steve R

        BTW In all chess tournaments prior to the pandemic all mobile phones etc are removed from the players during game time as the ELO grading level of the numerous brute force and neural network AI engines far exceeds the strongest human player.

        1. Thanks, Steve. I wasn’t irrelevant, but the editor Roger Walter deserves the majority of the credit. And, to be fair and as I will document in a later post, the MAV did demonstrate that it contains at least a pocket of sense.

          And, yes, the obvious solution is to simply get rid of all these devices. If VCAA wants to teaching coding, or Googling, then let them. But school mathematics doesn’t need these devices and it is quite evidently poisoned by them. In fact, on two separate occasions, a heads of maths group sent a letter to VCAA voicing concerns about CAS in VCE, and both times VCAA effectively told them to bugger off. The VCAA response was disgraceful, and the heads’ meekly accepting this response was far, far worse.

    4. Hi Steve, data science was a really good option for me since it let me maintain a duality between mathematics and computer science and let’s me study the topics I want. Unfortunately, I’ve never been able to obtain the source code as Wolfram is a big company and their code is closed off. However an interesting story is that when I was in America, employers at Wolfram showed how they built code and it was extremely complex to say the least. Mathematica code is something that happens to grow in terms of messiness real quick…

  4. On the one hand, this is a great credit to Sai and students like Sai who display great ingenuity and application in spite of the VCE system.

    On the other hand, it just makes me feel like I’m pissing up against the wall.

    1. SRK, it’s worse than that. A major point of Sai’s article is that one doesn’t need to be a good programmer: one simply needs to know, or to hire, a good programmer.

  5. The article by Sai was very interesting – and well written too. I look forward to seeing its impact.

    I remember one class (not a test) where I allowed students to use a calculator and one student simply asked Siri – and it worked.

    In another class, which was a test, a couple of students forgot to bring their calculators and asked if they could use their phones which have a calculator. Naively I said OK – what I did not realise was that they then had access to the internet. Once bitten etc

  6. How on earth was it ever considered a good idea to allow Mathematica to be used in a mathematics exam? Am I missing something here? Calculators are already bad enough as it is.

    I feel as though I’m in the Twilight Zone. Why is it not outrageously obvious to everyone, teachers, curriculum designers, just everyone, that a tool used in research mathematics to trivialise standard calculations (and more) can turn any high-school level mathematics exam (and most university-level ones) into a coding exercise?

    It’s like athletics training in virtual reality.

    1. @Marty can you please delete one of these? I tried to delete it when I posted it, I thought I succeeded, but apparently not. Thanks!

  7. Glen, it’s worse that. As I replied elsewhere, Sai shows that VCE maths is sliding below “coding exercise”, to be a Google exercise.

    As for the Twilight Zone, no you’re not in it, you’re just watching it. But think of us in it. Victorian school mathematics, starting with primary school, is batshit insane. It is all driven by technophillic cretins.

    1. Yes, well, it is generally true that most “coding” can be reduced to a Google exercise! I’m assuming that students don’t have internet access in their mathematics exams… right? RIGHT?!?!

      Preventing this craziness from crossing into NSW or anywhere else can only be guaranteed by completely excising it from VIC. Ugh I can still hardly believe it.

      How do teachers put up with this? I feel like going on a protest march.

      1. Yes, Glen, that’s the model. We work on eradication in Victoria and, until we’re successful, keep the borders closed.

  8. Glen, I think most teachers would agree with you.

    But VCAA has an agenda that it is pushing. Unless teachers can get organised and start pushing back in a practical way, this DuLL agenda will get bullied through to complete implementation. The thin edge of the wedge was schools agreeing to participate in the Pile-of-Shit Program. The decision makers at those schools have a lot to answer for.

  9. Sai ,

    Thanks I wasn’t expecting a yes! as it would be interesting to know which numerical iterative or approximate methods are behind nsolve ,solve, desolve … for intractable functions in VCE CAS exams

    Regards
    Steve R

    BTW
    For those programmers who don’t have access to the Mathematica platform there are alternative options available

    Scilab free version of Matlab ( another propriety CAS system)
    https://www.scilab.org/download/6.1.0

    Tableau Public – cut down version of one propriety visual database presentation tool
    https://public.tableau.com/s/

    R
    https://www.r-project.org/

    Most of the propriety analytical database software that I have used has R driving its statistical analysis
    These objects can be manipulated directly by Python ,Java or C++ programmes

  10. A question: Could the work that Sai has done with Mathematica be done with a CAS calculator?

    A comment: In my limited experience, I have noticed that the students who are better at mathematics are also better at using their CAS calculators. Hence the calculator advantage is not spread uniformly across all students. Perhaps it increases inequality. (Must be a thesis in all this.)

    1. Very good question. Sai is the best person to answer, and others here are much better placed than me, but in brief I think the answer is yes and no.

      Definitely handheld CAS machines can be programmed, and plenty of teachers and students share sophisticated programs. This gives a significant advantage to those who have access to such programs. But I don’t think it is nearly as easy, to create or to apply, or can be nearly as extensive as what Sai is doing with Mathematica.

      You make a good point, that handheld CAS calculators are most powerful in the hands of good students. I think that is also true of a lot of Sai’s programs, but definitely not all. Applying RFTest doesn’t require a whole lot of brains. In general, a significant point of powerful, tailor-made programs is that they can trivialise computation, even if the user knows little of what they’re doing.

Leave a Reply

Your email address will not be published.


The maximum upload file size: 128 MB.
You can upload: image, audio, video, document, spreadsheet, interactive, text, archive, code, other.
Links to YouTube, Facebook, Twitter and other services inserted in the comment text will be automatically embedded.

This site uses Akismet to reduce spam. Learn how your comment data is processed.