One type of educational horror that we haven’t yet written about are SACs, those internal assignmenty-examinationy things that make every second week of Year 12 studies a living hell. It is a tricky topic since SACs are school-based, often teacher-specific, and our primary goal is to attack inept authority. In that regard, schools and beleaguered teachers are in a weird middle ground, part victim and part villain, and they already have plenty of critics. Nonetheless, SACs are the sea in which students and teachers swim (or sink), and mathematics SACs are typically appalling; the overwhelming majority of mathematics SACs that we see are pointless, anti-mathematical, error-strewn blivits. So, something has to be written about such SACs of shit. And, we have a plan.
Our hand has been forced a little, however, by an email we received from a VCE student. The student is taking Mathematical Methods CBE, the trial version of Methods that uses Mathematica instead of CAS, and the student wrote about a recent Mathematica-based SAC at their school. We then asked a teacher at the school about the SAC, and they confirmed the students’ report.
This then is the students’ story, exactly as written to me.
I’m not sure of how many other students you know that are doing CBE methods but my sac today served pretty well to show how awful things can go, so a new perspective is always welcome right?
Starting from the top, we have school-provided laptops with Mathematica preinstalled. So we go in, and we have to utilise this thing called a palette which takes control of Mathematica (I have many complaints against the palette) and downloads the SAC from some remote server. No problems here right? Well, I’d imagine 200 people simultaneously downloading an item from a server would MAYBE just MAYBE cause some congestion in the network. Hell breaks loose here, a class has their file downloaded and enters reading time while the other 4 or so classes are in utter chaos. The downloader is failing over and over while also saying it succeeded. This goes on for a good half an hour before the teachers collectively decided that the sac would be rescheduled to Thursday. The class that began reading? Oh, they just stop. God knows what they could have done, taken photos of the sac with a snipping tool, copied the files over, the possibilities are endless. This whole thing is almost appalling yet terrifying because this is what I’ll have to do at the end of the year. I’ve got a load of other concerns, among the unending variety of methods to do a question with Mathematica and how an assessors report would assign marks, to which our official VCAA quiz provided pissant “solutions” that were often wrong.
Anyway, my slightly irritated take on the abhorrent state of the CBE system, which I thought may be interesting to you.
To that, the teacher at the school added the following:
[The former Head of mathematics at the school] made a deal with the devil and agreed to the school doing Methods CBE. I don’t feel the consultation process valued the feedback from myself and other teachers – it was always going to happen despite the misgivings of other teachers. I can’t help but think that my feelings on the issue is the reason I’m not teaching Maths Methods this year (for the first time in 9 years). There are many problems with this deal – it makes my blood boil. The current head of maths is a very decent guy and has done a fantastic job dealing with the mess he inherited.
I heard that the SAC was a disaster and actually saw events unfolding from afar – like watching a car crash in slow motion. Blind Freddy could have seen what was going to happen. As I left school, I saw the VCE coordinator and the current Head of Maths running around grim faced.
All of the student’s concerns are legitimate. Furthermore, the SAC was meant to run until 4.45 pm, so many students will have made alternative and inconvenient arrangements to accommodate this and now they have to do it all over again. Not to mention what it’s done to the stress levels of many students. Not to mention the time and resources that had to be expended re-writing the SAC. At every stage VCAA have washed their hands of CBE problems and left the school to do its dirty work, using the students as the guinea pigs.
1. The palette provided by VCAA had a bug.
2. The VCAA server failed. VCAA are trying to blame the school for both errors and no apology has been given. Re: The server fail. VCAA said that the school should have downloaded from the server prior to the SAC starting (which is not practical). VCAA are saying everything worked fine at the other CBE schools (which all have small student cohorts as opposed to our school’s cohort of over 200, which makes a big difference).
That’s it. Our own point of view is that SACs are all but guaranteed to be awful and Mathematica in the classroom is all but guaranteed to be awful. Here, however, those predictable awfulnesses are beside the point. The point here is VCAA’s Trumplike level of incompetence combined with VCAA’s Trumplike unwillingness to accept responsibility.
10 Replies to “VCAA’s SAC of Roaming”
Mathematica is used at VCE level only because David Leigh-Lancaster is a Mathematica nut, and managed some deal between the VCAA and Wolfram Inc. It’s the work of the devil, as Mathematica, for all its strengths, is wildly unsuited to school mathematics. (I happen to think it’s wildly unsuited to human brains, but maybe that’s just me). And this tale of woe shows the vital importance of good infrastructure, which is never taken into account. People just assume it will “just work”. What a dog’s breakfast. VCAA should be ashamed of themselves, but of course it’s never their fault.
Thanks, AI. I think it has to be argued, but I am also of the opinion that Mathematica is unsuitable in schools. What can’t be argued against is that VCAA is attempting to ram it in, and is doing so incompetently.
I would add that it’s not Mathematica per se that’s the issue here. It’s having to download an assessment from a server, having to back-up work etc. That’s what CBE is – a Computer Based Examination. NAPLAN has consistently provided solid evidence that students get screwed over with computer-based assessment.
On a related note: In the Specialist Maths Exam 2 this year (and I assume last year, maybe the year before, and next year) students using Mathematica are competing against students who are using a CAS-calculator. I wonder whether VCAA have any idea how relatively trivial many of their exam questions are for the students using Mathematica. For students using calculators, it will be like competing against athletes on steroids (except the steroids are legal in this case). I don’t think the VCAA exam setting panel have the first clue how to write fair questions that are technology neutral.
A simple example: Multiple Choice Question Q5 from the 2019 VCAA NHT Specialist Maths Exam 2. Can be done in 30 seconds using two simple lines Mathematica code – one line for getting the radius of the circle and one line for getting the centre (with no clever pre-defined commands required, by the way).
This is not mathematics, it’s computational mathematics at best. The best coders will get the best results, all else being equal. There’s a big difference and VCAA should be honest enough to admit it up-front.
Thanks, JF. To the extent that fairness is on VCAA’s radar, I think they are quite happy to tilt the scales in the direction of Mathematica.
Thank God I did my matriculation back in 1962. Things weren’t perfect, but they worked pretty well. I feel sorry for today’s students having to put up with such a shitload of utter crap. Do they still call it education?
Yes, they do. That doesn’t make it true.
A key concern brought up by the student is the marking scheme (or lack thereof) for the CBE exams. While I lean towards the mindset of providing an appropriate number of statements and a correct answer to receive all the required marks, Mathematica creates an almost uncountable amount of ways to skin a question as John Friend has mentioned. This raises the problem for the end of year CBE exam, what constitutes a working mark? In my research, I have found no assessors reports or instruction defining any of this and I highly doubt schools participating would have received instruction on what to do as well. The opposite also is true; why not just define a function that could potentially do all the work required in a question? It quite literally could be perceived as “legal steroids” and demolishes the whole concept of mathematical thinking where the best coders prevail. This description of a desolate wasteland created by David Leigh-Lancaster is further heightened by the unorganized mess that is the fuse education site which intends to provide tutorials on using Mathematica for various topics. Not only are the notebooks weirdly organized, but a lot of the code is also inefficient or outright not required for the subject. (Side note: In my findings on the fuse site I found a link to “Mathematics Level 8 using Mathematica” available here: https://fuse.education.vic.gov.au/ResourcePackage/LandingPage?ObjectId=28231446-8dfa-46ca-bcb1-927a4bb9508a&SearchScope=All. I could not even fathom why year eight students would even need to touch Mathematica for anything in their studies).
MK, you have totally nailed a really significant issue. Mathematica is so powerful and trivialises so many questions that in a question worth more than 1 mark, many students will undoubtedly be wondering how to show ‘appropriate’ working. And will undoubtedly get penalised when they don’t.
And you’re right, there’s been no advice from VCAA about this. In particular, none of the Mathematica solutions to the exams provided by VCAA shed any light on this.
VCAA will most likely blame teachers rather than accept and acknowledge their large share of responsibility – already I can see their chastisement in the 2019 Examiners Reports.
Further, perhaps to Knucklehead’s point above, as a teacher who once taught Mathematics using Mathematica, a long time ago in a universe far, far away… three thoughts kept recurring:
Graph sketching. The time required to do this in Mathematica is very different to the time required to do this by hand (even with the assistance of a calculator) and students are not being marked on their ability to sketch a graph any more as draw using software.
(relatively minor point) “show that” type questions which I already believe are annoyingly written in VCE Mathematics and more annoyingly graded and reported upon are surely very different when writing answers in the Mathematica interface.
VCAA has stated in different forums that students are not meant to write calculator syntax in their answers. Is Mathematica code considered “working out” and if so, does this not completely contradict their previous claim?
Thanks to MK and JF and Number 8. Of course the lack of clarity and consistency and any decent attempt at fairness are all major issues the Mathematica trials. I haven’t looked carefully at this, since my own preference is to nuke Mathematica and CAS and Desmos, and all the other technocrap gimmicks killing students’ and teachers’ thinking. But non-nuke-inclined teachers have every right to be pissed.