Same as usual, except a little different. Following the Bennett review,*
The VCAA is committed to publishing: the VCE Mathematics examination papers within 5 days of a paper being sat.
So, the secret exam business will no longer be so secret.
Anyway, on the with the General exam. Go for it.
*) See VCAA’s Examination Improvement page, specifically VCAA’s bulletin on exam/report publishing, and our post on it all.
UPDATE (04/11/24)
VCAA has now published the exam, here. (Thanks, Ron. Link fixed. 06/12/24 Link fixed again.)
UPDATE (06/12/24)
And the marking guide is here.
Visited 127 times, 1 visit(s) today
According to the replies to https://www.reddit.com/r/vce/comments/1ggxskn/comment/lutb5yo/, apparently the exam gave some students an advantage if they’d manually installed something on their CAS
Isn’t that always the case?
True 🙁
Hungarian algorithm question. There are a few different pre-written programs for this, but I note that not all schools seem to make students aware of these and maybe not all teachers know that such things can be readily downloaded.
Apart from that, I thought it a decent exam. Better than the last couple of years anyway.
If I had to guess, Q9, Q16 and Q40 will not be done well by a lot of students but this is because they are written to require a bit more thinking.
I couldn’t find any misleading questions in this paper either. Not saying they weren’t there, just that I didn’t find any.
Thanks, RF. CAS poisons everything.
Is there some reason why this instance of Stupid CAS Tricks is more notable?
One factor is that you either downloaded a program that does it completely for you (possibly trivialising the question) or you do it by hand. This is a large advantage over those who did not know about it or whose CAS model does not have a widely available program to download.
Tungsten, are you possibly implying that CAS is a complete disaster, making mountainously unlevel the playing field and utterly trivialising the learning and the examining, and that the sooner we put all the machines on a barge, along with David and Kaye, the better off we’ll all be? Not meaning to put words into your mouth or anything, just checking.
The User Defined Functions (UDFs) and Widgets (whatever they are) have been fair game on CAS papers forever. MAV even runs sessions for teachers on how to best use them.
In Methods and Specialist this means Paper 2 and a lot of the time they are more useful for checking an answer than anything else.
However, in a paper where every mark is an answer mark and every minute saved is valuable… yes, I think the difference between calculators that can automate standard processes and those that cannot is an unfair advantage.
I do not know which approved calculators can and cannot be programmed to do a Hungarian Algorithm nor how commonly known such a file is for students to download and install (any student with internet and a TI NSpire CX CAS – which is the dominant calculator at the moment – could have installed the file mentioned above). Does that mean it is fair?
I genuinely do not know but think even the possibility is too much in a high-stakes exam.
Only the Texas (ti) from what I can see. I don’t think there’s a e activity written for the Casio yet. I’ve been trying to find the program code for it.
I do find it frustrating as I have never looked for cas tools for Hungarian as it’s an easy process and students often do it quite well. I’m assuming VCAA had the ‘vision’ this would be more challenging for students and were unaware this could be done with program on the ti.
I don’t think they really check technology beyond the basics, and they definitely check with emulators which have faster processors than the handhelds.
VCAA just love fobbing it off saying ‘students need to know multiple strategies to approach questions in the instance their CAS does not provide an appropriate answer
Is it “fair game” for Methods students to generate their working out using LazyMath (https://lazymath.github.io/)? Considering the fact that requiring working out would typically separate students who solve the question themselves from students who put the question into their CAS
Jay, I’m not sure I understand this. Can you explain?
Essentially, many LazyMath UDFs provide suggested lines of working in their output to help students secure working marks, allowing students with this software downloaded to answer questions without any actual working or understanding. One of the many reasons why each CAS should have to be reset before the exam.
Jesus H Christ. This is turning into a medieval religious ritual, the Monty Python monks banging themselves on their heads with planks. It is all meaningless.
Aside from the total rewriting of the curriculum and study designs for each maths subject, do you think that removing CAS from VCE could be a step towards reform? Most (non-maths) subjects have only a single exam and a 2-hour, 80-mark non-calculator exam could ensure that students know how to actually do maths and not just which buttons to press on their handheld computer. I only have experience in Methods and Specialist so I’m not sure if this is applicable to General, but the obvious and unfair advantage provided by UDFs continually frustrates me.
Parkie, that’s the Dorothy Dixer to end all Dorothy Dixers. But to answer, yes of course removing CAS from VCE would be a step, and a massive leap, forward.
This is utter madness. And the only reason teachers tolerate and/or approve of this madness is because they have been indoctrinated with the madness and, more and more each year, the majority of teachers were themselves taught with this madness.
It is a cult. It has to end.
Frankly, I’ve always been surprised that there doesn’t seem to be an overt market in CAS code. Unless I’m just not hanging out in the right places.
I’ve written a number of custom functions over the years and happily given them to students. I say happily not because I think CAS is a good thing (I think it’s vile and, as Marty says, poison) but happily as in without guilt or shame. If VCAA are going to make the exams a hideous exercise in CAS bashing, and if, because of that, proficiency with the CAS (rather than proficiency with mathematics) begins to play a significant role in a student’s mark for the exam, then I will supply my students with that advantage where I can.
I hate it. I hate it. I hate it.
Yea I totally agree. Every year there’s a question that the Casio handheld can’t do (or can’t do as easily as the TI) especially in mm or sp.
I’m pretty sure we are the only state that uses the CAS too aren’t we?
I don’t have a lot of experience making programs on the CAS, I think I definitely need to look into it.
As of last year, Western Australia and Tasmania also use CAS and South Australia and Queensland use graphics calculators.
The hicks leading the hicks.
I don’t think the CAS weight in Tasmania or WA is nearly as high. No one has drunk (had David and Kaye force down their throats) as much Kool Aid as Victoria.
Ron you are right to do what you are doing (and right to hate it). 97% of a VCE teacher’s job is to obtain (legally) the best possible marks for his students, 2% is trying to teach them some mathematics, and 1% is complaining about and to VCAA when appropriate.
There has been a bit of a market. Tutors and tutoring companies sometimes advertise UDFs for their students and there have been a few people selling them. Recently a large tutoring company bought out the major seller of TI UDFs to provide their own students exclusively. Now there are freely available options, which probably meet or beat paid products, for TI (such as Lazy Math linked above), Casio and Mathematica (including Sai’s).
Where do I find the paper?
It should be online in a few days (if you believe VCAA).
Otherwise, find someone who works in a school that had students sitting the exam. Teachers are allowed a copy of the paper 30 minutes after the end of reading time.
I found it here: https://www.youtube.com/watch?v=NmVEQH9c6tw
Note the title of the post.
There are copies in the Victorian maths teachers’ Facebook group (keeping it unpublished for so long was just crazy in the modern world).
That’s interesting. If I were a teacher I wouldn’t be posting the exam now,
Why? Copyright?
Of course. VCAA’s taking weeks to post exams etc is obviously nuts, but that doesn’t make it legal to post exams beyond fair use.
True but it would be a very brave VCAA to do anything about it. And the reason teachers are sharing it is to discuss it and post their solutions. Seems pretty fair use to me (though don’t know legal definition).
It is most definitely not fair use. It may be reasonable use, although I don’t even believe that, but the law is clear. VCAA has ordered stuff down in the past.
Really? Like JJ said, that’s extremely brave of them, given those teachers are doing the job they should have been doing (at least prior to this year) of making the exams available in a timely manner
When did this happen, by the way? I can’t see anything about it from a quick Google search
I don’t know if VCAA has contacted reddit etc in the past, but at least one YouTuber got a slap across the wrist in a previous year.
In past years I would have more sympathy with teachers/anyone posting the exams, because VCAA was self-servingly slow. This year, presumably teachers are not in the main aware of VCAA’s commitment to post early.
In general, I think it is reasonable to give authorities some space on such matters, as long as the authority is also being reasonable.
I think that a private Facebook group is less of an issue for VCAA as it isn’t easily searchable like Reddit or Youtube – and not as potentially commercial.
Yeah it’s not Fair Use/Dealing, but it almost fits within the (very reasonable) VCAA Terms of Use:
As a point of comparison, NESA is much stricter on copyright than VCAA, but the MANSW Facebook group still shares copies of the exams quicker than NESA can publish.
It has now been published
Click to access GenralMaths1-2024-w.pdf
For amusement, examine the url closely
Thanks, Ron, and I couldn’t give a stuff about the URL.
This is better
https://www.vcaa.vic.edu.au/assessment/vce-assessment/past-examinations/Pages/GeneralMathematics.aspx
Hilarious. They spotted the spelling mistake and changed the URL.
Ron, this the smallest of beer. Knock it off.
Student who did a lot of previous papers found this one challenging. I didn’t think it was too bad.
The Hungarian CAS thing does annoy me a bit. It’s an easy process and students don’t generally struggle with it. Didn’t even cross my mind someone would feel the need to make a program for it. And it looks like it’s only on the TI.
There are some approaches to q 39 by doing the algorithm, add Edgar, you add a 5th job, give all members very high scores for this, do the algorithm and you get 39.
Q40 if you map out the network using numbers you can quickly see key features and how they link in with one another. Thus you can rule out c and d, a has 2 dummy links so that can also be ruled out.
Looking forward to seeing exam 2, MM and SM next week.
I haven’t seen the paper yet, but I mentioned in a comment further up the page that I’ve written CAS programs and given them to students. Your Hungarian algorithm comment (that it’s an easy process which students don’t seem to have trouble with) stood out to me as a salient example.
Most of the programs I’ve written don’t do anything astounding, but what they do do is take a time-consuming, error-prone process, and reduce it to a few keypresses. And where minutes equate to marks, the student who has at their disposal a tool to enable a calculation which would take minutes by hand to be done in mere seconds has an advantage over one who does not.
I’m not disagreeing with you, Ron. All I’m saying is I’ve never even thought of looking into making code for Hungarian due to not many students finding it difficult but I guess I haven’t considered that for some students it can be quite time consuming.
I am still on the hunt for an e activity for Hungarian for the Casio for next year, or even the code for the ti. If anyone finds any, let me know
If it helps, a C# (not entirely sure what that is) version of the algorithm was apparently adapted in 2017 for TI by a programmer named Rolf Putter.
If you can read the code and know how to program in the Casio environment (neither of which I can help with) then you may be in luck…
“Q40 if you map out the network using numbers you can quickly see key features and how they link in with one another. Thus you can rule out c and d, a has 2 dummy links so that can also be ruled out.”
I think A, C, D would each require at least one dummy activity.
Oh you’re right! I have already ruled c and d out bc of connections. So a c d could have been easily and quickly ruled out bc of a dummy link.
Good pick up.
How many students, realistically, do you think would recognize the dummy activities quickly enough to answer this question by eliminating wrong options? Genuine question, I have not taught this subject since 2004…
I tried option A – saw it couldn’t work, tried option B and it seemed to work. The idea of searching for dummies didn’t enter into my thinking.
(I’m sure there is a joke in there at my expense…)
Hard to speculate, but I don’t think it’s outlandish that of the students that made a reasonable attempt at the question – which is still likely to be a minority – a decent proportion considered dummies. This is simply because when you check if the network could represent table A, there’s a fair chance you’ll notice that it can’t because a dummy activity is required from L to N. Then it’s natural to check if the other tables would require a dummy activity.
OK… finished the exam and understand the concerns about Q39.
Yes it is possible (even easy) without a special CAS program but it does take time. More than the par time per mark you would expect given the number of marks and number of minutes.
Is it that much quicker with the special coded program? I’m not sure yet – need to run it a couple of times. For a student with no practice using the CAS feature I’m not sure it would have been a huge advantage. That does not make it right though.
I had never heard of the CAS program until some of my students brought it to my attention after the exam. I have done a bit of research and you still need to remove a worker at a time and rerun the algorithm on the CAS and then note the lowest time. It does calculate min allocation in seconds so would definitely cut down the time for some students but students who just added the worker and the dummy job could have done it in the same amount of time, for sure.
I have had more tech savvy students write their own e activities before which is think is actually pretty awesome as they have had to understand steps involved to write their own code.
Regardless, I guess GM is never really about the method anyway, it seems more about getting the answer and interpreting it.
Is anyone else puzzled by the mathematics performed on an ordinal categorical variable in questions 9 and 10. What does the mean of the Summer Olympic Games numbers indicate – let alone the standard deviation?
Hi, VJ. The GM teachers will better be able to comment, but this seems to be an instance of a recurring confusion/disagreement within GM (with VCAA not being consistent on the issue). On top of that, the 2024 question seems to be very badly worded.
The implicit, underlying issue is: what kind of variable is “year”? I don’t think there is a clear answer, and I don’t think VCAA has a consistent answer. It seems better to ask how “year” is used in a given context, and it certainly can be used as a numerical variable, as a measure of time, in the appropriate context.
Ignoring the extraordinarily clumsy phrasing, “the number of the Summer Olympic games” appears, as you suggest, to be an ordinal thing. However, if we accept that Olympic games occur at 4-year intervals, then it is reasonable to consider that “Olympic game number” to be a measurement of time, with a 4-year unit.
I think that’s the meaning of the scenario for Q9 and Q10. But, again, the wording is pretty shocking. And, how kosher the whole scenario is for GM, others would have to say.
I would agree Marty, except the 5th and 6th summer games were 8 years apart and the 10th and 11th games were 12 years apart. The question was about game numbers, shown here as 5, 6, 11 and 12.
OK, I see. Thanks.
It’s not great, but I’m not that fussed. The lousy wording concerns me more than the pretence/assumption of equal spacing. Given that assumption, the question is intrinsically fine (modulo wording), even if it is extrinsically wrong. But definitely a little quirked, and unnecessarily so.
Dunno. However some years ago I did an exercise is trying to predict the number of medals that Australian would win at the Olympic Games from the number of athletes that we sent. This appears to me to be a more interesting problem. And the fit was surprisingly good.
Thanks to everyone who has been commenting. I’ve been flat out, and in any case these issues are largely outside of my world, except for noting that we have yet more evidence that CAS poisons everything. But i’ve been reading the comments and have been very interested; I’ll provide some replies, and probably ask some questions, tomorrow.
A number of people have commented about question 39 possibly conferring an unfair advantage on students with the Hungarian Algorithm program on their TI.
I’d like to query another point in the question, namely whether the answer should be B (2 hours) or D (4 hours) due to a possible ambiguity.
The wording includes “When a new allocation is made and Edgar takes over one of the tasks, …” and I am wondering if this is ambiguous, with one of the possible meanings being that new allocation is identical to the original allocation, except that Edgar is now doing one of the tasks and one of the original workers is sitting it out. (The other interpretation, which is being assumed in most places on the internet, is that the new allocation is made which must involve Edgar taking on a task, but the other workers could be reallocated too.)
I can see a couple of reasons in favour of this interpretation being at least an acceptable interpretation, although I am not trying to argue that this is the only interpretation (I am merely trying to argue that it is ambiguous.)
(1) The phrase “take over” makes it sound as though we are working from some basis. For instance, if I told you that in a particular school, the principal is going to take over a class, you would assume that one class would be changed to have the principal as their new teacher, and the other classes would be left alone. Admittedly, it would be odd to say that there is going to be a “new allocation of classes” before saying that the principal will “take over a class”, but if an announcement came out saying “There is going to be a new timetable and the principal is taking over a class”, you might be surprised to learn that not only is the principal going to take a class, but actually all the classes are going to be shuffled between teachers.
(2) A pragmatic concern I have is that the Hungarian Algorithm is used to solve the problem of assigning n tasks to n workers. Students are expected to first do this for 4 tasks to 4 workers, but then (if the currently preferred interpretation is correct) they are expected then to optimally assign 4 tasks to 5 workers, which does not correspond to a square matrix.
This could be done by inspection / CAS, but the actual algorithm is intended for n × n matrices, not arbitrary rectangular matrices. There are ways around this, for instance, you could expect students to solve four separate 4 × 4 versions of the problem (corresponding to each of the four existing workers not being used) or you could solve a 5 × 5 version of the problem, in which there is a “Dummy” task of 0 hours.
In either case, it seems unreasonable to expect students to complete the Hungarian Algorithm at least twice in order to solve a 1-mark question, especially given that the second time they solve it, they would need to perform some sort of transformation in order to use the algorithm.
~~
I think one reason that people are not talking much about this ambiguity is embarrassment.
I was a little embarrassed that when I originally did the question I got 43 hours, and then just looked for a task that Edgar was to take over (to arrive at an improved time of 41 hours) yielding the answer B.
When it is pointed out that actually there is an even better allocation once Edgar is available (taking 39 hours, so answer D would be correct), it is natural to feel a little foolish for not having considered it, and forget that it could have been the phrase “take over” that leads one down the path to the answer of B.
I agree, I read it a few times trying to gauge if I was swapping Edgar in for someone or reassigning everything with him in the mix.
The whole ‘feel’ of the papers seemed to different this year. Similar style of writing to the northern hemisphere papers this year.
I expect to see the same level of ambiguity in MM too unfortunately.
Hi, B. I’ll look at Q39. Can you suggest other questions, on either GM1 or GM2, which you regard as ambiguous? Less critically, can you point to examples of how the “feel” is different?
I’ll note that MM exams (and SM exams but a little less) have a history of being written atrociously. So any new “feel” might be an improvement. But, I have no opinion on the GM writing, except if and when there is an obvious or argued stuff-up.
I think the word ‘ambiguous’ in reference to the whole paper was a little too harsh on my behalf, q39 was the only one (so far- I haven’t really done it all myself yet) I really had to read and read again to make sure I was understanding what was required.
It still do think the paper does a different vibe to the writing, I am not sure if that is because there is a different writing team or the edits from Monash? I also feel as though the MM papers this year seemed to be written a little differently than the most recent ones. It could be in my head. Who knows.
Thanks, B. As I wrote, I don’t think Q39 is ambiguous.
Thanks very much, Bryn, and everyone. I avoid GM whenever possible, but I guess I’ll have to look at the question. Flat out, but will try soon.
Given it’s pretty widely accepted that question 39 gave a strong advantage to students who had CAS programs installed (and hence didn’t serve it’s intended purpose to test students’ mathematical abilities and knowledge of the General Maths study design), and given General Maths is known to be an extremely competitive subject where a single mark makes a large difference, do you think it would be reasonable if VCAA were to invalidate that question? I highly doubt they will, but I’m just curious what your thoughts on that are?
(Also, one time is a mistake, but I really really hope they never do a one mark Hungarian Algorithm question again)
Not really one for me to judge, but I’d doubt if they would invalidate the question and, based upon what people have written, I doubt that they should.
What VCAA *should* do is invalidate David and Kaye, and this entire idiotic game. CAS has to go, along with all the snake oilers selling it.
To be honest, the CAS is actually a really useful tool. It definitely doesn’t belong in end-of-year exams, but otherwise it’s extremely useful for stuff like quickly simplifying a function, easily identifying mistakes in working out, and verifying what domain an identity is valid for
Bullshit, with a capital Bullshit.
Thanks to everyone who commented on Q39, the Hungarian question. I’ve finally had the chance to take a look at the question and to read people’s comments. This is not my world, but here are my thoughts (which should be taken with at least a few grains of salt).
*) Bryn’s pondering on possible ambiguity is very interesting, but to my mind there is only one natural interpretation: the tasks are assigned via Hungarian, and then Edgar is simply brought in to “take over” one of the tasks (with all other assignments remaining the same).
That is, I agree with Bryn’s original interpretation, giving a final answer of B (saving 2 hours), but I would put it more strongly. I don’t have all that much sympathy with the alternative interpretation, that all workers may be reassigned. I think you have to work quite a bit to read the question the second way, combined with an autopilot approach that “this is a Hungarian question” and so “I must have to do Hungarian again”.
It also seems clear to me, for the too-much-work reasons that Bryn outlines, that VCAA intended the “just bring Edgar in” interpretation.
*) Having said that, the question is not well worded, and there is at least a case that the question is sufficiently ambiguous to declare it so, and so consider the alternative answer, D, also correct. I don’t think I’d agree with such a decision, but I wouldn’t consider it absurd.
*) Given the “just one Hungarian” interpretation is correct, I would assume that significantly diminishes the benefit of TI program? The question with this interpretation is not all that long.
*) Also, it feels as if the majority of TI program kids would have done the double Hungarian? That would end up being pretty ironic.
I watched videos on the Ti hungarian program being used (Maffs guru?) anyway, he had to do the algorithm 5 times, and each time swap someone out for Edgar and run the program again hich seemed clunky. I don’t know if he did it in the most efficient way or if he is the ‘go to’ for this sort of thing but this is where my students directed me when I asked about the CAS program.
there were a few teachers at my school that did it several different ways, either adding in a bogus activity or doing a hungarian as a 5×4 (which involves a bit of logic), one other teacher made some observations about the original allocations to then rule out ‘the weakest link’ in his words. All these methods would have found the answer quicker than the cas program.
It is a shame, as I feel like the original direction of the question was for it to be a challenging question for students to use skills they have gained in an unfamiliar way. I actually like the idea question apart from it being a little confusing guaging if we were just swapping Edgar or adding him in the mix and taking the best 4.
I still haven’t had a good read over the whole paper, I have only done the harder questions, but my stronger students did report it felt harder than previous years. (i.e. previous exam 1s they were scoring 39 or 38/40 with 20 mins to spare, where as this one they believe they got about 35/40 and were more pressed for time) I guess we will see with the grade distribution.
Regardless, there were no students terribly upset about the CAS program that I saw at my school, it was mentioned on the Friday after the exam and then no one mentioned it again really.
I saw the HS article on the other post, (lol, what a dramatic headline), is there anywhere I can read it without giving murdouche any money?
Sorry B, this makes no sense to me. As I wrote, it seems clear to me that you need only do Hungarian once. If you are doing Hungarian more than once then I think you have misinterpreted the question.
So you think it reads as just a straight swap from the original allocation? I don’t think that would be difficult enough for the second last question of the module.
I’m second guessing myself now and I’m going to read the question again. (I was originally confused by the question was going when I first read it so I absolutely could be wrong. )
Yes, that is my natural reading of the question, separate from any considerations of difficulty and the like.
But also, considering what is required with the suggested alternate reading (which I gather most people think is the correct reading), it seems clear to me that my reading of the question is what VCAA intended.
I think the issue here may be a GM culture thing, where autopilot, even more than in MM and SM, is the status quo for a question. So, although the question is much easier the way I’ve interpreted it, and although I think it is the natural reading, I think it was hard for a GM student to read it that way, because they have been taught to autopilot: “it’s a Hungarian question, so I have to keep doing Hungarian”.
Of course, I could be entirely wrong. GM is not my world.
I think the ‘new allocation’ written towards the end of the question is intended to suggest a completely new allocation is done. And I am basing this off the fact that it’s the second last question of that section, and is usually a complex question.
Like I said, however, I first read it as a single swap, then I decided it couldn’t be that simple, but a ‘new allocation’ still applies to one person so I could be wrong.
I openly admit, I am prone to not reading things carefully (even this blog at times) so I am not the best person to analyse how a question reads. I can see reasons for both options.
I am assuming we will have to wait for the examination report next year to get the answers?
Has anything been said about the solutions being released earlier this year too?
Thanks, B.
Well, as I wrote, the question is not well worded and there is definitely an argument that the wording is sufficiently ambiguous for the question to be declared “ambiguous” for grading purposes. I’m just of the opinion that it is not bad enough for that.
(One shouldn’t pretend that “ambiguous” is always a black and white thing. There will very often be a contrived interpretation to a question, with it being difficult to avoid this other than by overly legalistic writing. There are often trade-offs, and one cannot shout “ambiguous” whenever such an interpretation exists. Which is also not to excuse VCAA for when they gratuitously screw up the wording, and which has hardly been a rare occurrence.)
The sentence at issue begins,
“When a new allocation is made and Edgar takes over one of the tasks …”
You got stopped by the phrase “new allocation”, and I agree this suggests redoing Hungarian. But then “and Edgar takes over one of the tasks” is referring to the exact same thing, and does definitely not suggest Hungarian. Whatever the intended interpretation, this is bad wording, with either the first part or the second part redundant and misleading. But in sum, I read it clearly as just Edgar taking over a task.
You suggest that having just one Hungarian would make the question too easy (at that stage of the exam). It’s your world, not mine. But again, I don’t see it as easy for GM students to transition mid-question from autopilot Hungarian to ad hoc swapping in Edgar (even though the swapping is of course easy in and of itself.) And the multi-Hungarian seems much more over-hard than the single-Hungarian seems under-hard.
VCAA has indicated that marking guides for all VCE mathematics exams will be released by 4 December. See here and, more generally, here.
Now that VCAA has published their answers, it appears that the answer marked correct for question 39 was D. That is, the wording of “taking over” is ignored, and instead it calls for the running of the Hungarian Algorithm twice…
However, following the various exam scandals that have transpired since the exam was sat, I don’t imagine there will be much public outrage.
I would love to see statistics to indicate how students perform on this question relative to the rest of the paper. This wouldn’t be definitive proof of whether a question is broken or not, but it would be interesting to know, “Of the students who gave the answer B for question 39, what is their average score for the rest of the exam?” and “Of the students who gave the answer D for question 39, what is their average score for the rest of the exam?”
Answers posted here:
Click to access 2024genmaths1-markguideresponses.pdf
Jesus.
Bryn, the question is most definitely broken; the psychometric voodoo is neither here nor there for this determination. But we don’t know yet how the question was graded. As Curious George commented on the Marking Guide post, the current guide doesn’t indicate how weirdnesses and ambiguities were handled. I’m not willing to bet on it, but I’d be surprised if B wasn’t also awarded correct. It damn well should be.
But independent of how the question was graded, I’m simply shocked that the intended interpretation was double-Hungarian. The wording is crappy, and one has to ask how that wording was permitted. But for the intended interpretation of double-Hungarian, the wording is *really* crappy.
Also, one has to ask why the hell double-Hungarian was intended. Whilst I’m not familiar with the algorithm (I didn’t do General Maths), everything I’ve heard suggests that it would be very impractical under timed conditions except when using the CAS program? Or perhaps double-Hungarian wasn’t intended but VCAA decided that it’s more correct🤔
The main reason why I would be interested in the psychometric stuff is that I think it could help VCAA in their exam review process.
For instance, it could become a policy at VCAA that if it were found that a question did a poor job of distinguishing between students, then an additional layer of scrutiny/checking could be provided to a third party to ensure that the question is being marked fairly.
This is especially an issue with multiple choice questions, where there doesn’t seem to be a group of markers who are sitting around discussing whether they think a question is poorly worded/ambiguous (because it just gets marked by a computer so issues can be swept under the rug.)
At the risk of opening a different can of worms, I’d say this analysis would also force GM exam writers/assessors to come to terms with questions asking whether a distribution is symmetric or skewed, where it is not at all obvious (I think Further Maths Exam 1 2022 Question 1 is a good example of this where 54% of the state chose the “wrong” answer of E, but the graph is not at all obvious.)
Additionally, Exam 1 2016 Question 2 is an example of a question where, again, the majority of the state chose the “wrong” answer, because many students believed that it doesn’t make sense to classify a binary variable as being ordinal.
So my main comment on the psychometric stuff is that it could add an extra layer into the accountability, so there is a process for dealing with poorly worded or broken questions in multiple choice exams. The scrutiny is more likely to already exist in written response questions because they are being marked by humans, who at least in theory, can flag their concerns in the initial meeting.
The thread has narrowed so, Jay and Bryn, I’ll reply to you both with fresh comments, below.
Oh shit, I think you might actually be correct about the UDF giving the wrong answer. That’s hilarious😂😂😂 (I do feel a tiny bit of sympathy for people who used it and got it wrong, but I hope they take it as a lesson to do things fairly, with integrity, instead of trying to bend the rules as much as they can get away with)
Any idea where the marking guide is? I’m surprised it’s not available at https://www.vcaa.vic.edu.au/assessment/vce-assessment/past-examinations/Pages/GeneralMathematics.aspx, perhaps it got delayed?
Of course it got delayed.
Jay, I totally agree with your latest comment on the double-Hungarian. I’ve been focussing on the wording but, even worded well, the double-H would have been ridiculous.
Bryn, I’m fine with psychometric voodoo. I’m simply making the point, for the hundredth time, that a question can be broken even if psychometric voodoo gives the all-clear. The Hungarian question is broken.
I don’t like your suggestion to institute a policy to use PA to add an extra layer of scrutiny. It feels too subjective and unwieldy. If there is an argument or clear evidence that a question is flawed then that scrutiny should come automatically (even if, in the past, it never did). Then, the PA may help guide VCAA on what to do. But, without a flag raised by someone, I think it’s muddying and unnecessary to have PA peering over everyone’s shoulder.
As to your pet hate, the ordinal variable thing, I’m with you: I think VCAA has been nuts on this. But I don’t think the solution is to inject PA. I think the solution is for VCAA to stop being nuts.
Yep I agree that the psychometric stuff is going to miss things which are nevertheless broken. But as one step that could be automatically implemented at a policy level, it doesn’t seem bad to say that “if a question fails some psychometric benchmark then it will automatically get flagged for independent review.” I don’t know if it would muddy things by sometimes flagging questions which turn out not to be broken.
Of course, I would prefer a mechanism for schools to raise concerns about questions. I’m not sure what the threshold would be that would require vcaa to respond publicly, as they do have limited resources and might not be able to respond to every single thing, but perhaps if two different schools flag a question as problematic then that could warrant a response.
Thanks, Bryn. I still think you’re looking for an administrative fix to a cultural problem.
There is already a mechanism for schools, or anyone, to raise concerns about exam questions. The problem has been that VCAA, by means of its general arrogance and its specific mathematical cluelessness, thoroughly perverted that mechanism. VCAA’s strong default position has always been “we don’t make mistakes”, so that they would look for any quarter-way plausible argument to avoid admitting error.
Will this perversion lessen in the post-Bennett era? I honestly don’t know. There are a couple reasons to hope so: Bennett forced VCAA to admit they do make errors; and VCAA has now engaged people with serious mathematical clout, who are not of a type to deny reality. On the other hand, the people who made idiots of themselves in 2022 are presumably still around.
We’ll see. The 2024 exams were clearly less flawed and less clunky than in previous years, but there were still some hiccups, the Hungarian question probably the worst in terms of affecting students. If there is no indication that two answers were accepted for the Hungarian question, that would be a bad sign of continued arrogance and denial.
I reached out to VCAA to express my concerns regarding Exam 1 MCQ 39 and heard back from someone in the VCE Curriculum Team, who reported that the question was investigated and then found to be unambiguous with a correct answer of D (i.e., at least two applications of the Hungarian Algorithm were required.)
I did reply expressing that I disagreed with the findings of their investigation and sought permission to share the conclusion on this blog. I include the relevant part of our email correspondence below, reproduced with permission.
To address your concern, I want to assure you that this matter has been thoroughly reviewed. During the exam period, two separate investigations into this exact question were conducted before the results and exam specifications were finalised. In addition, I carried out a further investigation and consulted with a third investigative panel. All three reviews confirmed that there is only one correct answer to question 39, it is D, as per the published marking guide.
Thanks very much, Bryn, both for raising the Hungarian concern in a formal manner with VCAA and for reporting back to the blog. I am not sure what to say. VCAA’s response to the issue with this question has been so, so appalling. I am perplexed. It is exactly this denial of the undeniable that made VCAA a laughing stock in 2022 et al. How can this same nonsense still be occurring? Why do VCAA think they can try this on and not again appear like fools?
It also seems so unnecessary. The 2024 exam questions were, in the main and given the constraints, well written and error-free, and definitely so in comparison to previous years. VCAA had done well. (Mathematically. Obviously there were administrative problems …). They could have conceded this one without opening any floodgates.
Needless to say, the three “investigations” are evidence of nothing without knowing who conducted the investigations, the effective ToR of their investigations and, most importantly, the arguments for their conclusions. Again, I don’t know why VCAA still thinks it can get away with these crap reviews, where they announce their conclusions but don’t explain them, don’t show their working. Absolutely no one will buy this anymore, if they ever did.
Furthermore, what can “investigation” mean? It is not a question of mathematical truth here: it is a question of meaning, of the proper or best interpretation of the question prompt. As Bryn and I have described at length, above, there is *plenty* in the plain reading of the question prompt that points towards a single Hungarian. So much so that I am astonished that the single Hungarian was not the intended meaning.
My only question is whether VCAA really believes that the question is not (sufficiently) ambiguous or whether VCAA knows and is lying about it. For 2022, the denials were so utterly implausible and devoid of reason, I assumed at the time that VCAA must be lying, but it turned out I was wrong: they really were that stupid. This feels the same. I find it so implausible that VCAA cannot see the ambiguity, it feels like they must be lying. But maybe VCAA really is, still, capable of being that stupid.