VTAC Converts an Own Goal

VTAC, the Victorian body responsible for figuring out ATARs and the like, is, of course, a professional and widely respected organisation. VTAC is staffed by very well-qualified boffiny stats types, just quietly doing their boffiny stats thing, and they don’t make mistakes. Well, except for this. And this. And really, really this.

Anyway, a few months ago the Evil Mathologre alerted us to some new(ly discovered) weirdness courtesy of the always-perfect VTAC. Or, maybe it’s courtesy of ACTAC, which is the national umbrella group encompassing all of the State-TACs. It’s difficult to tell, since, much like QANTAS, it seems that ACTAC cannot be contacted, appearing to exist only as an abstract concept. Anyway anyway, to the weirdness.

One of the genuinely tricky tasks that VTAC/ACTAC is assigned each year is the conversion of subject scores for prerequisites from one state to another. Let’s imagine, for example, that we have a NSW student, Eddie Wow, who has completed HSC and then moved to Victoria. In Extension 2 Mathematics, Eddie received a 42; that is at the upper end of Band E3, which ranges from 35 to 44. Now Eddie wants to know whether that score is good enough for him to be accepted into advanced mathematics in first year uni in Victoria. Well, Monash requires a 35 in Specialist Mathematics for their advanced subject, and Melbourne Uni requires a 38 for their accelerated stream. So, the question is, how to convert Eddie’s 42 in HSC Extension 2 Mathematics into a nominal VCE Specialist score.

The answer is, of course, there is no answer. Any such conversion scheme will necessarily be clunky, with a very high element of guesswork and arbitrariness. But the problem won’t go away, and one must do what one can. Unless, it seems, you’re VTAC.

This is how VTAC/ACTAC converted NSW HSC to Vic VCE for 2021: What does this all mean for Eddie? Eddie scored in Band E3, which VTAC has declared “Meets [a] Victorian Prerequisite at a specified level of 32”. Since 32 is below 38 and 35, Eddie loses. Eddie loses, that is, unless the Mathologre or their equivalent employs some discretion to ignore this nonsense.

Even without considering New South Wales’ bands, there is something starkly nuts about VTAC’s broad brush scheme. From 32 to 41, and ditto 23 to 31, covers a massive range of achievement in Specialist Mathematics, and in any subject. Which isn’t a big problem if the scores in between are uncommonly prerequisites. Such scores in between are very commonly prerequisites.

Why do this? Why instruct everyone to use such an unusable table? We asked.

We emailed VTAC and very quickly we received back a long, friendly and detailed reply. Our VTAC correspondent alerted us to the existence of ACTAC, and they noted the narrow purpose of the VTAC conversion tables, for determining broad prerequisites. This was opposed to the ATAR, “designed to be a nationally equivalent fine-grained rank”. As such, for the purposes of the subject conversion tables,

“it is neither useful or necessary to perform equipercentile matching or a similar process on cross-state systems to establish a fine-grained conversion at the score level”.

On NSW and Victoria specifically, our correspondent noted the use bands in NSW for these prerequisites, while the “vast majority” of Victorian prerequisites are listed in bands of 5, from 15 to 35. And, somehow thus,

“As New South Wales uses HSC performance bands to assess prerequisite satisfaction for its own HSC students and for interstate students, it is entirely appropriate for VTAC to state the equivalence for Victorian universities using the NSW performance band levels”. 

More generally, our VTAC correspondent assured us that these conversions were all professionally done, with the State TACs comprising “expert level technical groups comprising mathematicians, statisticians, academics and education experts”. Our correspondent concluded,

“In summary, the method used to convert Victorian course prerequisite score requirements to NSW HSC performance bands is not an error requiring correction, but a robust process used and refined for over 20 years, developed by mathematically-qualified expert groups in a national collaboration between educational administration bodies in every state and territory.”

All very convincing. Except that the unusable VTAC table remains unusable.

Ignoring, for the moment, VTAC and VTAC’s unusable table, what should be done with our friend Eddie? It is unclear, but a reasonable approach is suggested by a second table, provided by NSW’s UAC, which associates 2021 NSW bands with VCE scores: This second table suggests that HSC’s Band E3, from 35 to 44, corresponds best-guessingly to VCE scores in the range 28 to 37. Then, further linear guessing would convert Eddies’s 42 to a nominal Specialist score of 35. Enough for Monash, but not enough for Melbourne. Lots of guesswork, but pretty clearly a more sensible and more useful guess than VTAC’s “at least 32”.

Why does VTAC not do something like this? We asked.

It took a while to find the time, but a couple weeks ago we replied to VTAC. We noted Victoria’s increments of 5 made VTAC’s table typically useless. We also suggested (before we discovered the second table and perhaps somewhat inaccurately) that the lack of granularity in VTAC’s conversion was systemically lowballing NSW students. We also indicated how impressed we were to hear that the conversion tables were the product of mathematically qualified expert groups, and so on.

We have not heard back. Presumably VTAC are just too busy with their mathematical experting and robust processing. You know them boffins.

17 Replies to “VTAC Converts an Own Goal”

  1. All these Govt education departments staffed by boofheads *ahem* I mean boffins are the same. VCAA, VTAC, ACTAC, VRQA, DET … The list goes on like a Staus Quo song. And they all have SFB. It’s natural selection.

    I’d \displaystyle like to think that Victorian universities are aware of this long-running comedy and apply an element of common sense for students like Eddie Wow. Some things I hope they would consider:

    1) Extension 2 Mathematics is a far superior subject to Specialist Mathematics, and this in itself requires a ‘degree of difficulty’ scaling above and beyond a Performance Band conversion. If Specialist Maths is a forward \displaystyle 1\frac{1}{2} somersault tuck, then Extension 2 Mathematics is a forward \displaystyle 3\frac{1}{2} somersault tuck.

    2) The UAC table.

    But who knows … Victorian universities are very parochial – they all consider themselves superior to everyone else (particularly the one with a big campus in Parkville, who would probably relish telling Eddie close but no cigar).

    1. Thanks, John. Your point (1) is of course highly relevant. Presumably there is scaling within NSW that takes proper note of that, but I didn’t try to investigate that aspect in this post. Re your point (2), in general I think 1st year coordinators have a reasonable amount of discretion in determining who enters advanced subjects and the like. But, yes, there is no question that Unimelb Maths has a history of asshole arrogance in this regard. Whether that is still the case, I don’t know.

      1. The tricky thing of course is that the universities require that the \displaystyle unscaled study score satisfies the threshold.

        I don’t know if NSW scales its Performance Band scores or not, but if it did, I suppose the HSC score would have to get ‘unscaled’ and then converted to an ‘equivalent’ (unscaled) VCE Study Score. I would still want that (unscaled) ‘equivalent’ score multiplied by a ‘degree of difficulty factor’ because I do not think an unscaled VCE Study Score of 38 in Specialist is worth more than a HSC converted and ‘unscaled’ study score equivalent of 35 in Extension 2 Mathematics 2.

        (What I’ve just said probably should get multiplied by a degree of difficulty factor of 10).

        The bottom line is that they’re trying to compare a score in Specialist Mutton with a score in Extension Sirloin. And coming up with sausage.

        It’s all very rubbery and slippery and non-transparent. Despite what VTAC says, it’s voodoo mathematics at its finest. You’re right, “1st year coordinators [probably] have a reasonable amount of discretion in determining who enters advanced subjects and the like”. I have no doubt a NSW Extension Maths 2 student would get very fair consideration at Monash University. But as for some of the other universities, indeed …

        Of course, it would be very simple if a university simply said what score it required in a subject from each other state. So, for example, a university would say that it required a HSC score of X in NSW Extension 2 Mathematics for entry into its advanced 1st year uni maths etc. Problem solved and no need for the witchdoctors and their arcane magics. Does this seem like too much common sense to ask for?

        1. So… are we in agreement that the universities (two in particular) are the issue and that VTAC’s attempt to “help” is no help at all…?

          Because, if we are, the only solution I can see requires two steps:

          1. Replace VTAC with a Mathematician (or someone with a brain and a heart)

          2. Replace the universities’ directors of admissions with a Mathematician (or someone with a brain and a heart).

          I can’t see any other approach working any better than what is currently used.

          1. The problem with point #2 is that it is a temporary unreliable fix. Sometimes, for example, the School is asked about entry requirements. When that happens, we can make good decisions. Often however it is handled by a nameless administrator, and they will be guided by tables and computer systems.

      2. A long time ago, I went to a talk by a leading Australian statistician who spoke about how scaling was done at the time in NSW. He reckoned that he was one of a few people in NSW who understood how it worked.

        1. Thanks, Terry. I have no doubt that scaling is intrinsically very difficult to get right, or “right”, which is probably the best that one might hope to achieve. But VTAC’s knee-jerky employment of that difficulty in their “how dare that anyone question us experts” responses, combined with their blatant stuff-ups, makes them a laughing stock.

  2. Well I work at one of the “problem universities” and have a lot of friends working in maths at the other (including the person with discretion over whether Eddie Wow goes into advanced 1st year or not). We all collectively rate HSC ext 2 as being so far above Spec 3+4 that’s it’s not even funny. We always let them come into the advanced programs at 1st year. A band 6 HSC ext 2, we would waive 1st year altogether if we could!
    The mandarins in charge of admissions at Vic unis are all PR flunkies and apparachiks of the corporate managerialists who run the unis. It would be PR death to admit even the faintest admission that one state system in mathematics is superior to another. Hence the clearly fraudulent declaration of Spec 3+4 having aspects of superiority over HSC ext 2. North Korea has something to learn from this crew.

    The maths depts are not fooled by these highly paid vipers of disinformation but we are intimidated into publicly playing the universities’ vacuous game . But we know what the reality of what mathematics is.

    1. I’m glad to hear that there’s some common sense applied to these things.
      Re: The mandarines and other flunky fruit … This does not surprise me one little bit. VCAA, its stooges and its VCE live in spin city. They fool some of the people most of the time, and most of the people some of the time but not all of the people all of the time. It’s days are surely *ahem* …. numbered (as are the ATAR’s).

      (And you have implicitly explained why a university does NOT say what score it requires in a subject from each other state. And perhaps even why there’s so much smoke-and-mirrors in the conversion of scores from one Yr 12 system to another).

      Believe it or not, there was a time back in the 80’s and early 90’s when mathematics in Victoria was superior to mathematics in NSW.

    2. Thanks very much, 5th C. Very interesting.

      Of course to anybody who spends a half minute comparing Ext 2 and SM, the difference is undeniable, and hilarious (in the Terry Gilliam sense). And of course the university mandarins are intellectually and morally corrupt, if not outright insane. But also in general university mathematicians pay little attention to school mathematics, except to whine ignorantly about it, and so it isn’t clear to me who does or does not know about the HSC-VCE distinction.

      Two questions:

      1) Where, either publicly or university-internally, has anybody claimed that “Spec 3+4 [has] aspects of superiority over HSC ext 2”?

      2) If the maths departments are not fooled, is there any chance that they might get off their collective ass and try to hammer VCAA into offering not-shit? Or, are the mathematicians still happy with their now decades-long strategy of aimless whining?

      1. It’s not a direct assertion of superiority, but I’ve always found the words “The Education State” on Victorian license plates to be funny.

  3. Feel like there is a very simple way to solve these issues… a national mathematics (or a set) of exams. Many countries with many times our population have such tests.

    Another not so simple but very common approach is to have entry tests held by the universities themselves… that ensures their requirements are met.

    1. For better or worse, neither has any real chance of working in Australia. Australia’s Federal system means the states are responsible for education.

Leave a Reply

Your email address will not be published.

The maximum upload file size: 128 MB. You can upload: image, audio, video, document, spreadsheet, interactive, text, archive, code, other. Links to YouTube, Facebook, Twitter and other services inserted in the comment text will be automatically embedded. Drop file here

This site uses Akismet to reduce spam. Learn how your comment data is processed.