ACARA Didn’t Expect the Sydney Inquisition

Catholics Schools NSW has begun some serious interrogation of ACARA. Led by CEO Dallas McInerney, their chief weapon is sense, and truth. Their two weapons are sense and truth, and intelligence. Their three weapons are sense and truth and intelligence, and a pastoral ethic. Amongst their weapons are …

Anyway, onto the interrogation. Greg Ashman has tweeted a letter from McInerney to ACARA CEO, David de Carvalho. (We’ve reproduced McInerney’s letter as best we can, below). Dated February 7, McInerney’s letter hammered De Carvalho for ACARA’s planned changes to the reporting of NAPLAN results. Conveniently flagged yesterday by “this will be great” coverage from SMH‘s education stenographer, Lucy Carroll, Australia’s Charlie Brown education ministers then approved ACARA’s changes.

We’re not going to investigate or try to confirm the details. NAPLAN is junk, making any debate framed around NAPLAN a junk debate. But McInerney makes an important claim, and we trust that McInerney, who a reliable source described as “great”, is correct.

Part of the reporting and interpretation of NAPLAN data involves what are known as the “national minimum standards”. These are declared standards (Bands) for each of the subject domains at each of the tested year levels. The minimum standards vary by year level and year and subject domain, but tend to be around 14%. Now, according to McInerney, those “at risk” kids, at below national minimum standards, are to be grouped in a new category. This new category is to be called “Developing” and will initially be calibrated to be at 10% for each year and subject domain. (11/02/23. At least that was the plan some time before 07/02/23: see the comments made below.)

So, the 14% of kids who can do bugger all will magically decline to 10%. Which is the modern method of addressing a problem, of course, to simply define improvement into existence. And, although McInerney does not express concern, one must note the language: “Developing”? The entire point about these kids is they are not developing. It is a cliché to refer to Orwell, but … Orwell.

And De Carvalho and ACARA? Do they care that their idiotic change has been criticised for being idiotic? It’s a fair bet that, as always, they couldn’t give a stuff.

**************************************

UPDATE (13/02/23)

Prompted by Alex’s comment, we’ve done a little more digging and checking. Here’s a little sorting:

  • On Tuesday, 07/02, McInerney sent his letter to De Carvalho regarding ACARA’s proposed changes to NAPLAN reporting. McInerney’s principal objections, seemingly based upon an ACARA presentation, were: (a) the discontinuity (“time series break”) in reporting method, and; (b) the lowest, “at risk”, group of students being reduced from around 14% to an initial 10%. According to McInerney’s letter, this lowest group was to then be referred to as “Developing”, although his letter included no objection to the term.
  • On Friday morning, 10/10, and ahead of the education ministers’ meeting to be held later that day, there appeared Lucy Carroll’s free kick report for ACARA. Carroll faithfully repeated ACARA’s selling points, in particular the “higher achievement standards” that ACARA’s new scheme would entail. Carroll did not clarify why the proficiency standards would be higher, and was silent upon the issues McInerney had raised with ACARA.
  • Also on Friday morning, The Australian‘s Natasha Bita had a very critical report on the coming ministers’ meeting (Murdoch, paywalled). Titled Politically correct NAPLAN to drop minimum standards, Bita had clearly been fed, at least indirectly, by McInerney’s group. Bita quoted Mcinerney and his letter at length, and echoed his concerns. Bita also zeroed in on the “Developing” label for the lowest group, repeating the “politically correct” slap of the title. Bita wrote generally about the proposed changes, quoting an ACARA spokesman on ACARA’s recommendations: “Options for ministers to consider … more challenging but reasonable standards …”. It is not clear whether or how this is consistent with McInerney’s 10% claim, and Bita writes nothing else of proposed higher standards.
  • Later Friday, the education ministers met, and subsequently both the ministers and ACARA issued media releases. These releases stated that the lowest group would be titled “Needs additional support”. The media releases referred to a “challenging but reasonable standard” in the new proficiency levels, but provided no indication of what this meant. In particular, it was not clear whether McInerney’s 10% objection had been addressed. McInerney’s time series discontinuity concern was only vaguely and poorly addressed in the media releases.
  • Friday evening, Natasha Bita had a follow-up, Stronger standards for 3Rs in tougher NAPLAN test (Murdoch, paywalled). Bita noted the removal of the term “Developing”, writing that “education ministers had vetoed politically correct report cards”. Bita provided no detail on the supposedly higher standards, effectively just quoting the media releases, and was silent on McInerney’s lowest 10% calibration concern. She noted the issue of time series discontinuity had not been properly addressed.

So, where are we? First of all, it is clear that ACARA proposed using the ludicrous term “Developing”, and were either overruled or were ridiculed into withdrawing the proposal. Secondly, it seems pretty clear that the NAPLAN time series will be screwed. Thirdly, we have not been able to locate any detail of the new proficiency standards, and so we cannot confirm whether or in what sense the standards may be higher. Fourthly, and in particular, we do not know whether McInerney’s fundamental objection has been addressed, whether the “Needs additional support” group will be initially calibrated at 10% or something higher. On Sky News yesterday, Amanda Stoker claimed that 10% remained the initial calibration, but Sky should be taken with a grain of salt, and Stoker should be taken with an Eno.

Fifthly and finally, Natasha Bita is a clown. Seriously, Natasha, “politically correct”? It’s not exactly a term of objective reporting. You could at least pretend a little better. And if you’re gonna whack these nuts with thoughtless and pointless slurs, at least go with “woke”. You sound like Bronwyn Bishop.

**************************************

UPDATE (13/03/23)

It is still not precisely clear how NAPLAN categories will be determined. An Age report today, however, suggests that McInerney’s concern was warranted, that the “Needs additional support” group will be initially calibrated at 10%, or similar:

A spokesman for the Australian Curriculum, Assessment and Reporting Authority (ACARA) said preliminary work that was undertaken to set the four new progress levels found about 30 per cent of students would not have met the proficiency benchmark in 2022.

This includes about 10 per cent of students in the lowest-performing category called “needs additional support”. Last year, about 7.3 per cent of students were, on average, below the national minimum standard.

There are two confusing, possibly inaccurate, aspects of this. First, the “about 10%” reflects, but not exactly, McInerney’s stated concern of an initial calibration of 10%: there is presumably no “about” about it. Secondly, the 7.3% of students below the “national minimum standard” is comparing the wrong apples with the oranges: the (perhaps about) 10% is not supposed to be replacing the students below national minimum, but the students “at risk”: the latter, wider group, is around 14%.

it appears that McInerney was right and the reporters were, as usual, wrong. Some day, maybe they’ll learn to not trust ACARA.

************************

11 Replies to “ACARA Didn’t Expect the Sydney Inquisition”

  1. The time series was interrupted when NAPLAN went 100% on line.

    The situation reminds me of what happened in another country in time of drought. The wall of a major dam was marked with a red line that indicated when the country was going into a severe water shortage. I recall one legislator suggesting a simple solution: lower the line.

    1. The format is a lot different now that it is online. Students do different questions and are directed to different difficulty levels based on the number of correct responses they do in a section. Also, I can no longer see the exact question they answer, just an “exemplar” question that is related to the outcome tested. The usefulness of NAPLAN has really tanked (from it’s already low position).

        1. I was able to see the questions in the paper test each year and how each student did on them via SCOUT (online database of school things). This was useful to find any trends in the type of questions or topics students were stuffing up on.

          SCOUT is available to all NSW teachers from what I understand. At least if you could be bothered clicking on each question, you could piece together the paper exam.

          1. I know. Teachers used to be able to see them easily enough. But the plebs couldn’t get them from ACARA before the FOI. If there is a single reporter who ever looked at, much less thought about, a single NAPLAN numeracy question, there is no evidence of it.

  2. I don’t know how much this changes their argument, but from the ACARA media release

    “The numerical NAPLAN bands and the national minimum standard will be replaced by the following 4 levels of achievement:
    • Exceeding
    • Strong
    • Developing
    • Needs additional support.”

    “If your child has not yet achieved proficiency, then they will either be in the Developing category or the Needs additional support category.”

    I don’t see the percentages they are allocating for each though.

    1. McInerney says something clearly different (p2). I guess they either changed the labels, or McInerney got it wrong. Perhaps the percentages were (also) changed, but I have no idea. If so, it raises other questions.

      I gather McInerney’s information came from an ACARA presentation before 07/02, and thus before the approval and media release.

      1. 😂 Apologies! But the timeline is very thorough and explains what happened more. Thanks for putting the work in. Guess we will have to wait until ACARA publishes more details (whether that’s before NAPLAN or when the results are released, who knows).

        Shame about losing the time series, but as Terry and Potii mentioned, with a change to when students are tested and going online (and how that significantly changes how the students are being tested through individualised questioning), it would be hard to accurately convert the scores on the old system to the new one.

Leave a Reply

Your email address will not be published. Required fields are marked *

The maximum upload file size: 128 MB. You can upload: image, audio, video, document, spreadsheet, interactive, text, archive, code, other. Links to YouTube, Facebook, Twitter and other services inserted in the comment text will be automatically embedded. Drop file here