We’ve posted on the general nature of PISA’s mathematics questions here and here, and the main point is the sheer awfulness of what is being tested. One question, however, seemed worthy of special note. The following is the first of the PISA 2012 test questions included in this document of past questions, followed by a guide to its grading.

maybe a flawed plan – though I’m not sure if this is the right place for bad puns 🙂

A good pun on the pun.

There are more than 9 solutions to finding the area of the shape by only using four measurements. If you choose to measure the diagonal from the bathroom to the terrace then you get more combinations.

Thanks, Potii. That was the falsehood that caught my eye. There’s also something in the question itself that bugs me (apart from the plain awfulness of the question).

The word

efficientis problematic for me… sure, I would have taken the approach listed as the second solution if I had to do such a question as a measurement exercise, but that doesn’t make it moreefficientthan other approaches, does it? Efficiency here seems to be measured by the number of terms in the calculation, but as we all know, some calculations are by their nature more efficient than others.And Marty’s puns are wonderful.

Thanks, RF. I do like me some words. In the idiotically fuzzy world of numeracy, perhaps “efficiency” is ok, but that is definitely a poor constructed sentence.

The article right before the boldface four in the question is definitely a poor choice -).

Yep, it’s fatal. Thanks, Frank.

Whoops! Sorry, Franz (with a “z”).

It seems to me that the missing extra solutions with diagonals (mentioned in the comments by Potii and Marty) could have been ruled out as follows: stipulate that the rooms must be measured in real physical space (that is in the house — and, in view of the walls having to be included, around it), not just on the floor plan. This should then have been stated clearly in the question.

Thanks, Christian. Your suggested rephrasing wouldn’t quite do it, since other diagonals are physically measurable. Probably any wording that ruled out diagonals would, in effect, make the question worse.

It’s obviously not great, but I’m not overly concerned that the question as written permits diagonals, since few students will think along such lines. The “there are 9 possible solutions” in the grading guide, however, really, really gets up my nose. It is absolute and plain wrong. It also suggests that PISA doesn’t bother to employ a competent mathematician to, at minimum, scrutinise their questions and solutions.

The much greater flaw in the question itself is, as Franz pointed out, the use of the definite article, implying there is exactly one solution. That is a serious error, a moronic own goal.

Yes, sorry that I overlooked this! Also, to my knowledge, in the applied sciences, a “well-posed problem” is one which, among other things, is required to have a unique solution; hence we may dub this a “well-meant but stuffed-up problem”.

I’d contest the well-meaning.

But Marty… why would PISA hire a Mathematician? (Unless they wanted to test Mathematics).

What they need is a professor of Mathematical Literacy (or someone with a D Ed in Real World Mathematics)

Sarcasm intended, hope was obvious.

Yep, the sarcasm was pretty clear. But, even if (given) PISA doesn’t wish to test mathematics, they have a good reason to consult with mathematicians: they might avoid looking like idiots. (The VCAA might also wish to take note.)

Not likely to happen, but since your raise the point, do you know if NSW hire outside professionals? Their exam papers seem much less idiotic in this regard.

Hi, RF. Very good question, and I have no idea of the answer. It is indeed impossible to imagine the current VCAA clowns writing, or passing, a high level NSW exam. Presumably John Mack, or an equally knowledgable alternative, is watching over them, but I don’t know who. But I think the difference has to be due more to the quality of the inside professionals, not the outside ones: a man’s gotta know his limitations. And some men are more limited than others.

OK… so looking at the question (because I have no other mathematics to do at the moment and this seemed like a good idea) there is no statement I can find that the angles are right angles and without this made clear there is no way to actually complete the question (at the level it is aimed at).

Next, there are 6 measurements and any 4 of them will give the required dimensions. 6C4 is 9, so at least they got this bit correct.

RF, I’m willing to concede them the right angles (perhaps). I think you might have to check your 6C4.

Yep. 15. Bad day… For some reason I thought (at the time) that 6×5 was 18… really bad day.

The wording is there to make it a “real-world problem” … yet their solution requires you to make completely un-real-world measurements of the outside of the apartment (it has to be the outside because they specifically say the wall area is included). What are you supposed to do, get a tape measure and stilts?

Also their “measure the area of each room” example method fails to account for the walls.

So at least in part it becomes a familiarity test instead of an aptitude test — because if two students are unsure how to do this, the one who does not recognise the setter’s intent has a significant disadvantage, in that the fallacious example method and physical implausibility of the desired answer will blow them off course.

Thanks, David. Most of the “real world” stuff seems unreal and unworldly in this manner.