The 2019 TIMSS results are just about to be released, and the question is should we care? The answer is “Hell yes”.

TIMSS is an international maths and science test, given at the end of year 4 and year 8 (in October in the Southern Hemisphere). Unlike PISA, which, as we have noted, is a Pisa crap, TIMSS tests mathematics. TIMSS has some wordy scenario problems, but TIMSS also tests straight arithmetic and algebra, in a manner that PISA smugly and idiotically rejects.

The best guide to what TIMSS is testing, and to what Australian students don’t know and can’t do, are the released 2011 test items and country-by-country results, here and here. We’ll leave it for now for others to explore and to comment. Later, we’ll update the post with sample items, and once the 2019 results have appeared.

UPDATE (08/12/20)

The report is out, with the ACER summary here, and the full report can be downloaded from here. The suggestion is that Australia’s year 8 (but not year 4) maths results have improved significantly from the (appalling) results of 2015 and earlier. If so, that is good, and very surprising.

For now, we’ll take the results at face value. We’ll update if (an attempt at) reading the report sheds any light.

FURTHER UPDATE (08/12/20)

OK, it starts to become clear. Table 9.5 on page 19 of the Australian Highlights indicates that year 8 maths in NSW improved dramatically from 2015, while the rest of the country stood still. This is consistent with our view of NSW as an educational Switzerland, to which everyone should flee. We’re not sure why NSW improved, and there’s plenty to try to figure out, but the mystery of “Australia’s” dramatic improvement in year 8 maths appears to be solved.

UPDATE (09/12/20)

OK, no one is biting on the questions, so we’ll add a couple teasers. Here are the first two released mathematics questions from the 2011 year 8 TIMSS test:

1. Ann and Jenny divide 560 zeds between them. If Jenny gets 3/8 of the money, how many zeds will Ann get?

2.

(The second question is multiple choice, with options 0.043, 0.1043, 0.403 and 0.43.)

To see the percentage of finishing year 8 students from each country who got these questions correct, you’ll have to go the document (pp 1-3).

What follows is the article Mathematica and the Potential Gaming of VCE, by Sai kumar Murali krishnan, which has just appeared in Vinculum and which we have written about here. Sai’s article is reproduced here with Sai’s permission. Sai can be contacted by email here.

INTRODUCTION

Last year I completed VCE, including Mathematical Methods (CBE) and Specialist Mathematics. At my school these subjects employed the computer system Mathematica in place of handheld CAS calculators. The CBE (Computer-Based Examination) version of Methods also entailed the direct submission of SACs and the second (tech-active) exam on the Mathematica platform.^{1}

Mathematica is extraordinarily powerful and, as it happens, I consider myself a decent programmer. During VCE, I entertained myself by creating custom functions to automate tedious computations, which I then shared with my fellow students. We were able to store these functions in a paclet (package), ready for use on the SACs and the exam. While handheld CAS calculators can also store (less complex) custom-made functions, Mathematica’s vast in-built library and ease of use moves it into a different class. Mathematica enables the creation of exam-ready functions to perform any computation a student might require.

I have witnessed, and experienced, many problems with the implementation of Mathematica, but in this article I will focus upon the two most glaring and most important issues. First and foremost, Mathematica is so powerful that it can trivialise the testing of the mathematics for which it is purported to be a tool. It enables any student who can program in Mathematica or, more perversely, who has a friend, teacher or tutor who can program in Mathematica, to perform well in VCE mathematics. Secondly, and as an inevitable consequence of this trivialisation, the current partial implementation of Mathematica could create a grossly unfair competition, an unfairness enhanced in Methods CBE by effectively permitting Mathematica code to be submitted as an answer. The students equipped with handheld CAS calculators are the victims. Armed with toys sporting 70s Nintendo displays, they are being outgunned by students deploying full-screen guided missiles.

In this article I will illustrate how Mathematica can trivialise exam questions in Mathematical Methods. In Part 2, I provide an example of the use of Mathematica’s in-built functions. In Part 3, I consider the application of custom-built functions. In Part 4, I summarise, and I indicate why I believe the problems with the implementation of Mathematica are only likely to worsen.

IN-BUILT FUNCTIONS

We begin by looking at Question 5, Section B of 2019 Exam 2, which concerns the cubic .

The question first prompts us to find the tangent at x = a, which we perform in one step with the function TangentLine.^{2 }We then find the intersection points Q and P with two applications of the function Solve. Next, the area of the shaded region as a function of a is found by subtracting the area under the cubic from a triangular area: the former is found using the function Integrate, and the latter is found directly from the coordinates using the functions Polygon and Area. Finally, we are required to find the value of a that minimises the area, which is found in one step with the function ArgMin.

What follows is the complete Mathematica code to answer this five-part question:

This solution requires little mathematical understanding beyond being able to make sense of the questions. In particular, the standard CAS approach of setting up integrals and differentiating is entirely circumvented, as is the transcription. In Methods CBE, the above input and output would be considered sufficient answers.

CREATED FUNCTIONS

We’ll now venture into the world of custom Mathematica functions, where programmers can really go to town. We’ll first look at the topic of functions and the features of their graphs. Mathematica does not have a built-in function to give all the desired features, so I created the function DetailPlot. To begin, I use a module to gather data about a function, including endpoints, axial intercepts, stationary points, inflection points and, if required, asymptotes. I then turn the module into an image to place over the graph.

Let’s fire this new weapon at Q2(c), Section B of 2016 Exam 2, which concerns the pictured quartic. We are given the equation of the graph and the point A, and we are told that the tangents at A and D are parallel. We are then required to find the point D and the length of AE.

And, here we are:

With very little input, DetailPlot has provided a rich graph, with every feature one might require within easy reach. The intersection points are ‘callouts’, which means that the points are labelled with their coordinates. In particular the coordinates of D and E have been revealed by DetailPlot, without any explicit calculation. We can then press forward and finish by finding the length of AE, a trivial calculation with the in-built function EuclideanDistance.

In the next example I demonstrate that a multi-stage question can still be trivialised by a single piece of pre-arranged code. In the multiple choice question MCQ10 from 2017 Exam 2, the function undergoes the transformation and we are required to identify the resulting function. For such questions I created the function Transform, and then the in-built function FullSimplify polishes off the question:

My last example is on functional equations, for which I created two functions, FTest and RFTest. I will illustrate the use of the latter function. For MCQ11 on 2016 Exam 2, the equation f(x) – f(y) = (y – x) f(xy) is given and it is required to determine which of the given functions satisfies the equation. Here is my entire solution:

CONCLUSION

It is impossible to have a proper sense of the power of Mathematica unless one is a programmer familiar with the package. This article presents just a few examples from the vast library of functions I created for Mathematical Methods and I found even more so for Specialist Mathematics. My libraries for both subjects barely scratch the surface of what is possible.

Creating such packages requires skill in both programming and mathematics, but the salient point is that any subsequent application of those programs by another student requires no comparable skill. The programs I have written may improve the performance of mathematically weaker students. Conversely, any student without access to such programs or, worse, is required to use handheld CAS instead of Mathematica, will be at a significant disadvantage.

This demonstrates the potential power of Mathematica to change the focus of VCE mathematics and, consequently, to debase its teaching and its assessment. True, the same issues had already arisen with the introduction of handheld CAS; clever teachers and cleverer students have always engaged in creating and sharing push-a-button CAS programs. Mathematica, however, has massively elevated the seriousness of these issues, all the more so since only a fraction of students have access to the platform.^{3}

Technology, including Mathematica, calculators, spreadsheets and the many online programs, have tremendous potential to assist students with learning, understanding and applying mathematics. What is important for educators is to be careful that students are not using this technology to bypass learning and understanding mathematics.

1. All non-CBE students take the same tech-active exams and are considered in the same cohort for ATAR purposes. The Methods (CBE) exam appears to differ in only a superficial manner, and it appears that CBE students have not been considered a separate cohort since 2016.

2. The examination diagrams have been redrawn for greater clarity.

3. Although the Victorian Government offers Mathematica to all schools, to date many schools have not implemented it.

Most people are familiar with doctors’ “Hippocratic Oath”:

First, do no harm.

Yes, this aphorism does not appear in theHippocratic Oath, which is also probably not Hippocrates’. And, yes, the meaning of and fealty to this statement are not nearly so straight-forward. Still, the statement gives a beautifully clear and human principle, a guide on how to think about the difficult work of treating a person in one’s care.

Which brings us to mathematics. We feel that mathematics teachers need a similar guiding light, the Mathematic Oath:*

First, tell no lies.

As with the doctors’ oath, the implications of the Mathematic Oath are not obvious, respecting the oath is not always so simple. But it is a light, telling us the way. And it also tells us the wrong way: if you are a mathematics teacher and you are not telling your students the truth, then you are doing wrong.

Hi, interested to know how other teachers/tutors/academics …give their students a feel for what the scalar and vector products represent in the physical world of and respectively. One attempt explaining the difference between them is given here. The Australian curriculum gives a couple of geometric examples of the use of scalar product in a plane, around quadrilaterals, parallelograms and their diagonals .

The Dunning-Kruger effect is well known. It is the disproportionate confidence displayed by those who are less competent or less well informed.

Less well known, and more pernicious, is the Dunning-Kruger Effect effect. This is the disproportionate confidence of an academic clique that considers criticism of the clique can only be valid if the critic has read at least a dozen of the clique’s self-indulgent, jargon-filled papers. A clear indication of the Dunning-Kruger Effect effect is the readiness to chant “Dunning-Kruger effect”.

One of the unexpected and rewarding aspects of having started this blog is being contacted out of the blue by students. This included an extended correspondence with one particular VCE student, whom we have never met and of whom we know very little, other than that this year they undertook UMEP mathematics (Melbourne University extension). The student emailed again recently, about the final question on this year’s (calculator-free) Specialist Mathematics Exam 1 (not online). Though perhaps not (but also perhaps yes) a WitCH, the exam question (below), and the student’s comments (belower), seemed worth sharing.

Hi Marty,

Have a peek at Question 10 of Specialist 2019 Exam 1 when you get a chance. It was a 5 mark question, only roughly 2 of which actually assessed relevant Specialist knowledge – the rest was mechanical manipulation of ugly fractions and surds. Whilst I happened to get the right answer, I know of talented others who didn’t.

I saw a comment you made on the blog regarding timing sometime recently, and I couldn’t agree more. I made more stupid mistakes than I would’ve liked on the Specialist exam 2, being under pressure to race against the clock. It seems honestly pathetic to me that VCAA can only seem to differentiate students by time. (Especially when giving 2 1/2 hours for science subjects, with no reason why they can’t do the same for Maths.) It truly seems a pathetic way to assess or distinguish between proper mathematical talent and button-pushing speed writing.

I definitely appreciate the UMEP exams. We have 3 hrs and no CAS! That, coupled with the assignments that expect justification and insight, certainly makes me appreciate maths significantly more than from VCE. My only regret on that note was that I couldn’t do two UMEP subjects 🙂

OK, playtime is over. This one, like the still unresolved WitCH 8, will take some work. It comes from Cambridge’s Mathematical Methods 3 & 4 (2019). It is the introduction to “When is a function differentiable?”, the final section of the chapter “Differentiation”.

Update (12/08/19)

We wrote about this nonsense seven long years ago, and we’ll presumably be writing about it seven years from now. Nonetheless, here we go.

The first thing to say is that the text is wrong. To the extent that there is a discernible method, that method is fundamentally invalid. Indeed, this is just about the first nonsense whacked out of first year uni students.

The second thing to say is that the text is worse than wrong. The discussion is clouded in gratuitous mystery, with the long-delayed discussion of “differentiability” presented as some deep concept, rather than simply as a grammatical form. If a function has a derivative then it is differentiable. That’s it.

Now to the details.

The text’s “first principles” definition of differentiability is correct and then, immediately, things go off the rails. Why is the function f(x) = |x| (which is written in idiotic Methods style) not differentiable at 0? The wording is muddy, but example 46 makes clear the argument: f’(x) = -1 for x < 0 and f’(x) = 1 for x > 0, and these derivatives don’t match. This argument is unjustified, fundamentally distinct from first principles, and it can easily lead to error. (Amusingly, the text’s earlier, “informal” discussion of f(x) = |x| is exactly what is required.)

The limit definition of the derivative f’(a) requires looking precisely at a, at the gradient [f(a+h) – f(a)]/h as h → 0. Instead, the text, with varying degrees of explicitness and correctness, considers the limit of f’(x) near a, as x → a. This second limit is fundamentally, conceptually different and it is not guaranteed to be equal.

The standard example to illustrate the issue is the function f(x) = x^{2}sin(1/x) (for x≠ 0 and with f(0) = 0). It is easy to to check that f’(x) oscillates wildly near 0, and thus f’(x) has no limit as x → 0. Nonetheless, a first principles argument shows that f’(0) = 0.

It is true that if a function f is continuous at a, and if f’(x) has a limit L as x → a, then also f’(a) = L. With some work, this non-obvious truth (requiring the mean value theorem) can be used to clarify and to repair the text’s argument. But this does not negate the conceptual distinction between the required first principles limit and the text’s invalid replacement.

Now, to the examples.

Example 45 is just wrong, even on the text’s own ridiculous terms. If a function has a nice polynomial definition for x ≥ 0, it does not follow that one gets f’(0) for free. One cannot possibly know whether f’(x) exists without considering x on both sides of 0. As such, the “In particular” of example 46 is complete nonsense. Further, there is the sotto voce claim but no argument that (and no illustrative graph indicating) the function f is continuous; this is required for any argument along the text’s lines.

Example 46 is wrong in the fundamental wrong-limit manner described above. it is also unexplained why the magical method to obtain f’(0) in example 45 does not also work for example 46.

Example 47 has a “solution” that is wrong, once again for the wrong-limit reason, but an “explanation” that is correct. As discussed with Damo in the comments, this “vertical tangent” example would probably be better placed in a later section, but it is the best of a very bad lot.

And that’s it. We’ll be back in another seven years or so.