This one comes courtesy of frequent commenter, John Friend. It is an example from Cambridge’s Mathematical Methods 34.
UPDATE (19/08/21)
It amazes me at times what does and does not concern some commenters. That’s not intended as a criticism. Well, it is, but it isn’t. And, it is. It’s complicated.
I want to make a point about this example, and it seems to me sufficiently important to make the point hard, and as an update rather than a comment. We can all easily agree that the above example – the question and the solution – is bad. What I’m not sure is appreciated, however, and what I want to make absolutely clear is that the example is poisonous.
One can argue the benefits of a properly solid mathematics education, but one of the undeniably beneficial byproducts, if not the main game, is training and practice in reasoning. Mathematics is so good for this because, by and large, the objects and expressions and words and statements have precise meanings. This permits you to think about and argue and clarify the meaning and the logic in a clear and circumscribed setting. No one’s going to throw in a “maybe Newton’s cooling is just a conspiracy” in the middle of solving a DE.
This logical setting is undermined when a textbook/teacher/examiner uses terms in an unclear manner. Of course some lack of clarity is inevitable, because we have humans conversing. And, to be overly pedantic, particularly when there is no serious danger of confusion – and that is way too common – is to err badly in the other direction. Mathematics is a language for humans to converse with humans, and it should be used as such, balancing precision and clarity.
The example above, however, is not remotely a failed but good faith attempt at this. It’s not an instance of a clumsy error. The example above is consciously teaching students to ignore the clear and plain meaning of the words, and to answer an entirely different question from that asked. That is poison. Moreover, the question easily could have been worded to have asked directly for the symmetric solution. It is entirely gratuitous poison.
Now, I understand why commenters are querying more generally about the example. Honestly. You guys in the Methods World – God rest your souls – you have to figure out how to survive and how to ensure your students survive. It’s your job. But it’s not clear to me that it is sufficiently understood how bad is the above example. It’s not clear to me that your lessons on this will begin, for example,
“OK, so are we all clear on why the person who wrote this is a half-wit, and that in mathematics ‘convention’ means something entirely different? Great! Now, what does the half-wit want us to do …”
I just want to be sure everyone is keeping in mind, always, the Mathematic Oath, and which on this occasion we can reword as
First, don’t poison your students.
What a shame.
What could’ve been a good question, is a sh*t question. Too much ambiguity.
Saying
leaves much to be desired (but that applies to the textbook as a whole, generally).
Good opportunity to say that
and 
are equidistant from the mean (or similar mathematically rigorous language).
When you cater to the lowest common denominator, you get crap results.
Yep. The question they wish to ask is good and natural, so why not simply ask that question? But, there’s another aspect of the example that irritates me, maybe even more.
Depending on which type of calculator you use, you would not enter the area as written.
Casio can handle this better than TI machines.
Also, you don’t “get the answer” with “this facility”.
Hi, RF. I’m not sure what you’re driving at here. Not that I care much, since it’s CAS crap, but others might.
The first sentence of the suggested answer, about the infinitely many ways of capturing a probability of 0.95, has nothing to do with the normal distribution, but rather applies to all distributions with densities (even some others), making the sentence worse than useless. And what the model answer names as a “convention” can be motivated much better: it has the property of minimizing the length of the two-sided confidence interval, which is how this question becomes of practical (statistical) interest.
Thanks, Christian. I’m not fussed about the infinite solutions line applying more generally, although it’s not a great line. I’m not sure how much “we see” about such solutions in a class with minimal sense of continuity.
But the thing which really irritated me was using the term “convention”, which you, correctly, put in scare quotes. To refer to this one-off and out-of-the blue simplification as a convention is plain wrong, and idiotic.
The exercise could have been framed as an open-ended question that leads to the realisation that there are infinitely many such intervals which is a useful insight.
Christian is correct to say that the basic idea applies to other distributions. But here the general idea is presented in terms of the normal distribution.
Note that the example is not about confidence intervals at all. But the idea may be useful when students come to learning about confidence intervals later.
Q. And why do people so often talk about 95% confidence intervals rather that 96% or 74% etc?
A. Convention.
Reference
Young, D. S. & Mills, T. M. (2014). Choosing a coverage probability for forecasting the incidence of cancer. Statistics in Medicine, 33 (23), 4104-4115.
It’d be a pretty lousy open-ended question. It is also inconsistent with the blasé “we see” beginning in the text solution.
One could ask the reader to find
and 
such that …
The solution would be quite different from the one proposed.
Yes, that would be a logical thing to do:
“Find c1 and c2 such that Pr(c1 < X < c2) = 0.95 and [insert second constraint]. Give your answer correct to [insert positive integer] decimal places."
Thanks to you both for your replies, Marty and Terry.
Marty: Upon reflection, I tend to agree with you on the first point with area and probability. I was probably too censorious.
Terry: I have not said that either the question nor the answer should touch on confidence intervals. In fact, this question is, IMHO, a very good way of showing how probability and statistics are separated. This is also true if, as here, the teacher is aware of how probability is used towards a statistical end. How the teacher goes about all of this will have something to do with the ordering of material in the school curriculum. From my vague memory of Marty’s posts, and even more vague ones of my own, I believe that stats has been pushed to the front rather far. Marty has pointed out the importance of a good probability education before delving into stats, a tenet to which I subscribe as well.
For a really audacious teacher, who additionally wants to get away from it could perhaps be of interest to discuss the concentration function.
Sorry for the incomprehensible sentence at the end, which I meant to read,
“For a really audacious teacher, who additionally wants a nice probabilistic context in which to cover issues somewhat related to this, it could perhaps be of interest to discuss the concentration function.” I concede that this is some distance off, which is partly why I won’t go into details. Perhaps an open-ended one for gifted students.
Christian, I think we share similar views, or, as too many say these days, we are in agreeance.
I am slowly coming around to the view that statistics should be taught separately from mathematics as in NZ.
2015-Vinculum-52(4)-stats
Why so slow? It’s obvious.
I thought the question was stupid. But then I saw the solution.
Thanks, Glen. You mean the literal question or the intended question?
The literal question.
The solution pisses me off for multiple reasons, all of which have already been mentioned :). The question can be fixed, the solution should just go straight in the trash.
There’s one more aspect of the question, suggested by the solution, that really irks me. (But not as much as the bullshit ‘convention’)
Given that the 68-95-99.7 approximation is part of the course, is that not a reasonable approach here – or if not, why not?
In Fenimore Cooper’s Literary Offenses, two of Mark Twain’s rules for writing are,
Say what you are proposing to say, not merely come near it.
Use the right word, not its second cousin.
The same principles apply for setting exercises.
Indeed,
the ‘answer’ found using the 68-95-99.7 approximation appropriate? Given the question as stated, and even taking into account the stupid made -up ‘convention’, what 
be an appropriate answer …?
Some clarifying context: The 68-95-99.7 approximation is part of the
section of the textbook. This example is from the section titled “Calculating normal probabilities”.
SRK – your approach would be reasonable if the question made any suggestion that c1 and c2 were the same distance from the mean.
You could set c1 to minus infinity and then solve for c2 and obtain a perfectly valid solution the way the question is currently written.
And I’m being a bit liberal with the use of “perfectly valid” here.
Indeed. That gives the minimum possible value of c2. And similarly, you can set c2 to plus infinity to get a maximum value of c1. But, if we accept that the
of the question was that students apply the idiotic ‘convention’ that Cambridge has invented, there’s still a big idiocy that’s been skirted around but not yet explicitly mentioned. There’s an infinite number of possible ‘answers’ (c1, c2) (within the scope of Maths Methods) for another very simple reason …
In other words, maybe Cambridge is using another invented ‘convention’ when getting its answers …
JF – are you referring to the rounding to 3 dp, when no degree of accuracy is mentioned in the question?
Bingo!
VCAA have a global instruction that
“In all questions where a numerical answer is required, an exact value must be given unless otherwise specified.”
This is one of the rules of the game and so we train our students to follow VCAA instructions such as this.
So, by not specifying an accuracy for the answers, Cambridge is asking our trained students to give an exact answer. And of course, within the scope of the course, no exact answer is possible.
But then Cambridge gives an answer correct to 3 decimal places. So now students are confused:
1) Why has an approximate answer been given when no approximation has been specified?
2) Why 3 dp? Why not 4 dp? Or 5 dp? Or integer answers (from the 68-95-99.7 approximation rule)?
3) Does this mean its OK to give answers to 3dp when no approximation is specified?
How fucking hard is it to
for an answer correct to 3dp or whatever!?
The over-arching idiocy of the example is the lack of clarity in what sort of answer is required. We have:
1) a fabricated Cambridge ‘convention’.
2) no specification of accuracy.
Marty is correct in calling out 1) as “gratuitous poison”. From a pragmatic point of view, 2) is equally gratuitous poison.
It’s well within VCAA’s ‘capability’ to make error 1). But not even VCAA makes error 2).
Maybe a dumb question, but what would a well-meaning student get who wrote something like “The exact value of
is the parameter such that the equation *integral equals blah* holds”?
What that student got would depend on the intelligence of the assessor.
0 for not giving the answer in the required form.
In which case the well-meaning student would ask where in the question the required form is stated. After which the well-meaning student would then point out the VCAA instruction.
At which point the assessor would be ready to string up the idiot who wrote the question and marking scheme.
It was sarcasm, JF.
Aha. I misunderstood. Unfortunately, your comment was perfectly plausible!
Plausible because I can well imagine some assessors out there working off the dodgy Cambridge marking scheme that would give it zero because they have no brains.
So, how many students would you expect to write c1=88, c2=112 and then move on with their lives?
I would guess a fair few if I were to give this question as it is currently written.
RF, I’m not sure of the point you’re making. You can’t possibly be defending the example.
Not for a moment.
I’m suggesting very few teachers would think there is anything wrong with it as the answers are “close enough” to what many of them believe (in my experience) to be a hard-and-fast rule (or they did before VCAA changed the study design to say 1.96 standard deviations instead of 2).
VCE probability is (in my opinion) the easiest part of the course, yet seems to be an area that students do poorly on. Who is to blame? Teachers, textbooks or the curriculum?
And yes, trifecta bets are permitted.
Separate topic… is anyone else bothered by the lack of a vertical axis in the picture?
I’m not too fussed by that. I would have liked to at least seen x = 100 labelled on the horizontal axis, however.
From the wording of the question, is the answer c1=88 and c2=112 any less reasonable than the answer given in the solutions?
I think the title of the blog gives a good clue for the idiotic so-called ‘convention’. Maybe an alternative title like ‘Abnormal Answers’ might suggest (what I think is) a piece of idiocy yet to be explicitly mentioned. It’s a piece of idiocy I don’t think even VCAA would commit.
btw I suspect that in some classrooms the answer c1=88 and c2=112 would be considered an exact answer …
Thanks for putting into more coherent sentences what I was going for…
The moment it refers to the curve as a “normal distribution” there is an implication that the mode and mean (and median) are the same value, surely?
Or does the text not say this (I haven’t read the book in much detail, and for good reason).
Yes there is. But I don’t think that’s relevant to any of the idiocy in the example. Where were you heading with this?
On the update: of course Marty.
On the correct answer being not what was asked: this actually happens quite a lot, especially at primary school levels. For a good student who wants to receive As, they are poisoned long ago. Sad.
I remember a compulsory internal PD on teaching gifted students. We were asked to solve some puzzles. The usual banal stuff. One of the puzzles was
Fe Fe Fe
Fe Fe Fe
Fe Fe Fe
I always like to subvert this sort of PD (especially when it’s compulsory) and test how genuine the presenters really are. So, although I knew what the ‘correct’ answer was (Nine Iron), I decided to be ‘gifted’ and volunteered a different answer.
The reaction of the presenter was not entirely unexpected.
I chose to discretely leave the PD. I had to explain afterwards to school leadership why I left. I said I had better things to do with my valuable time and suggested that they not waste it with compulsory PD conducted by bullshit artists.
Teacher PD is a special kind of hell. When I am forced to do PD (rarely) at my university, I normally derail it (in good faith). By the way, the presenters at my PD are no better educated than those at your PD — maybe they will do both gigs.
What was your answer and what did the presenter do btw?
My answer was clowder. Spoilers below.
The presenter mumbled something like “Huh, that’s interesting. Anyway most people say Nine Iron. OK, let’s move on to the next puzzle …” I considered how a gifted student might feel about such a reaction (disappointment …?) and then decided I had better things to do with my time.
The Fe are in lines, hence Felines, otherwise known as cats. There was therefore a collection of cats. The collective noun for a group of cats is clowder – a clowder of cats.
Actually I consider the 6 felines much more obvious than the 9 irons, and you don’t even have to know the chemical symbol for iron … (The are other felines but the vertical and horizontal ones are the most obvious).
I was lucky that I’m a cat lover and actually knew the word clowder. Otherwise I probably would have said Pride (lions are cats and so you have a group of lions). Anyway, the fact that a presenter on teaching gifted students couldn’t acknowledge some (admittedly smart-ass) ‘meta lateral thinking’ disillusioned me into not sticking around.
That is a great answer.
Thanks, Glen. Of course there is a lot of this poison, some of it from carelessness and plenty of it from the arrogance of ignorance. What seems to me particularly remarkable about the Cambridge example is that the writer *knows* they are not asking the intended question, and *knows* that they are teaching students to look for ritual rather than meaning. That’s pretty special.
Marty and JF, perhaps I’ve just been conditioned to ignore certain things in textbooks, but upon re-reading this example, I cannot get over how awful the start of the question is:
“Suppose X is…”
Really?!?
How about random variable X is normally distributed…?
You’re right, RF. The “Suppose …”
awful. The whole question is a turd pile.
I’d modify your suggested improvement to:
Let X be a random variable that follows a normal distribution with mean 100 and standard deviation 6.
Ugh! Yep, that is pretty awful.
Hey everyone,
Can any of you guys recommend some good books/courses to get a deeper knowledge of stats and probability? I’m a relatively new methods and specialist teacher and I’m looking to improve my knowledge in this area. I learn a lot from this blog (thanks), so sort of wanted that personal touch vs. google. To be specific, the content dealt with in the methods and specialist courses is pretty much the boundary of my knowledge – I never studied stats or probability in my maths degree.
I can’t speak for others, but I found a copy of T Apostol “Mathematical Statistics” from the 1970s, read it, asked a few people I knew worked at universities (including Marty) a few specific questions, read the VCAA exams, realized I’d wasted 80% of my time on stuff that never gets asked, even though the study design hints at it, went to a PD, realized I now knew more than the person presenting, went back and asked Marty a few more questions and now consider the new probability and statistics to be the easiest parts of a Methods or Specialist exam.
But that is just me. Others may have different experiences.
Awesome, I’ll definitely check it out. Thanks Red Five!
Hi MC. You will learn a lot simply by teaching this stuff and thinking about it.
I’ve found the following book pretty good (I bought it back in 1991 for 9 dollars – reduced from 59 dollars – from a discount bin at Monash University):
Probability and Statistics for Modern Engineering by Lawrence Lapin.
I’m sure you could get a decent second hand copy or borrow a copy from a university library. Be prepared for only a couple of chapters to be relevant to your purposes (this applies to all the textbooks on probability and statistics you might look at).
You might also find Schaum’s Outlines of Probability and Statistics useful – it’s not hard to find a pdf copy on-line.
Once lock-down lifts, you could do worse things than spend an afternoon at a university library browsing through the statistics books.
I totally agree with RF that “the new probability and statistics [are] the easiest parts of a Methods or Specialist exam”. It particularly advantages hard-working rote learners. Unfortunately it’s a load of shit at this level – I would not consider what’s prescribed in the Study Design as mathematics.
Hey JF, thanks for your response! I’ll buy the book – I’ve got a growing collection now. I’ll pick up Schaum’s too, I’ve got the linear algebra one and have found it useful. Yeah, it seems to me that it’s a bit disjointed from the rest of the specialist course. I look forward to gaining a proper understanding of how heavy the load of shit is in comparison to other normally distributed loads of shits.
Further to JFs comment – they Type 1 and type 2 error stuff is really annoyingly basic, you can pretty much ignore it until the day before the exam.
When it comes to the 1-tail and 2-tail tests, throw all logic out the window and read the VCAA (or textbook) question as though they have no idea about actual statistical analysis. See some of Marty’s previous posts on this.
When it comes to the other probability stuff, once you understand the difference between the integer multiple of a random variable and the sum of identically distributed random variables, you’ve basically covered everything VCAA is going to throw at you.
When I teach this at SM Year 12 level, I spend a lesson (1 is enough) covering what they “learned” in Methods Year 12 as a precaution, because a lot of that seems to be rote-learned as well. Annoyingly, the two subjects use different formulas for exactly the same calculation at times…
Google (and Wikipedia) is actually pretty good with a lot of statistical stuff, provided you are quite specific in your search and obey the golden rule: just because VCAA uses a particular word or phrase it doesn’t make it the accepted convention.
Hi, MC. I’ve never taken a stats course in my life (don’t tell Red Five). So, the other guys here will give you much more sound advice than me. But I did want to make one point, which I haven’t seen made clearly.
After VCAA ratcheted up the stats, and after vomiting with disgust, I did some reading; I knew the VCE material was a perversion, but I didn’t know of what. The reading was intrinsically interesting, and it made clear to me what the VCE textbooks were trying and failing dismally to make clear. But, in the end it seemed a *lot* of reading to sort out very few concepts and theorems.
Now, I’m sure there are more efficient ways to read proper literature and get a handle on the VCE material. But, in the end, it’s trying to get a handle on an absurdly watered down version of very not much. So, I think it is in the nature of this exercise that it is inefficient. Which is fine if you’re happy to learn for the sake of learning, but frustrating if you’re simply trying to get a decent handle on the solid stats specifically underling the VCE swamp.
You may or may not recall Marty, but a “colleague” of mine once asked which answer you thought was correct on a VCAA exam – the question was on the new stats material.
You wrote F – who cares?
It prompted me to wonder – is this VCAA statistics actual statistics or some weird looks-like-statistics-but-is-actually-something-else crap that a smart person would thus simply rote-learn a bunch of standard answers and focus their attention elsewhere?
I then asked someone (whom you have met – their initials are also RF) who *really* knows their statistics. He told me the VCAA questions read more like philosophy than statistics.
And that is where the story ends.
Yes, I remember that question.
I’ve asked at least three stats professors whether the VCA stats topics were as pointless as they appeared, and they all had already come to that conclusion. They are were all of the opinion that VCAA had not screwed up by their presentation of the topic, but by having the topic in VCE at all.
Clearly some ignoramus (or plural) with far too much influence thought differently. I can accept this as simple ignorance. But then some imbecile who should have known better supported the ignoramus (or plural) by not rejecting the suggestion.
VCAA has the opportunity to fix its screw-up with the new Stupid Design, but won’t.
One of the
failings of the Probability and Statistics topic in Specialist Mathematics is its treatment of independent random variables.
The formal definition for independent *discrete* random variables is explicitly stated in the Stupid Design and taught in Maths Methods. But …
The Stupid Design makes references to “independent random variables X and Y with normal distributions”. What does this actually mean mathematically? NOWHERE in the Study Design is there a formal definition for what it means for two *continuous* random variables to be independent.
The mathematical meat of this topic is totally ignored in a zealous vegetarian quest to get to recipe-driven hypothesis testing.
But it is
hypothesis testing as it stops after a single test.
Surely, somewhere there would be a mention of… “OK, so you rejected H0, so what do we do now…?
Or does that make too much sense?
Independence is a complicated matter. See Sections 3 and 7 of Stoyanov, J. (1987). Counterexamples in probability. Wiley.
Indeed.
(btw I have that book and it is excellent).
RF, what was the question?
If you recall when VCAA published the “specimen” exams in 2016ish it was one of the last multiple choice questions on the paper.
Basically asking had you memorised the definition of a Type 1 error.
As Marty said: who cares?
Ah yes, that’s right. The wording is poor. Is that what your “colleague” was querying?
@MyCool
Hoel, P. G.
Introduction to mathematical statistics
Click to access 302_01_Hoel_Introduction_to_Mathematical_Statistics.pdf
I see that the new edition of Cambridge retains the exact wording for this example. Clearly it was either too difficult for the author to re-word it, or the author cannot see how stupid the wording is.
Or, no one looked.
Actually, that seems the most likely. The sections with no new material seem to have kept all the errors from the previous edition, and were probably perfunctorily vetted at best.
It’s probably nice that some concepts like CIs and normal distributions are permeating into high school. But I did not get that (in a pretty strong public school curriculum). And did fine in chemistry, physics, engineering, including at the college level.
I did take a probes and stats class in college, but it was an elective. And I though the central limit theory and the rest of that stuff was pretty powerful. But again, the lack of it didn’t hurt me in science and engineering, even graduate chemistry (which granted, really doesn’t rely on statistics the way medicine or particle science needs to).