In Defence of Impossible Precision

John Allen Paulos has quite a good column on innumeracy, first he asks readers to assess the following headlines:

1. After the Packers' Super Bowl victory, an exuberant Aaron Rogers Shook Hands with Everyone in the Stadium.

2. Experts Fear Total US Housing Costs (Rents plus Mortgage Payments) Will Top $2 Billion in 2011.

3. Only by Completely Eliminating Foreign Aid Can We Eliminate the Deficit.

What is wrong with them? Well, I don't expect anyone reading this not to have noticed that shaking 40,000 hands would take at least 10 hours, that $2 billion comes to about $10 per person or that Foreign Aid is such a tiny portion of the US deficit that even eliminating it entirely wouldn't make a big dent (this doesn't, of course, mean that it shouldn't be eliminated, only that if you're obsessed with the deficit, you have bigger fish to fry).

He then moves onto the following headline:

4. Number of Americans with Alzheimer's Believed to Be 5,451,213.

The supposed problem?

4. The problem here is that the number is ridiculously precise. Definitions of Alzheimer's vary and it's difficult to determine whether a single individual is suffering from it, much less whether five million plus are. Such impossible precision is common.

Well, yes, the number is ridiculously precise. No, no-one does think that we can measure the number of Americans with Alzheimer's to that degree of accuracy, but so what? If you do a survey of Americans, do some calculations, and your best estimate of the number of Americans with Alzheimer's comes out as 5,451,213, what number, exactly, does Paulos want you to report?

Assuming that you've done your sums correctly, 5,451,213 is an unbiased estimator of the number of Americans with Alzheimer's. Rounding your guess to 5.5 million does systematically worse than just reporting the estimator you got out of your calculations, so what exactly is the rationale behind it?

Yes, numbers like this should probably be reported along with some estimate of variance, and maybe it's a convention that we assume the number of signficant figures of a number to be a proxy for the size of its error bars, but it doesn't have to be that way: I look forward to a day when numbers like "5.5 million" get scoffed at by popular mathematics writers for being "overly round" or "not accurate enough".

What should be in a maths class?

I have recently (well, in the past two years) read two very interesting essays on the teaching of mathematics. At first glance, they seem to be almost diametrically opposed, but I tend to find myself agreeing, overall, with the thrust of both. The essays are Paul Lockhart's A Mathematician's Lament and Conrad Wolfram's TED talk on mathematical education. A couple of quotes from each which provide a brief summary.

From Wolfram's talk (trancript available here):

I want to see a completely renewed, changed math curriculum built from the ground up, based on computers being being there, computers that are now ubiquitous almost. calculating machines are everywhere and will be completely everywhere in a small number of years. Now I'm not even sure if we should brand the subject as math, but what I am sure is it's the mainstream subject of the future.

From Lockhart's Lament:

The art is not in the “truth” but in the explanation, the argument. It is the argument itself which gives the truth its context, and determines what is really being said and meant. Mathematics is the art of explanation. If you deny students the opportunity to engage in this activity— to pose their own problems, make their own conjectures and discoveries, to be wrong, to be creatively frustrated, to have an inspiration, and to cobble together their own explanations and proofs— you deny them mathematics itself. So no, I’m not complaining about the presence of facts and formulas in our mathematics classes, I’m complaining about the lack of mathematics in our mathematics classes.

So, Lockhart thinks we should be teaching mathematics as an art form, and Wolfram thinks we should be introducing more computers into mathematics lessons so that people can concentrate on doing the bits that are actually useful. Which of them is right? Well... both, but mostly Wolfram.

The question we have to ask ourselves before we can even begin to compare the two essays is why do we teach mathematics at all? So far as I can see, the only sensible justification for having mathematics as a subject that everyone in the world should be taught up to a relatively high level is because it's useful. You can't do physics, or engineering, or any sort of science, or do derivatives trading, or even decide which mortgage to get, without knowing quite a lot of mathematics. For this reason, everyone should be taught the basic mathematics that they need to know in order to do these things (or the basics they need to know in order to learn the specific maths they wnat to use).

Note that one corollary of this mode of thinking is that most of the mathematics you learn probably shouldn't be taught in a maths class. It's much easier to learn how to get from acceleration to speed than it is to learn how to differentiate a function. Yes, it is useful to then point out the possible generalisations (getting from acceleration to speed is the same as getting from jerk to acceleration) but I don't see any reason why these topics can't be introduced concretely. I personally have serious trouble doing any calculus that I can't do using physical intuition, and I think it's fair to say that I am an above-average student when it comes to learning maths. Calculus should be taught when you're doing engineering, statistics should be taught when you're analysing the results of experiments, graph theory should be taught when you're trying to solve scheduling problems.

Lockhart, on the other hand, seems to think that we should be learning mathematics because, essentially, mathematics is awesome. I happen to agree with him that mathematics is an exceptionally beautiful art form. I'm happy to sit back and bask in the glory of Cantor's diagonalisation argument, or the ingenuity of Karp's reductions between NP problems, but I'm not sure that I'm willing to contend that everyone should be forced to. Yes, if you want to be a mathematician you have to learn that mathematics is actually an art, but most people who study mathematics don't want to be mathematicians, and most people who study mathematics *shouldn't* want to be mathematicians. For these people, learning about the art of mathematics is little more than an intellectual curiosity, on a par with learning about Titian or Shakespeare.

In other words, Lockhart is right, inasmuch as we want people to study mathematics for it's own sake. Wolfram is right, inasmuch as we want people to study mathematics because it's useful.

Now, I happen to tend strongly towards the idea that the only things we should be teaching in schools are things which are potentially useful, but that obviously isn't the prevailing wisdom - everyone in this country is still forced to do an English Literature GCSE. Lockhart-style mathematics is a perfectly good substitute for art class, or critical theory. Wolfram's mathematics is a necessary prerequisite for doing just about anything else.

The Story of a Proof

I started writing this blogpost about 6 months ago after reading an interesting post by RJ Lipton on why mathematicians prove things. I can't find that post any more (maybe it wasn't by RJ Lipton...?), but it was general speculation on why mathematicians are actually interested in formal proofs of their theorems.

A recent experience of mine in my own research seems particularly relevant here, and I'm going to try and explain what happened (this will, at the very least, satisfy my promise to Michael Brough at about the same time of getting some more serious mathematics into my blog). This is pretty much the first time I've ever tried to explain something technical in a non-technical setting, so let's see how it goes...

Click here to read the rest

Writing Mathematics: Formalism obscures intuition?

I have read quite a large percentage of the books that are currently widely available on the topic of writing mathematics, and several that aren't. I've also been involved in teaching the Mathematical Writing course run at the QMUL. There is one serious issue that I've encountered: the main problem seems to be that most people start with the assumption that you already know everything that you're going to write, and the basic structure of your proof, but don't know the best order to put the words on the paper. In my experience, this is far from true.

I'm currently writing up my thesis, and have several new ideas which I've never written down in complete formal detail before. This process is difficult. The ideas in are generally simple enough that I think I could explain them to reasonably bright 15 year old in about 10 minutes given a piece of paper, but sufficiently abstract that to write them down formally has involved pages of writing that even I barely understand. These are proofs that I can (and have) run through in the pub with non-experts on the back of a napkin, and they translate into 10 pages of symbols that even I can barely follow. (I'm working on a post containing an example, but it will require drawing some pictures, so I'll probably get round to it some time over the weekend).

It is all very well saying that one should write an informal verbal summary of what you are going to do before you do it, but when the formalism is so far removed from the key idea, this becomes difficult. Also, when you are writing a maths paper, you have to make damn sure that the "summary" is still technically accurate. There can be no hand-wavey 'look, it just works' and (importantly) no interaction with the audience - you dont' know if they 'get it', so you have to put down everything it might take for them to get it. In my experience, the heavy formalism has often meant that the proofs I write down end up so complicated that I'm not even sure I would 'get' them if I had to pick up the ideas directly from reading the papers. The formalism masks the intuition, and the intuition isn't quite formal enough to be appropriate for the paper.

This is an interesting topic. It's one that several people have no doubt spent a lot of time thinking about, but it's not one that seems to be discussed. Even in a mathematical writing course, the tendency is to focus on technicalities. To be fair, this is usually adequate for undergraduates, and you do need to get the technical stuff right even when writing down simple ideas, but this is something that needs addressing: exposition of mathematics is a difficult skill, and probably one that there's not enough focus on in training academics.

Maybe not everyone has this problem - maybe not everyone has elementary ideas which don't translate nicely into formalisms - maybe their ideas are inherently more technical, or maybe they're just better at finding formalisms than I am - but I do find it interesting that it's happened to me almost every time I've tried to write out a formal version of a proof that I've created myself.

PS - the vast majority of this post was written over a year ago - the posting of it was sparked by having a discussion with Andy in which he mentioned that he is currently having exactly the same problem, and by me finally figuring out how to access the drafts of my old posts...

Advice to a First Year Me

The first, and I think most important, thing that you need to know is that research is *hard*. Really, really hard. You've done an undergraduate degree already, and that probably wasn't hard. You probably sat in lectures following everything the lecturer said, and the longest you've ever been stuck on an exercise is probably measured in minutes, or at worst hours.

Research is not like that. You will spend months thinking about things and getting nowhere. You may think you've understood that, but read it again, and try to actually understand what it means, I do literally mean months, and I do literally mean nowhere.

Your progress will come in fits and starts, and the fits and starts will be separated by long periods of nothing. Months of nothing is hard work emotionally. Almost everyone I know who has done a PhD in maths has complained at some point about the long periods of nothing. This even includes those people who I actually have managed to give this advice to before they started, and it even includes myself, after having given out this advice. You will be depressed because you don't feel like you're achieving anything. Don't worry, this is normal!

The following is approximately a graph of my perceived progress so far in my PhD thesis. the time axis doesn't have a scale, but it is roughly linear, and spans 3 years. Note that the flat bits really are genuinely flat and, if I'm perfectly honest, the spikes probably aren't quite spiky enough.

Note that I'm *not* saying that the time spent doing nothing is literally wasted. Obviously you need to check which ways of proving something don't work before you can find the ones that do, and, perhaps more subtly, there are so many hidden connections buried in mathematics that any time at all spent reading any mathematics is quite likely to turn out to be useful to you one day (I'll have an example of this in another post I'm preparing for later in the week). But it sure doesn't feel like that when you're going through the periods of nothing.

You are now hitting your limits. You are going to be doing something that is genuinely hard work. This is going to feel like hard work. Be ready for it.

The Prisoners and the Chessboard Solution

 Two weeks ago, I posted the following puzzle:

So, the evil guy who kidnaps people and sets them bizarre mathematical tasks which they have to complete or be executed has kidnapped you and your friend Bob. He sets you the following bizarre mathematical task:

I will take you into a room in which there is a chessboard. There is a coin on every one of the squares of this chessboard, showing either heads or tails. I will then tell you which one of the squares has the key to the door of the prison under it. After this, you will be allowed to turn over exactly one of the 64 coins and then be escorted from the room.

Immediately after this, Bob will be brought in, and will be asked to find the key. If he can locate it you're both free to use it to unlock the prison and leave. If not, you'll both be eaten by rabid wildebeest.

You and Bob are free to discuss a strategy before you are taken into the room with the chessboard - what should you do?

Adrianna emailed me a (sort of) solution less than a few hours later. I've posted my solution (and hers) below the fold).

Click here to read the rest

The next number in the sequence

We'll start today's post with what I think is a relatively easy set of puzzles. What is the next number in each of the following sequences?

a. 1 2 3 4 5 6 7 8 9 10 11 12

b. 1 1 2 3 5 8 13 21

c. 8 8,000,000,000 8,000,000,008

d. 31 28 31 30 31 30 31 31 30 31

e. 3 1 4 1 5

f. 1 11 21 1112 3112 211312 312213 212223

g. 1 1 4 1 9 2 16 3 25 5 36 8 49 13

Click here to read the rest

Iterated Prisoner's Dilemma

Just a little note to say that we're running an Iterated Prisoner's Dilemma competition in our department. If anyone is reading this, and would like to enter a strategy, they're more than welcome. I've already written a description of the tournament, which you can find here, so I'm not going to write it again.

Incidentally, while writing that website, I decided I should probably put something on my academic page, you can read what I decided to write here - I'll be updating it with a few bits and bobs that I've done over the past few years (mostly the outreach stuff I've done) as soon as I get back from Jamaica. If it looks familiar, that's because it looks exactly like Andy's academic homepage - thanks for the template Andy!

The Prisoners and the Chessboard

There's a long tradition in mathematics of puzzles in which someone asks some prisoners to perform some bizarre task. Usually, if they can't complete the task, they are to be executed. Here is one of my favourite examples of the genre (which I only heard for the first time relatively recently).

So, the evil guy who kidnaps people and sets them bizarre mathematical tasks which they have to complete or be executed has kidnapped you and your friend Bob. He sets you the following bizarre mathematical task:

I will take you into a room in which there is a chessboard. There is a coin on every one of the squares of this chessboard, showing either heads or tails. I will then tell you which one of the squares has the key to the door of the prison under it. After this, you will be allowed to turn over exactly one of the 64 coins and then be escorted from the room.

Immediately after this, Bob will be brought in, and will be asked to find the key. If he can locate it you're both free to use it to unlock the prison and leave. If not, you'll both be eaten by rabid wildebeest.

You and Bob are free to discuss a strategy before you are taken into the room with the chessboard - what should you do?

Unit Fail in the Evening Standard

From an article by Anthony Hilton in the Evening Standard today:

It is very difficult to believe that the capitalist system has been seriously improved by the arrival of computers able to trade in a microsecond and to exploit movements in share prices visible for just that amount of time.

It is even harder to believe that it will be improved further when the next generation of technology — including that of the stock exchange itself — will increase the available buying and selling speeds to milliseconds — millionths of a second.

Do they even employ any copy editors any more? And why would you pay this guy to comment on this sort of thing?

Monty Hall, Monty Hell and Tuesday's Boy

Consider the following problem (which I've quoted from the wikipedia page):

Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to you, "Do you want to pick door No. 2?" Is it to your advantage to switch your choice?

 Well. Is it?

Click here to read the rest

Tube or False (or just nonsense)

There is a new advertising campaign on the London Underground... I'm not entirely sure what it's advertising, as everyone who sees the posters is already on the Tube and presumably already aware that it exists. Anyway, the campaign is entitled Tube or False, and consists of 8 statements about the system for us to guess whether they are true or false.

One of these is:

Every week, our escalators travel the equivalent of 2 times round the world. 

I'll post a spoiler under the fold, but pause for a second to decide if you think that's plausible.

Click here to read the rest

Queues

I have two things to say about queues today. The second one is inspired by Steven Landsburg, and I was reminded of it by the fact that I spent 40 minutes of today standing in a queue to get on the London Eye. Time which was essentially deadweight loss to the world.

The first is simpler, and I was inspired to write it by AJ Jacobs. AJ Jacobs is the author of The Year of Living Biblically, and I'm currently reading his new book "My Experimental Life". He's one of those journalists, who does crazy things and writes about them (he claims because he has nothing interesting in his past to write about - although why he can't just make stuff up like everyone else, I don't know). Anyway, the chapter of the book I was just reading is called The Rationality Project. Jacobs, having read Predictably Irrational and Nudge (but no doubt not having actually read Tversky and Kahneman) decides to rid himself of all of his irrational biases. He fails in funny ways, of course, and has a few interesting things to say along the way.

Anyway, at the end of the chapter he has a few points to make. Things he does differently since spending a month trying to be as rational as possible. Some of these are sensible: "I read menus from the bottom up" (because they are designed to be read from the top down - and are carefully designed to 'nudge' you into buying whatever the restaurant wants you to buy that day); "I spend a few minutes each week reading Michelle Malkin's conservative musings" (because he disagrees strongly with Michell Malkin, and only reading things by people you agree with is a sure way never to change your mind).

The one that I take issue with is this: "I make a note every time I'm in a fast moving grocery line". Now, the idea behind this is obvious:

We all are predisposed to notice and remember the bad stuff... when we're on a checkout line behind an eighty-two year old man paying with a sack of pennies and nickels.

But it's a really, really bad example to choose of this particular phenomenon because we actually do spend more time in slow-moving queues than in fast-moving queues. This is obvious once you think about it: slow-moving queues move slower! So if all queues were the same length, but with random speeds (which were impossible to predict before you join) you would still spend most of your time in slow-moving queues. You're just as likely to choose a fast queue, but every time you do choose a fast queue, you get to the front of it quicker.

If you want to avoid believing in the signficance of random events, you should make a note of every time you take your umbrella out and it does rain (or choose not to take it and it doesn't). You should remember every time you have a really strong feeling that you're about to get a phonecall from your grandmother and then you don't, or every time you meet someone who doesn't share your birthday.

I said I had two things to say about queues, but it's late, and I've spent longer than I expected writing about that one. I'll do the other one tomorrow.

How not to draw a graph

The Daily Mail had an article earlier this year reporting some research which stated that people are happy at the age of 15, gradually drop off in satisfaction until the age of 45, then get happier again until a peak at 74. There are a few potential problems I can see with the research (asking people to self-report on their happiness has always seemed pretty dodgy, and have they properly controlled for some years just being less happy than others?) But that's not the issue here. The issue is the "graph" they used to illustrate the research.

Let's have a look at it:

Yes, really. That is a genuine graph directly from the website (in fact, that picture is being hosted on their website, I couldn't be bothered to download it). Look at the scale... The horizontal axis doesn't have one! The distance from 15-40 is the same as the distance from 40-50! Why would anyone make the graph look like that? Would they really be unable to make the same point using a graph with sensible axes?

Let's have a look:

That doesn't really look any less impressive (apart from the fact that I drew it in about 10 minutes in GIMP, and the Daily Mail hires qualified graphics people), and the patterns are still pretty clear. The slight upward trend in the 40's starts before 46 (but that's also true on the Mail's original) and the *massive* peak at 74 does start to look a bit suspicious... could it be a small-numbers effect?

Why did the Mail choose to go with the inumerate version? Apart from the obvious, I genuinely can't see how theirs is better than mine on any dimension, and it's pretty obvious in which ways it's worse.

HT to Dave in the comments at BishopBlog

'Minor Delays'

I have spent a long time wondering exactly what is meant by 'minor delays' on the Tube. They have a fixed amount of track, and a fixed number of trains running on it. The only way for their to be delays is for the trains to slow down... how is it possible that 'a customer incident' which happened over an hour ago is still causing the trains to run slower?

I can just about see why there are delays at peak times (the trains run slower because they spend longer in each station) but how do 'delays' propogate through the system? I can see how there might be bunching, but 'delays' (or is that what they mean by delays?)?

Bridge Probability

I've been re-reading Victor Mollo's classic bridge book 'Card Play Technique: the art of being lucky'. I've come, once again, to the sections about probabilities that I just can't get my head round. Consider the following passage:

If the Ace of Clubs is right, all is well. If not, the contract will depend on guessing the diamonds. How then, should we set about it? The man in the street will draw trumps quickly, sway in his chair slowly and mutter something like: "Well, it's six of one and half a dozen of the other".
But is it?
To the expert there is a vital difference between the Club and the Diamond positions. The latter will be the subject of guesswork. The former lends itslef only to prayer. The Club must be played first, and the reason is that it will provide a clue to the Diamonds.
If East has the Ace of Clubs, West will be credited with the Ace of Diamonds.

There is similar reasoning throughout the book, just one hand later:

West has pleaded guilty to 7 points hearts and to the King of Diamonds: 10 in total. East has only 3. It is more likely that the defender's high-card strength will be divided between them 10-5 than 12-3. Therefore the best chance is to play East for the King of Clubs.

I just can't believe that this reasoning is valid. The deck doesn't know which cards we assign points to, so it is no more likely to put the Ace of Clubs in a different hand to the Ace of Diamonds than it is to put, say, the Three of Clubs in a different hand to the Ace of Diamonds. Similarly, while it is true that a 10-5 distribution is more likely, a priori, than a 12-3 distribution, surely this ceases to be the case when you've alread placed the remaining points 10-3 (assuming that the principle of Vacant Spaces doesn't come into play - ie, every player has followed with a small card to each of his partner's honours).

To repeat: the deck doesn't know which cards we assign points to!

Maybe this sort of reasoning is a useful shortcut to some valid reasoning, or maybe I'm missing something (I certainly hope so). Can anyone shed any light?

Whacky Formula: The Happiest Day of the Year

You might think that all the Irish people you saw smiling last Friday were just experiencing some good old-fashioned Schadenfreude at the abysmal perfomance of the England football team, but no, they'd probably been reading the Examiner. If they had, they'd've found out that Dr. Cliff Arnall has some 'research' explaining why June 18 is the happiest day of the year. He has a formula, in fact, for telling us exactly how happy every day of the year is.

The ex-Cardiff University lecturer’s complicated formula is: O + (N x S) + Cpm/T + He.

My, that does look complicated... what on earth could it all mean? Luckily there's an explanation for the formula in the article (although no explanation for why (N x S) is in brackets. You can read it below the fold.

Click here to read the rest

Correlations vs. Causation.

A while ago, I said I'd write a post with my favourite "things which are correlated". Well, now I've finally figured out how to do page-breaks, I guess it makes sense to write it now. Here are my favourites. Any suggestions as to why they might be correlated? (answers below the fold)

1. Countries which put Fluoride in the water have much higher cancer rates than countries which don't.
2. Schoolchildren's shoe size is very strongly correlated with their performance on spelling tests.
3. The number of shark attacks on a given day is strongly correlated with ice cream sales.
4. You are much more likely to wake up dehydrated if you sleep on a sofa than if you sleep on a bed.
5. The number of prostitutes working in a city is positively correlated with the number of policemen.

Click here to read the rest

Why is the suicide rate so low at Foxconn.

Foxconn is a chinese company that you may have heard a lot about recently. There has, according to the BBC been "a string" of suicides there recently. According to the Times there is a "spiralling suicide crisis". Reuters says there has been "a spate of employee deaths".

10 people have committed suicide this year at a particular Foxconn factory in China (where a lot of parts for the iphone are made, apparently, which I think is why this is supposed to be a news story). The factory employs 300,000 people. I make that a suicide rate of (approximately) 7 people per 100,000 per year. The suicide rate in China is about 13 people per 100,000 per year. In other words, working for Foxconn cuts your chance of committing suicide almost in half!

How is it possible for every single person everywhere in the world's media to have gotten this story exactly backwards? There are a lot of very clever people suggesting "explanations" for the suicide rate at Foxconn. Unfortunately, they're trying to explain why it's so high, when they should be doing the opposite! This should lead us to seriously doubt these people's 'explanations' when the phenomena they're describing happen to be real. If your theory can explain anything, it has no explanatory power at all.

However, I think the main story here is how this became a story. What is going on? Why did someone decide to report the suicide rate at Foxconn instead of, say, the suicide rate among Tesco employees (another company which employs around 300,000 people, at least in the UK). And how did no-one notice that what they were reporting was in no way interesting.

What exactly are confidence intervals?

So, a friend (who would probably prefer to remain nameless) is currently looking for jobs in finance, and sent me some sample interview questions that they like to ask people. One of them is the following:

A drug trial gives the result that the drug works better than the
placebo, with 95% confidence. What exactly does this statement mean? What further assumptions are needed to be able to deduce that the probability of the drug working is actually 95%?

Now, I wasn't entirely sure I knew the answer to this question. I thought I did, but despite having studied rather a lot of statistics, I don't think anyone ever actually told me what a 95% confidence interval was, so I did what everyone does faced with such a situation, and checked the wikipedia article. Wikipedia is singularly confusing on the matter, but it gives the answer as roughly the following.

Click here to read the rest