\documentclass[11pt]{article}



%% LaTeX Preamble - Common packages
\usepackage{geometry}                % See geometry.pdf to learn the layout options. There are lots.
\geometry{a4paper}                   % ... or a4paper or a5paper or ... 

\usepackage[usenames]{color}

\usepackage[utf8]{inputenc} % Any characters can be typed directly from the keyboard, eg éçñ
\usepackage{textcomp} % provide lots of new symbols

\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{epstopdf}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}

\usepackage{flafter}  % Don't place floats before their definition
%\usepackage{topcapt}   % Define \topcation for placing captions above tables (not in gwTeX)
\usepackage{natbib} % use author/date bibliographic citations

\usepackage{amsmath,amssymb}  % Better maths support & more symbols
%\usepackage{bm}  % Define \bm{} to use bold math fonts

%\usepackage[pdftex,bookmarks,colorlinks,breaklinks]{hyperref}  % PDF hyperlinks, with coloured links
%\definecolor{dullmagenta}{rgb}{0.4,0,0.4}   % #660066
%\definecolor{darkblue}{rgb}{0,0,0.4}

%\hypersetup{linkcolor=red,citecolor=blue,filecolor=dullmagenta,urlcolor=darkblue} % coloured links
%\hypersetup{linkcolor=black,citecolor=black,filecolor=black,urlcolor=black} % black links, for printed output


\usepackage{hyperref}
\hypersetup{breaklinks=true,
pagecolor=white,
colorlinks=true}
\urlstyle{rm} %so it doesn�t use a typewriter font for url�s.

\usepackage{memhfixc}  % remove conflict between the memoir class & hyperref
% \usepackage[activate]{pdfcprot}  % Turn on margin kerning (not in gwTeX)
\usepackage{pdfsync}  % enable tex source and pdf output syncronicity


%\url{http://ihome.ust.hk/~tanjim/verylongaddresslikethisone-111111zxzxzxzxzxzxzx
%zxzxsqut_high.pdf}




\newcommand{\hlink}[2]{\emph{#1}  \url{#2}}

\renewcommand{\abstract}[1]{\parbox[t]{80 ex}{#1}\vspace{2 em}}

\newcommand{\figref}[1]{(see figure~\ref{#1})}

\bibliographystyle{plainurl}

\begin{document}

\title{A Random Rambling Rant}
\author{Mike Pearson}
\date{}
\maketitle

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=5in]{convincing.png}
\caption{\hlink{Convincing}{http://xkcd.com/833/} by Randall Munroe}
\label{convincing}
\end{center}
\end{figure}

When Hans Rosling started to use his amazing data driven graphics to tell the world that it was not the world it thought it was, studying statistics suddenly became interesting. Even better, it became accessible. As Rosling demonstrates with his deft use of large international databases, animations, and internet video~\cite{rosling_hans_2006}, technology now makes it possible to use data to tell exciting stories. In days gone by those stories would have lain undiscovered, 
%buried in acres of numbers, 
neatly tabulated, filed and locked away. 

There has never been a shortage of people telling stories, but now
%, if we are minded to, 
we can look at those stories with a degree of scepticism, find the underlying data, and make our own judgements. And if this all seems too much, we can rely on data savvy journalists ~\cite{harford_bbc_2011} and bloggers~\cite{goldacre_bad_2011-1} to pick up the trail. Still, a degree of numeracy together with common sense and an ability to spot a dodgy argument are statistical skills we all, as members of a democracy, should possess.

But what is statistics? In mathematics we often say `probability and statistics' in the same breath, and perhaps too often we convince ourselves that they are just different aspects of the same thing. It's perhaps a little simplistic, but in a sense I believe them to be opposites. Probability starts with a model, a theory, and predicts possible future outcomes; whereas statistics starts with the outcomes, the hard, unchangeable data, and attempts to validate a model or theory. 

%This inversion is really important to keep in mind because the possibilities for confusion are endless. A good model should predict the data we obtained with a high probability. It's easy to confuse that probability with what we'd really like to know -- the chance that the model is correct. In general, \begin{equation}P(\mbox{data obtained} | \mbox{our model is correct}) \ne P(\mbox{our model is correct} | \mbox{data obtained}),\label{confusion}\end{equation}  where $P(A|B)$ means the probability of $A$ happening given that $B$ has already happened. 

%A big part of the over simplification here is that statistical thinking has to include a lot more than just analysing numbers and data. I'm going to leave the humour to Randall Munroe, the creator of the xkcd comic \figref{convincing}, who makes the point so much better than I could.


%Data has provenance which cannot be ignored. Questions about funding, motivation, and methodology must be answered before attempting to make sense of any numbers. This is the vital non mathematical part of statistics and it's crucial if we want to arrive at anything close to the truth. And once the  numbers are understood there remains the non-trivial job of communicating them to others. 


%I'm sure that much of our confused thinking related to inequality \ref{confusion} comes from statistical techniques taught in school, and reinforced in media reports. In hypothesis testing we have alternate complementary models, usually called the null hypothesis $H_0$, and the alternate hypothesis $H_1$. We then proceed to calculate the probability of our data, given $H_0$. If this probability is below some arbitrary threshold (often 0.05), we reject the null hypothesis, saying that the alternative $H_1$ was found to be statistically significant at the 5\% level. Notice that the technique calculates the probability of the data given $H_0$ rather than the probability of $H_0$ given the data. The logic is back to front and extremely difficult to justify at school level even though the calculations are mechanical. Of course, if we were to attempt to find what we really want to know -- the probability of $H_0$ given the data -- the calculations would probably become very difficult even though the logic would then be clear. Modern statisticians often overcome this complexity of calculation problem using computer simulations, known as Monte Carlo methods. They can then get the logic right.



%In simple situations it is possible to get things the right way round without too much calculation. Take a good look at the carefully constructed school activity `Bayes Ice-Breaker' by Alan Jessop~\cite{jessop_bayes_2010}. 
 
% `Screening for disease and dishonesty~\cite{spiegelhalter_screening_2009} provides another easy to digest example of this Bayesian reasoning which is very necessary when you only have the results of one test to go on.

\section{Telling stories}

There is no question that Rosling's popularity is mostly due to his ability to tell stories with data. His excitement and enthusiasm is infectious and has led to a flowering of data led journalism, most successfully in the New York Times and in the Guardian. 

The Guardian Datablog~\cite{the_guardian_data_????} is a brilliant resource for teachers wanting to base lessons on topical data. The datablog publishes the data behind the current headlines. For example, with drought and famine raging in the Horn of Africa, you can download a spreadsheet showing the humanitarian aid to the region collected by the United Nations. PJ Harvey is in the newspapers as I write this for winning the Mercury Prize again. If that's of interest to your students, go to the data blog and download her sales figures.

It's not easy to find stories in data. Numbers in tables rarely leap out at you. One of the best ways to find the stories is to visualise the data, and it's really exciting to find one. One that I remember in particular appeared while working on a survival animation for the Understanding Uncertainty website. I was using the Human Life Database~\cite{max_plank_institute_human_2007} maintained by the Max Plank Institute to extend the animation to all the countries it contains, going back as far as the records allowed. You can see the resulting animation here~\cite{pearson_mike_survivalworldwide_2009}. Take a look at France, graph male survival, switch the scale to logarithmic, and then slowly wind the clock back to 1806. We have to thank Napoleon that we can take this dataset back so far, and we probably have to thank him too for some of its interest. The main story here is about war and its effect on survival rates of young men, but there are other gems in this dataset. Switch to women, and look at their survival rates from 1917 to 1919. Switch to the United States of America in the same period. You are probably looking at the post war flu epidemic which hit both young women and men. Now switch to Russia and look at what happened to male death rates after the fall of communism. What could explain that?

Wars have featured in statistics ever since Florence Nightingale drew her famous rose diagrams of British deaths during the Crimean campaign~\cite{short_nightingales_2009}. She needed to tell a story about unnecessary deaths in order to gain support for better sanitary conditions.

Modern journalists have taken up the challenge -- often with different motivations -- using technology to tell more personal stories of loss alongside the overall numbers. See for example the New York Times visualisation `Faces of the Dead'~\cite{new_york_times_faces_2010} for Iraq. In Britain, the war in Afghanistan has attracted similar attention. See the Guardian datablog for data and visualisations~\cite{the_guardian_british_2011}.

From war to sex. Always of interest to the sixteen year old and now with no shortage of data. The Guardian datablog once more obliges with a geographical breakdown of abortion statistics~\cite{the_guardian_abortion_2011}.  If you are brave enough to go there -- the site may offend -- the American online dating agency OKCupid now has a statistics blog called OKTrends~\cite{okcupid_oktrends_????}. This plots data from OKCupid's large database of users -- so expect bias here! It's clear that Rosling's influence has even reached the online dating scene and is now providing an unexpected career path for budding infographics designers. The less adventurous amongst you will have to make do with another xkcd gem \figref{datingpools}.

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=5in]{dating_pools.png}
\caption{\hlink{Dating Pools}{http://xkcd.com/314/}}
\label{datingpools}
\end{center}
\end{figure}

\section{Statistics in School}

So far I've neatly avoided talking coherently about what might happen in the classroom. 

We certainly have a good selection of problems for you to use in this October 2011 issue of NRICH. I love the idea behind `If the World were a Village'~\cite{nrich_if_2011} -- you might compare this to David McCandless's `if Twitter was 100 People'~\cite{mccandless_if_2009}.

 `Charting Success'~\cite{nrich_nrich:_2011} is a great stepping off point if you want to introduce your class to the world of infographics -- McCandless~\cite{mccandless_information_2011} will get you started again but follow his links to other practitioners as he is often more interested in aesthetics than communication. You may notice that social networks such as Twitter and Facebook are a favourite hunting ground for infographics designers. 
 
 The NRICH problem `Domesday Project'~\cite{nrich_nrich:domesday_2011} invites you to examine data on pet keeping gleaned from the project that the BBC ran in 1986. There are useful skills to be picked up here - see Pereira-Mendoza~\cite{pereira_mendoza_graphing_1995} for a very relevant article on graphing in primary mathematics.

I have not seen a better introduction to work on averages than `Wisdom of the Crowds'~\cite{grime_wisdom_2011} a video by James Grime (YouTube's SingingBanana) and David Spiegelhalter . Listen carefully to the discussion on the various averages that could be used to determine the crowd's answer. This could be a good experiment to conduct in a school -- especially if you can involve the whole school, and offer some prize for nearest guess to minimise silly responses. 

The Royal Statistical Society Centre for Statistical Education (RSSCSE) publishes a Teaching Statistics journal, maintaining a useful `Best of' series~\cite{rsscse_getting_2011} which are all well worth a read.

They have also published Census At School and Sports at School, but I must admit that I'd classify the Census at School data tool~\cite{rsscse_censusatschool_2011} as a nice try rather than an unqualified success. For me, this slick drag and drop charting sidesteps creative learning. 

I tried hard to find an interesting question to ask about the offered data set, and failed. Tools that I would have liked to use such as a 2D scatter graph and maps were unavailable. There was no possibility of time series analysis and the countries on offer were too similar. The exception -- South Africa -- was missing interesting data. 

To provide interest we want to find our own data. Where should we start?

\section{Finding stuff}

We have to start at the top so let's not forget the internet search engine. My favourite is unsurprisingly, Google. After all It's used by xkcd too \figref{dangers}. Search engines will lead you to facts and figures sites like Wikipedia. 

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=4in]{dangers.png}
\caption{\hlink{Dangers}{http://xkcd.com/369/}}
\label{dangers}
\end{center}
\end{figure}

Google and Wikipedia are incredibly useful, as I'm sure you are aware. However, they don't often lead you to structured, tabulated data. Harvesting numbers from web pages can be a time consuming and painful process. It's a shame that Google shut down their Google Squared service which was cleverly designed to assist us in creating tabular data by scraping web pages. 

That leaves Wolfram Alpha. Alpha is the result of a huge effort to build a \emph{computational} search engine. It features natural language input, access to linked data sources, and the power of Mathematica running on the server,  and it is very very clever. It does blurt out what it knows rather than what you asked it so you don't often get fresh answers by asking your questions in other interesting ways. It also tends to do a lot of analysis and graphing for you, which may well preempt your lesson plan. So use with care. Here~\cite{wolfram_uk_2011}, for example, is the result of a query on the UK population. Alpha is usefully conscientious about quoting its data sources.

\section{Cautionary tales}

This is probably a good moment for a word of caution. As we all know, you can't believe everything you see on the internet.

I've been working on the NRICH and Understanding Uncertainty web sites for some years. Over that time I've wasted many hours chasing problems that only show up in Internet Explorer. So like many, I was primed to believe a recent report~\cite{myslewski_its_2011} that a study had shown that Internet Explorer users were dumber than most. The report was of course an easily discoverable hoax dressed up to look like it was backed by data. There is an excellent, if dated, text that tells you all you need to know about `How to Lie with Statistics'~\cite{huff_how_1991}, reviewed by Helen Joyce on our sister Plus website~\cite{joyce_how_2009}, but in this case the hoax was easily unmasked using sound statistical thinking~\cite{spiegelhalter_spotting_2011}.

One of the pitfalls of using public data is known as the fishing expedition. As we explored in the November 2010 issue of NRICH, if you go looking for \emph{anything} interesting, chances are you'll notice \emph{something}, even in a purely random dataset. This is beautifully illustrated by Randall in `Significant' \figref{significant}. Hint: count the colours. 

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=3.2in]{significant.png}
\caption{\hlink{Significant}{http://xkcd.com/882/}}
\label{significant}
\end{center}
\end{figure}

\section{Must Read Books}

My one concern about mentioning texts such as `How to lie with Statistics'~\cite{huff_how_1991} is that the title reinforces the oft quoted `Lies, damned lies, and statistics' (Disraeli?, Twain) public perception. It definitely falls in the must read category however.

Edward Tufte's now classic text `The Visual  Display of Quantitative Information'~\cite{tufte_visual_2001} has a more positive approach with many examples of excellence in data visualisation discussed before he mentions less successful attempts in a chapter on graphic integrity. A useful rule of thumb promoted by Tufte is that a data visualisation should dedicate most of its ink to data. Even grids and axes are to be minimised or eliminated if they don't carry useful data. It's a minimalist approach that has its own powerful aesthetic, first developed by Otto Neurath and Gerd Arntz~\cite{arntz_gerd_????} working in pre war Vienna.

A good practical text with guidance on creating data visualisations is `Visualize this'~\cite{yau_visualize_2011} by the author of the `Flowing Data' blog, Nathan Yau.

\section{Institutional Data }

The earliest use of the term \emph{statistics} recorded by the Oxford English Dictionary is in 1770. It helps to explain the origin of both the word - it's to do with nation states - and its early acceptance as a science. 
\begin{quotation}\emph{The science, that is called statistics, teaches us what is the political arrangement of all the modern states of the known world.}
[W. Hooper tr. Bielfeld Elem. Universal Educ. III. xiii. 269] 
\end{quotation}

The UK Office for National Statistics~\cite{ons_ons_2011} trumps the OED by tracing its history back to the Domesday book. 
That may be rather tongue in cheek, but the point is well made. The earliest statistics were data collected by governments about the states they administered, so we should expect to find a lot of useful data in national databases.

As I write this, the ONS is about to launch its new website so there's not much point in me explaining how to navigate the old one. One thing to watch though is that much of the data is in the form of downloadable reports published as PDF files. 
If you are looking for datasets do not despair - there's a good chance that these reports link through to spreadsheets, and the reports are there to supply the provenance needed to interpret the data. All very useful! 

The ONS is responsible for the national census.  The 2001 census feeds data to a Neighbourhood Statistics~\cite{neighbourhood_statistics_neighbourhood_2007} web site. Here you will find interesting data for your local area. I expect this will all change once we have the 2011 census results, but at the moment the Neighbourhood summary is a good starting point, providing a number of summaries and visuals on health, education, crime etc. 

For worldwide statistics, the OECD~\cite{oecd_organisation_2011} is a fantastic resource. Look for the page called \emph{Statistics from A to Z} for a useful set of links to spreadsheet downloads and to web pages displaying extracts from the real-time OECD database. For more detailed searches, try OECD StatExtracts~\cite{oecd_oecd_2011} where you can make queries on the database and export the results in Excel spreadsheets or in other useful formats. If you delve too deep into the OECD iLibrary you will begin to hit subscription content.

The United Nations has a statistics division~\cite{un_united_2011}.
You'll find there a home page leading you to many topical reports such as progress reports and charts on the Millennium Development Goals. Useful poster material here.

% I would invite you to compare these two YouTube videos - one produced by the UN explaining its \hlink{Gateway to Global Statistics}{http://www.youtube.com/watch?v=mtuSaqly0eU} 
%- and one on \hlink{teen statistics}{http://www.youtube.com/watch?v=FaGfNBM8o7I} that YouTube features alongside. 

The UN site has a public \hlink{data search facility}{http://data.un.org}, but it is surprisingly patchy. 
`Coffee' produces 12 documents from 4 sources, whereas `Cocaine' produces none at all. On the plus side, this is one portal where you can access public data from a wide variety of institutions such as the World Heath Organisation, the International Monetary Fund, National Governments, and the UN itself.

The US government places a lot of information in the public domain.  Even the CIA publishes its CIA World Factbook online~\cite{cia_cia_2011}. NASA especially has a wealth of interesting data together with support for schools access. Try for example `My NASA Data'~\cite{nasa_my_2011}.

One goal of open government is that decisions should be transparent, and there has been a trend recently for governments to publish the data on which their decisions are based. The Guardian has a useful index of these sites called `Government data sites around the world'~\cite{the_guardian_official_2011}. Our own government is present and correct at \url{http://data.gov.uk/}.

The Royal Statistical Society \url{http://www.rss.org.uk} has for some time been publishing its excellent public-facing Significance magazine~\cite{rss_significance_2011} and has more recently launched its GetStats Campaign \url{http://www.getstats.org.uk} which promises more to come. Don't miss our own work in Cambridge, supporting Professor David Spiegelhalter's Understanding Uncertainty \url{http://understandinguncertainty.org} website. Lots of excellent articles, animations and videos there together with an active blog.

I'm just dipping my toe in the water here. Think of an institution, (e.g. the WHO), then type its name into Google along with the word 'statistics' and you are away. Happy hunting.

\section{What have I left out?}

I'm going to end by pointing you at a couple of must-see videos. The first is `Professor Risk'~\cite{spiegelhalter_professor_2010} by David Spiegelhalter, and the last takes us back to the beginning -- another Rosling production -- The Joy of Stats~\cite{rosling_joy_2010}. 

And on a final sadder note, take a look at xkcd `Lanes' \url{http://xkcd.com/931/}. 

Communicating this stuff matters. So much so that we've contributed a review article to Science Journal on probability forecasts titled `Visualising uncertainty about the future' \cite{spiegelhalter_visualising_????}. 


%\section{Science}

%Science lies on the boundary of what we know and what we don't know. It's a place where judgements are made based on available data with the risk of being wrong. I rather like these pieces from Richard Feynman. First he's talking on \hlink{the likelihood of flying saucers}{http://www.youtube.com/watch?v=wLaRXYai19A}, and here he's talking about the \hlink{development of physical law}{http://www.youtube.com/watch?v=b240PGCMwV0}. 

%The themes of scepticism, disrespect for authoritarian statements, together with an insane ability to ferret out the numbers behind science stories in the media appear in the writings of Ben Goldacre in his Bad Science blog. His article on Fish Oil supplements\cite{goldacre_fish_2010} is typical. The arguments that Ben puts forward here are not simple to understand. The detail can be overwhelming and the numbers can be tricky. We need a kind of statistical numeracy to understand them properly. 


%\section{Visualisations}

%\begin{description}

%\item[Flowing Data] \hlink{Data Visualisation News}{http://flowingdata.com} FlowingData explores how designers, statisticians, and computer scientists are using data to understand ourselves better - mainly through data visualisation. Money spent, reps at the gym, time you waste, and personal information you enter online are all forms of data. How can we understand these data flows? Data visualisation lets non-experts make sense of it all. 

%\item[Visual Complexity] \hlink{Designer visualisations}{http://www.visualcomplexity.com/vc/} Data visualisation gone mad! A large database of innovative data visualisation examples. Many are beautiful, some tell a story. Plenty to discuss.

%\item[Many Eyes] This is an open service hosted by IBM which allows contributors to submit datasets for visualisation. \hlink{Many Eyes}{http://www-958.ibm.com/software/data/cognos/manyeyes/} allows explore data using your eyes.Upload, visualise, and share. 

%\item[Google Books] \hlink{Books Ngram Viewer}{http://ngrams.googlelabs.com} This is great fun. Type in a list of words and see how many times they appear by year in the Google Books archive. Then try to explain the result. 
%\begin{figure}[htbp]
%\begin{center}
%\includegraphics[width=5in]{moonmars-big.png}
%\caption{Comparing uses of 'Moon' with 'Mars' 1800 to 2000}
%\label{default}
%\end{center}m
%\end{figure}
%Read up on the background to this tool - the new field of 'Culturomics' - in Science in the article \hlink{Quantitative Analysis of Culture Using Millions of Digitized Books}{http://www.sciencemag.org/content/early/2010/12/15/science.1199644}.

%\item[Wordle] Make your own \hlink{word clouds}{http://www.wordle.net/} from your datasets here.

%\item[Maps] \hlink{See UK}{http://apps.seme4.com/see-uk} maps the UK 's crime, transport, and educational statistics. A simple viewer of some of the data at data.gov.uk, against an abstract view of the geography. The data can be normalised by population and area.
%\end{description}

%\section{Competitions}
%\begin{description}
%\item[Kaggle] \hlink{Current Competitions}{http://www.kaggle.com/}  is a platform for data prediction competitions that allows organisations to post their data and have it scrutinised by the world's best data scientists.
%\end{description}

%\section{Blogs and Online Communities}
%Tim Harford's \hlink{More or Less Radio programme and podcast}{http://www.bbc.co.uk/programmes/b006qshd} Excellent analysis of numbers in the news.
%See \hlink{40 Fascinating Blogs for the Ultimate Statistics Geek}{http://www.bschool.com/blog/2011/40-fascinating-blogs-for-the-ultimate-statistics-geek}
%Ben Goldacre's blog \hlink{Bad Science}{http://www.badscience.net} One man's hard hitting blog offering data driven insights on the modern world and its medicine.
%\section{Books}
%\begin{description}
%\item[Visualize This by Nathan Yau]  Written by the author of the FlowingData blog. Gives practical advice on designing data visualisations.
%\end{description}

%\section{Comics}
%\begin{description}
%\item[xkcd]  \hlink{Self Description}{http://xkcd.com/688/}
%\item[xkcd] \hlink{Significant}{http://xkcd.com/882/}
%\item[xkcd] \hlink{Correlation}{http://xkcd.com/552/}
%\item[xkcd] \hlink{The Data So Far}{http://xkcd.com/373/}
%\item[xkcd] \hlink{Convincing}{http://xkcd.com/833/}
%\item[xkcd] \hlink{Lanes}{http://xkcd.com/931/}
%\item[xkcd] \hlink{Extrapolating}{http://xkcd.com/605/}
%\end{description}

%\section{Television and Video}
%\begin{description}
%\item[The Joy of Stats] You may have seen this documentary recently. The \hlink{Joy of Stats}{http://www.gapminder.org/videos/the-joy-of-stats/} presented (inevitably) by Professor Hans Rosling.
%\item[Professor Risk] How \hlink{Professor David Spiegelhalter}{http://understandinguncertainty.org/node/604} interprets the chances of something bad happening to him each day.
%\item[Videos on Understanding Uncertainty] \hlink{Collected video offerings}{http://understandinguncertainty.org/view/videos}. I particularly like `Wisdom of the Crowds'\cite{grime_wisdom_2011} a video by James Grime (YouTube's SingingBanana).
%\end{description}

%\section{Software}
%You don't have to pay anything to get the best statistical software around. 

%\begin{verbatim}
%http://labstats.net/articles/overview.html
%\end{verbatim}


\bibliography{oct2011-article}


\end{document}
