A population survey

A geographical survey: answer the tiny questionnaire and then analyse all the collected responses...

Exploring and noticing Working systematically Conjecturing and generalising Visualising and representing Reasoning, convincing and proving
Being curious Being resourceful Being resilient Being collaborative

Problem

Are you good at estimating? We have written a survey inviting you to estimate the answer to a question, and then compare your estimate with all the other people who have had a go.

Before reading on, please complete this very short questionnaire* on your own to contribute your estimate (and please just make a reasonable guess if you don't "know" the answers):

 



(Only fill it in once!)

Once you have filled in the questionnaire, click "Show" to see the results so far:

 

There are actually three slightly different questionnaires!

You filled in one of these chosen at random by the computer.

All three questionnaires ask: "What do you think the population of the Philippines is (in millions)?"  However, two of the questionnaires ask an additional "primer question" first.

On one questionnaire, the primer question reads:

 

"Do you think the population of the Philippines is more than 30 million?"



and on the other, it reads:

 

"Do you think the population of the Philippines is more than 70 million?"



What effect do you think the primer questions might have on people's estimates?

Click "Show" to see graphs of these different survey responses:

 

The results are shown in this comparative box plot (updated approximately every hour), where there is one box plot for each questionnaire.

 

Image
A population survey



In this graph, the "n =" values show the number of responses.  The red dashed lines mark 30 and 70 million, while the pale blue line shows the approximate actual population of the Philippines (about 105 million as of 2018).  The graph is cut off at 150 million (so larger responses are not visible, though they are used when calculating the box plots).

We can also distinguish between those respondents who answered "yes" to the primer question and those who answered "no", resulting in the following comparative box plot:

 

Image
A population survey



You can also see the raw data in this Google doc (which is continuously updated); the three sheets are for the three different questionnaires.

What is the impact of asking a "primer question"?

Why might this be the case?  What does this suggest about designing questionnaires?

 

You can download the R code used to generate these box plots: philippines.R; you can use R (or RStudio), a spreadsheet or other software to explore the data further if you wish.



 

* This anonymous survey is run through Google forms.  Google only provides NRICH with the date and time that the questionnaire was completed and the responses to the actual questions asked.  No other data is collected.

With thanks to Michael Posner of the Center for Statistics Education at Villanova University for suggesting this problem.



This resource is part of the collection Statistics - Maths of Real Life