Closed questions and higher order thinking

I know that Andrew Old often writes about the way that open questions are often, wrongly, seen as superior to closed questions – ie, it’s seen as being better to ask pupils questions that have lengthy answers and many possible answers rather than those that only have one straightforward right answer. I think one of the reasons for this is that open questions are seen as being effective promoters of higher-order thinking. That is, asking a question like ‘Why was there a war in Europe in 1914?’ is likely to lead to higher quality thinking than ‘When did Britain and France sign the Entente Cordiale?’

I would make two points here. First, it is true that closed questions are very effective at testing facts and knowledge. However, ‘higher order thinking’ such as analysing, evaluating and synthesising is in fact made up of ‘lower order’ facts and knowledge. To know why the First World War happened you have to have a very good knowledge of chronology and knowing when the Entente Cordiale was signed is an important, if small, part of that wider question. Indeed, the question ‘Why was there a war in Europe in 1914?’ requires a lot of facts to be able to answer successfully, and  one of the things that allows you to be able to analyse a deep, open question like this successfully is how many facts you have and how well you know them. This is as true of the academic level as the classroom level. I am currently reading Christopher Clark’s book on just this question. In the introduction, he notes that one of the problems with this topic is just how many facts have to be understood and marshalled, and that they are all in so many languages.  One of the things which makes Clark’s book so brilliant is that he has spent several decades mastering very many of these facts.  There is no way that a 20-year-old could write a book like this, however innately smart and innately good at analysing they were. Clark’s superb and revisionist analysis of the causes of the First World War springs from his detailed mastery of the facts. His excellent analysis is a function of his wide and deep knowledge.

Second, closed questions can in fact be extremely good at testing analysis and evaluation – in fact, they can often be more efficient and effective than open questions. For proof of this, take a look at the British Columbia leaving exam. (The link is to an absolutely excellent report by Common Core which features extracts from curriculums and exams from around the world.)

The BC leaving exam in history asks pupils to write essays, yes. But it also asks them to answer multiple choice questions. Here’s a typical example.

15. How did the Soviet totalitarian system under Stalin differ from that of Hitler and Mussolini?
A. It built up armed forces.
B. It took away human rights.
C. It made trade unions illegal.
D. It abolished private land ownership.

I think this is an excellent question which very definitely asks for higher order thinking. The reason why I think this question is so good is that it tests a finer gradation of understanding. Everyone knows the Nazis and Soviets were evil, and because they were evil, it is easy for pupils to just think that their regimes were the same. And of course the regimes were very similar. But they were different in interesting ways too, and this question probes that. A pupil who got this question right would have understood something important. A pupil who didn’t would have misunderstood something quite important.

And it isn’t just this question. I went through all of the literature and history multiple choice questions on this paper and all of them made me think hard about deep issues. I couldn’t have rote learned or memorised the answers to any of them. So I would say that the closed questions can be very effective at promoting higher order thinking. Of course, I wouldn’t suggest only having these types of questions – but as I said, the BC exam has a judicious mix of these and essay questions.

I know another technical criticism of MC questions is that it is too easy to guess them, but I think this criticism can be simply countered by having a penalty for wrong answers and/or a very high pass rate.

Update: I’ve since written two more posts on this topic, here and here.