Multiple choice questions, part two

Posted on 13-10-2013

In my previous blog post I gave an example of what I thought was an excellent multiple choice question, taken from the British Columbia leaving exam. It’s as follows:

15. How did the Soviet totalitarian system under Stalin differ from that of Hitler and Mussolini?
A. It built up armed forces.
B. It took away human rights.
C. It made trade unions illegal.
D. It abolished private land ownership.

In the comments on that post, Barry Naylor asked exactly why I thought that this question required higher order thinking. Here is why.

Firstly, the question itself is a good one. That would be the case if there were a short or long open question. It is asking pupils to compare and evaluate, and it requires a lot of knowledge about three different historical regimes – in fact, arguably it requires knowledge of even more types of political regimes, because in order to properly understand totalitarian regimes you need some idea of what totalitarianism isn’t.

Secondly – and here are the factors unique to the question as a multiple choice question – the distractors are carefully designed so that they home in on two important and frequent misconceptions.

Misconception one is that German, Italian and Russian totalitarianism were all the same. This is a very basic misconception, but one that I think is quite common. Popular culture is saturated with the images of these regimes and dictators, and most pupils easily understand that all three regimes were evil. But I think very many assume that they were all evil in precisely the same way. For the pupil who has made this misconception, all four statements will seem equally likely. They all seem like bad things, and these dictators all did bad things, so it will be hard for them to see that there was one bad thing more specific to one regime.

Misconception two is to do with the nature of communism and Soviet Russia. A pupil with a bit more sophistication will understand that Hitler and Mussolini were fascists, but that Stalin was a communist. They will understand that communism is based on collective ownership, and that communism is generally friendly to trade unionism – unlike fascism. They might therefore assume that, given that the Russian regime under Stalin was communist, the difference between the different regimes was option C.

Then, the correct answer, Option D, requires a relatively sophisticated understanding of a fairly complex idea – that of private property. There’s a famous (and I think probably apocryphal) story about Soviet bureaucrats visiting Britain after the fall of the Berlin Wall and asking their British counterparts how they set prices. The British showed them round Smithfield Market, the stock exchange and a supermarket. Then they ended up back in Whitehall, and the Soviets said ‘OK, that was lovely. Now where’s the office where you set the prices?’

The reverse problem is true with us. If you have grown up in a capitalist economy the very concept of private property is, paradoxically, quite hard to appreciate. I’ve had discussions about this point with smart 6th formers who just don’t get it. ‘What do you mean, you can’t own something? You’ve got to be able to own stuff! You can’t get rid of owning stuff!’

The problem in both cases is ideology – or the fish in water problem, as David Foster Wallace puts it. ‘There are these two young fish swimming along and they happen to meet an older fish swimming the other way, who nods at them and says “Morning, boys. How’s the water?” And the two young fish swim on for a bit, and then eventually one of them looks over at the other and goes “What the hell is water?”’

If you grow up in the modern West, private property is the water that surrounds you.  By asking about this concept, this question is testing whether you have properly understood one of the alien things about a different society.

So that’s why I think this question promotes higher order thinking. Of course, essays promote higher order thinking too. But essays are much less efficient. You can’t cover so many ares if you only assess using essays. In the time it takes to write one essay question you could probably answer 30-odd multiple choice questions of this type. Since I wrote this previous post, a colleague has told me that trainee doctors often have to sit multiple choice exams like this – along the lines of: patient 1 has problem x, y and z and interesting features a, b and c. What intervention would you recommend? Such questions are a highly efficient way of testing higher order thinking.