I’m currently knee deep in both knowledge recall tests and PPEs as part of a whole school assessment week. It is about to get worse. I should be marking now. And I haven’t even got it that bad: I have six GCSE groups which actually means, due to allotted GCSE time, that I have fewer classes than most other teachers.
What’s been most noticeable so far, and it’s something I’ve mentioned before, is how students have reacted and responded to questions which, in retrospect, weren’t phrased clearly enough. I do think that we should ask the same question in various forms, but those questions have to be clear: we can’t let a poorly phrased question skew the result. We mustn’t be noisy. If we want our measurements to be valid, and if we want to ensure that students’ answers tell us something reliable, then we need to ask crystal questions.
For example, we asked this question of our GCSE students:
Name two jobs which Stresemann did in the German government.
The answers we wanted were that he was both Chancellor and Foreign Minister. The answers we received were almost all – until I rephrased for a different class – wrong. Or at least not the answers we wanted. Most students who did answer told us that he signed the Dawes Plan in 1924, that he introduced the Rentenmark and helped make living conditions in Germany tolerable.
Better questions might have been:
Which two positions did Stresemann hold in the German government?
Stresemann had two jobs in the government. What were these?
Stresemann was both in charge of the Reichstag and international affairs. What are the names of these two jobs?
An even better way to do this might be to use multiple choice questions, which we have done in the past and probably will do next time. However,this depends on what we want to assess: MCQs might skew answers because of a misreading by the student – this would be an error, though not a mistake even though our resulting data might suggest otherwise. For example, I might give the following four options as possible answers to a crystal clear variation of the previous question on Stresemann:
Secretary of State
These options might tell me about the quality of the students’ knowledge.
It would be dangerous, however, to give options which were too disparate if that original question wasn’t crystal clear. For example:
Signed the Dawes Plan
Option 2 is an understandable answer to that original question, but it’s not correct.
- Are your questions crystal clear?
- Are you testing what you want to mark?
- Have you narrowed the likelihood of students making errors?
- If you’re using MCQs then how disparate are the choices, and is this deliberate?