An Enthusiasm Problem

Dr. Lee: Ben, I've been through your questionnaire, and I'm afraid it has what I'd call an enthusiasm problem. Almost every question is phrased in such a way that the respondent can only agree with you. Ben: Surely not every one. I did try to keep it neutral. Dr. Lee: Question six asks how much more useful they find the new timetable compared with the old one. You've baked the conclusion into the wording. Anyone who thinks it's less useful has nowhere to go. Ben: I see that now. I suppose I was so attached to my hypothesis that I couldn't see it. Dr. Lee: It's the commonest pitfall in the business, and, to be fair, you're in good company. Where the questionnaire does shine is the layout. It's clean, and nobody will abandon it halfway through, which is more than I can say for most I've seen. Ben: So do I scrap it? Dr. Lee: Scrap is rather strong. Rewrite questions six to nine, then pilot it with ten students from outside your own year group before you go any wider. Their confusion will be more informative than their answers. Ben: Do I have to postpone the main survey? Dr. Lee: Only by a week, I should think, though if the pilot throws up problems, I wouldn't be surprised if it took two. Don't let the deadline bully you into a hasty design. A flawed survey that arrives on time is still a flawed survey. Ben: Understood. Thank you. That was bracing. Dr. Lee: I'm told that's my speciality.