Posts

Too True?

/1 Comment/in , /by

Just a quick thought.

Can you put too much effort into making sure a question satisfies everyone in its lack of ambiguity?

People want quiz questions to be perfectly clear with perfectly unambiguous answers, and rightly so, but it can sometimes be a little muddier than that.

Just as an example, if you watch Only Connect, you’ll notice that sometimes when teams give an answer Victoria Coren Mitchell asks them to explain why they’ve given their answer, and either accepts or doesn’t. They haven’t given the expected answer, but they have found a valid sequence or connection or at leas their own way to explain it (sometimes more specific or less so than envisaged). It doesn’t happen very often, but OC is perhaps more likely to have that issue than a standard quiz, perhaps, and requires everyone involved to be very much on their toes to keep up with the quick minds of the contestants.

But even in a standard quiz, whether hosted by a quiz master or on the page of a book or magazine or on a website or in an app, a question can be technically sound but people might still find what they think is reasonable cause for alternative answers.

I’m not talking about faulty questions really, or those which are poorly phrased.

There are different cases –

1. a question where the participant has a slightly different understanding of what the terms on the question mean, and by their terms, they have a different answer … an example might be something like the use of World Cup Finals or World Cup in a question – people might misunderstand what those terms mean.

2. a question might be technically correct but actually be too complex, so that most of what is in the question points to a certain answer, but there is a little detail which means that is not the right answer. It might be deemed a little unfair or tricksy – an example might be: “What film did this actor make with this director, in which he played this superhero opposite this actress, in 2002?” … when every detail might point to one answer except the year (there being two such films made in different years).

So, in each case the quiz master can reasonably enough say “No, sorry, your answer is wrong, the question is right and you’ve misunderstood it or not listened closely enough …”

Of course it always ought to be possible to add in plenty of wording and definitions – but where do you stop? Where do you draw the line? Is it worth the effort not to cover any eventuality, not to have to deal with a query?

It would seem that the obvious answer is “Yes” in most cases, but as a question writer, brevity, clarity and the form of the question are also very important, and sometimes you can get bogged down with caveats. So sometimes you take a judgement call to go for brevity and be armed with the facts for clarification. Or sometimes you have to just bite the bullet and bin a question because you can’t narrow it down sensibly enough. The key is to be aware of what you are doing.

It is always helpful for the quiz host (whether at a pub quiz or at a corporate quiz night or on a TV show) to have the facts at hand to put anyone straight if they are a little befuddled. It often actually enhances the host’s authority (as it does, for example, that of Victoria Coren Mitchell on Only Connect) to be able to say e.g. Interesting, but in fact THIS was in THIS COUNTRY, not THAT COUNTRY, so close but no cigar …”

So, for what it’s worth, I think the answer is “be fair and be armed with the facts”.

Reads like a rather good slogan …

 

Levels of Questions

/0 Comments/in /by

This week, as most weeks, I was doing some question writing. I tend to have a few concurrent projects to work on, whether writing general stock for our hosted corporate quizzes or specifically for one special event, writing for a game we’ve been hired to provide questions for, or for a TV show.

Down the years, there have been 100s of projects and lots of different styles and target audiences for questions, which require shifts in focus and mindset. Few are more pronounced than one I experienced this week, though, when, on the same day, I went from coming up with the ideas for the fiendishly difficult BBC Quiz Show ‘Only Connect’ to writing short Buzzer questions for a quiz for Year 5 and 6 school pupils.

Every question is its own challenge to be addressed seriously, but it would be a little disingenuous to say that each question is as hard to write as the next. From germination to fruition, an ‘Only Connect’ question might well be several hours of work, might go through several stages, might need some serious research and brain power to get it just right. That  is not true of a question like “What is the capital of Spain?” which may suffice as a buzzer question for children.

Each question ought to have a home for whatever its level of complexity and/or difficulty. It is not quite true that there is no place for banal, facile questions, which is a view some quiz fans/quiz writers have. There is a place for such questions, in quiz machines, in buzzer rounds, as confidence-boosters in quizzes where teams are of a relatively low standard and no one seems all that keen on thinking very hard. In itself, such a question is obviously less satisfying to write than a beautifully constructed gem with a satisfying answer which makes people either go “Yesssss!!” or “Oh, of course. Damn” but putting together a round or sub-category which makes good use of simple questions and ends up being fair and enjoyable for its target audience is not to be sniffed at.

I find now, as I’ve always found, that my natural inclination is to write challenging but not impossible questions. That’s probably how most good question writers see it. So, if I’m charged with writing, say, 500 Multiple Choice questions for a specific project on a specific topic, equally dispersed between Easy, Medium and Hard, the chances are that, when I look back on the first 200 or so and give them a level, I’ll have written too many “Mediums” and I’ll have to consciously weight the next 300 more towards “Easy” and “Hard”.

The right levels can be hard to find sometimes. Both when writing questions for an unseen audience or when delivering a corporate pub quiz night for a particular group in front of you, you can initially find their knowledge base a little obscure and unexpected. I remember doing a quiz for young people where the first question was something like “Conkers come from which tree”? to be met by looks of what I thought was general bafflement. I really had to think on the spot about how to make this quiz work. The same group, I recall, were unusually strong on sport and recent politics.

I can’t always presume I find the level perfectly, but I’ve got pretty good at it, over 8+ years of writing questions for various groups and audiences all over the world. If someone tells me I’ve got the level of a question wrong, I might take a while to persuade, but I can be persuaded. I certainly have weaknesses. I don’t know if other question writers find this, but there are times when I really “want” the quizzers to know more about the subjects which are of particular interest  to me. I want my odd question about Tom Waits or mid-90s indie to go down well (they don’t). I want people to be led towards the correct answer on the slightly obscure classical reference (they sometimes are, hurray!)

But for the most part I’ve learnt to quell that instinct. I’m a pretty good judge of what people know, mainly through practice rather than some great underlying sixth sense, so hopefully those Year 5 pupils will have a good, fair buzzer round, and hopefully there’ll be some good questions of mine coming up on ‘Only Connect’…

Testing Testing

/0 Comments/in /by

As a QuizQuizQuiz Quiz Master and main Question Writer, all aspects of my job require testing things to see if they’re at the right level.

Of course, I do a sound check before every quiz. This is just as important when using our own portable equipment as when plugging into a venue’s own AV system, though the challenges are slightly different.

With our own  system, the challenge is placing the speakers so that everyone can hear properly with no one being blasted with noise, making sure there’s no feedback, that my voice sounds clear and crisp etc. Some rooms we turn up at can provide more of a challenge than others.

With a venue’s own AV system, whether in a pub, a hotel or a conference room in a company’s offices, it ought to be simpler, but there are pitfalls to avoid. The system should have been perfectly set up to suit the room, and often it is. Often it really is a case of plug in, play a little music, say a word or two, yes, this’ll be perfect. But as a quiz master, we’re very aware of how much more volume is required in a room full of 100 people than an empty room; aware of it in a way that sometimes a venue’s own AV specialist isn’t. Often, one has to politely suggest “I think I’ll need a little more than that on the mic” and be told “No, no, this is fine” when I know full well that as the hum of 100 people chatting and cheering and drinking grows, I really do need a little more on the mic.

Likewise, every now and then, a conference room’s sound may sound fine and clear at mid-volume, but may begin to struggle at a slightly higher volume –  a bit of hiss, a bit of crackle. Experience has taught me the importance of a rigorous test – or as rigorous a test as possible.

Likewise, testing is important for a question writer. We update the database for our corporate events regularly, write 100s and 100s of new questions a year, and we want these questions not just to be ok, decent, forgettable questions, we want them to be great, memorable questions. I have a  pretty good idea when I’ve written a question if it’s a cracker or not, but, in many cases, questions I think will be surefire hits get a muted response if not quite used right, while seemingly innocuous ones bring the house down.

So, we ask all our quiz masters to provide us with feedback on new questions they’ve used when we send out new questions. We’re always swapping ideas and thoughts on how a question has gone down or can be slightly improved – we all want every question we run to be a bit better than the last one we ran.

And, finally, in my capacity as the writer of our multiple choice questions, getting the right level is of paramount importance. Often, because of budget and timing, that testing has to be internal. If I’ve written 5,000 questions for a game, all of which  require a difficulty level, I (and the client) must rely on my own experience, my own hard-earned sense of what people know and what they don’t. I’m pretty good at it. As well as me, there’ll always be at least one other experienced question writer, editing and checking my questions, and if they  feel I’ve mislevelled a question, they’ll let me know.

Sometimes, if the budget’s higher, if the questions are more specialised, we can build a thorough question test into our schedule. You may know we’ve done that recently, with a game we’ve written and which we’re rather excited about. Because of the nature of the game, it was important to test the difficulty ramping.

And, we’re lucky, we’ve got a ready-made focus group, our Friday Quiz mailing list, who we think we’ve got a pretty good relationship with and who, frankly, enjoy a bit of a quiz. So we put a message out for anyone who wanted to help us testing and got a big response. It was hugely helpful to test how quizzers responded to all the questions in the game and will hopefully improve it for the wider audience we hope it finds.

Of course, part of the  experience of surveys and testing is not to blindly accept what basic statistics tell you. Above all, at QuizQuizQuiz,  we trust our own experience. We think we know what makes a good quiz question, a well-balanced round, a fair subject matter. Whenever that confidence is put to the test, we’re happy to see it confirmed (or not – and we learn from it!)