Too Easy/Too Hard

I recently recalled the first round of quiz questions that I ever set for QuizQuizQuiz.

It was March 2006, I’d been given a job with the company, I was full of myself and raring to go. I’d already run a couple of quiz nights, which I hadn’t written the questions for. I’d been given some rounds to write for our two weekly pub quizzes (now dormant) in Putney and Hammersmith.

I wrote various questions for various rounds and was assigned the 20 question Jackpot rounds for both quizzes. Excited to see how they went down, I was at the Fox in Putney on Monday night, not as quiz master, but as marker.

I had form with this quiz. In fact, I’d participated in it very successfully for several months – that’s how I got the job. It was a very high standard quiz and the jackpot round was, deliberately, the hardest round. Teams had to score a minimum of 17/20 to have a chance of the money. I knew the target audience, I knew how to pitch it, I thought.

Now, bear in mind, as someone who loved quiz nights, this was my blank slate. They often say about first and second albums that the first is full of the songs the artist has been perfecting their whole life, while the second is something they only have a few stolen months to write. Well, this was my first album, these were my questions, the best I had.

9/20 was the highest score. The quiz had been, as every week, buzzing at the usual expertise of the QuizQuizQuiz quiz master, it was at fever pitch for the culmination, the jackpot round. And my round killed it stone dead. Puzzled looks and shrugs, shouts of “it’s too hard”, “I don’t get it”. I was a little bit crushed.

I’m just looking at the round now on our database. Any gems? A couple, but yes it’s far far too hard, and there are quite a few ambiguous questions – the subject matter is a showy-offy display of my own interests – Scottish indie pop, linguistics, philosophers, 60s athletics and rock music, Medieval history, Art pranksters, ancient Greek, 90s comedy, old radio adverts, Pubs and Beer, African politics, cricket-playing Irish playwrights. I didn’t realise the extent to which my history was not shared history.

I should have, I had no excuse. I’d been to the quiz night for months. But I got it totally wrong.

I still get it wrong occasionally. I ran a corporate quiz with entirely new questions last month and slightly misjudged the first round so that scores ranged from 5 to 9 out of 12 rather than a preferred 7 to 11. I quickly adjusted the difficulty for the rest of the quiz.

Judging the difficulty of quizzes is something anyone can get wrong. People’s gauge is based on what they themselves know and don’t know. To some quiz masters, difficulty may not be that important if they think the questions are interesting enough, but it ought to be.

After 10 years of doing this, we’re now very good at gauging difficulty. We’ve seen 10s of 1000s of questions, we get statistics on how well they’re answered. We’ve turned it into a little bit of science.

It’s still not perfect, as the example of my recent opening quiz round shows. I thought the crowd would know a little more than they did. They were untested questions. But such instances of small misjudgement are pretty rare.

Despite the misadventure of the first round I ever set, I now have a confidence bordering on bullishness in the suitability of my quiz rounds. I have not written a round since where the highest score was less than 50% (well, not without intention and very good reason!)