Who decides the questions? How voting quizzes work, and are they democratic?

Someone has to choose the questions

An election is fought over hundreds of issues. A quiz asks about thirty or forty. That choice is the single biggest decision in any voting quiz, and it is made long before you arrive.

What is left out cannot count. If a quiz has no question on the cost of groceries, a party whose whole pitch is supermarkets gets no credit for it. If it has three questions on one topic and one on another, the first topic quietly counts three times as much.

How a question is worded matters too. "Should farmers pay for their emissions?" and "Should farmers not have to pay for their emissions?" ask the same thing, but a quiz full of statements where agreeing is the left-wing answer will push people who agree a lot towards the left. Good quizzes test for this. Not all of them tell you whether they did.

How a voting quiz works

Most quizzes, including this one, follow the same basic recipe. You answer each statement on a scale from strongly disagree to strongly agree. Each party has been placed on the same scale for the same statement. The quiz measures how far apart your answer and each party's position are, adds that up across every question, and turns it into a percentage.

The recipe hides three more judgement calls. Who decides where each party sits? Someone has to read a party's policies and decide whether it "agrees" or "strongly agrees". What happens when a party has no clear position? Treated carelessly, a vague party scores well with everyone. Does every question count equally? Some quizzes let you weight the issues you care about; some do not.

None of these choices is wrong in itself. The question is whether you can see how they were made. Our own answers to every one of them are on the how it works page.

Who decides where the parties sit

There are two common approaches. In one, the people running the quiz place each party from its published policy and then consult the parties, who can argue for a different placement. In the other, parties are placed only from what they have published, and the parties get no say.

Both have a case. Consulting parties can catch genuine mistakes. It also gives parties a private channel to influence how they are presented to voters, and you usually cannot see what was changed or why. Placing parties only from published material keeps them at arm's length, but it relies on the quiz-maker reading that material fairly, which is why every placement should come with a source you can check.

NZVoteCheck uses the second approach: the parties have no say in where they sit, and every one of the 222 placements links to the source it came from.

Are voting quizzes democratic?

The case for. A good quiz gets people thinking about policy rather than personality. It can reach voters who would never read six manifestos, especially younger people, and it can surprise people by showing that the party they assumed they would vote for is not the one closest to their views. That is democracy working better, not worse.

The case for caution. A quiz is a set of editorial choices presented as a neutral calculation. Nobody independent checks how one is built before it goes live, so how much you can see of its workings depends entirely on whoever runs it. Most people who take a quiz never read its method, and a percentage on a screen looks more objective than it is.

The bigger question is scale. When a quiz is run by a major broadcaster, its questions do not just shape individual results. Its aggregated answers can be reported as news about what the country thinks, which means the choice of questions can shape the national conversation as well as personal ones. The people taking it chose to, so these results describe its users rather than the electorate, however carefully they are weighted.

So are they democratic? They can be, if they are open about who built them, how, and with whose money. A quiz that hides its workings asks for your trust without earning it.

A checklist: is this quiz worth your trust?

Use these seven questions on any voting quiz, including this one.

1. Who built it, and who pays for it? You should be able to find a name and a funding source without digging.

2. Are the people who chose the questions named? An unnamed panel of experts is not the same as named people you can look up.

3. Is the method published in full? Not a summary, the actual maths, including how party positions are turned into numbers.

4. Does every party position have a source? And a date it was last checked, because parties change policy during a campaign.

5. Can the parties change where they sit? If they can, is that process visible?

6. Does it tell you its weaknesses? Every method has some. A quiz that claims to have none is not telling you everything.

7. What happens to your answers? Are they stored, shared or sold, and can you say no?

How NZVoteCheck measures up

It would be odd to publish a checklist and not apply it to ourselves, so here are our answers.

Built by one person, Alexander Sharman, and paid for personally, with no money from any party, broadcaster, donor or lobby group. The full method is published, including the scoring maths and the tests we run for bias. All 222 party positions link to their sources with a check date, on the party policies pages. The parties have no say in where they sit.

Our weaknesses are on the record too. The questions were chosen by one person, which is a real limitation however carefully it is done. The question bank currently has 21 statements where agreeing favours the right bloc and 15 where it favours the left, and we publish that rather than claim a balance it does not have. Te Pāti Māori's positions rest on fewer sources than other parties', because it publishes less sector-by-sector detail. Answers are stored anonymously only if you leave the sharing box ticked.

If you want to compare us directly with TVNZ's quiz, our side-by-side guide sets out where each one is stronger.

Before you vote

Treat any quiz as a starting point, not a verdict. Take more than one if you like, and pay attention when they disagree: that usually means your views sit between parties, which is worth knowing.

Then check the sources behind the parties you match, because a percentage is only as good as the placements underneath it. When you are ready, the NZVoteCheck quiz takes about five minutes, and every position it scores you against is one click from its source.