Mystery is both a technique and a genre in movies. As a genre, I believe pure mysteries are a unique type of story. They do not follow the typical rules of narrative. Rather, they are more like puzzles – like Sudoku or crosswords.
True mysteries are stories where the main character is acting as a detective (professional or amateur doesn’t matter) who is trying to solve a crime. The main conflict of a true mystery story is always “will the detective solve the mystery?”
Often there is no character arc or even much emotional content to the story. Think of the typical Agatha Christie and Sherlock Holmes stories, or TV detective stories ranging from CSI to Murder She Wrote. Mystery seems to be the only genre that can thrive without these elements. It does this by offering the audience a game.
The game of a true mystery is for the audience to try to figure out the solution before the film reveals it. If the mystery is to be successful, the audience better not win this contest – but they should feel like they could have if they were just a little smarter.
Like any game, there are rules to mysteries. In this case the rules are an unspoken contract with the audience. If we don’t play by the rules, the audience feels cheated and won’t like the film.
The main rule is the audience must see all the clues the detective sees. In the classic mystery where the detective gathers a group of suspects in a room and reveals the solution, it would be very unsatisfying if the detective revealed he had found a clue that wasn’t shown to the audience. In the game where the audience tries to out-think the writer, that’s cheating.
It works both ways, though. In most true mysteries the audience only sees what the detective (or detectives) see. Though they might not turn against the story entirely, it would be a little unsatisfying if we had information the detective did not.
Another rule is, except in rare cases, the guilty party must be a character we have met. If the killer turns out to be someone who lives down the road who doesn’t make an appearance until the final scene, it’s unsatisfying. Think of the movie The Usual Suspects (written by Christopher McQuarrie). Though logically Keyser Sose could be anyone or not exist at all, there is an implied understanding with the audience that he will turn out to be one of the characters we’ve met. Trying to guess which is what makes the movie fun.
(The exceptions to the above rule would usually be cases in which a character is unseen but present in other ways – someone the other characters talk about frequently, for example.)
The Silence of the Lambs (screenplay by Ted Tally) would seem to break the above rule. But I would argue that though the movie uses mystery and the main conflict is about whether Clarice will find Buffalo Bill before he kills again, the movie is not really about figuring out Buffalo Bill’s identity. It’s about Clarice and her battle of wits with Hannibal Lecter. The mystery about Buffalo Bill simply provides stakes to the Clarice/Lecter storyline. The Silence of the Lambs is a psychological thriller.
Planting and payoff are key to making a mystery work. “Clues” are another word for things that are planted. Sometimes the plant is made obvious – when the detective finds a murder weapon, for example. Other times they are hidden. This helps us outsmart the audience. If you plant a clue in such a way that the audience sees it but is distracted by something else, the audience hopefully won’t make the appropriate connections until it’s too late.
The Sixth Sense (written by M. Night Shyamalan) does this exceptionally well. Clues to Malcolm’s true nature are introduced often but disguised. Think of the scene where he goes to the restaurant to meet his wife on their anniversary. We think the scene is about the breakdown of their marriage. It is only later that we realize her failure to respond to him was a clue to a whole different mystery.
The Sixth Sense and The Silence of the Lambs are examples of movies that use mystery extensively but are not true mystery movies. They follow traditional narrative rules about characters on an emotional journey. When you’re doing a movie that revolves around a mystery it’s important to determine whether you are doing a true mystery in the classic sense or using techniques of mystery in another type of storyline.
But these two movies also demonstrate how combining mystery with another genre can result in powerful stories. Thrillers are a traditional narrative genre that often use a lot of mystery elements. Mystery can also be useful in science fiction, horror, drama, farce and many others.
“Red Herring” is a term for a clue that points to a different solution than the correct one. These are important to mislead the audience and offer alternative scenarios. It’s key that the red herrings not feel gimmicky, though. When the solution is finally revealed, the red herrings must be organic to the story. Sometimes you can accomplish this by using planting and payoff. Other times the key to making the red herring feel organic can be found in character motivation.
For example, in The Usual Suspects there are several red herrings pointing to Dean Keaton as Keyser Sose. Agent Kujan maps out the scenario that implicates Dean Keaton. This feels organic because of Agent Kujan’s personal animosity toward Dean Keaton from an unrelated event.
One of the best things about using mystery is when it’s done well you create real engagement with your audience. Just be sure to play by the rules!