An interview that feels natural is not the same as an interview that tells you whether someone can do the job. Most small business interviews feel natural. Most of them also produce decisions based on how comfortable the candidate made the interviewer feel, not whether they can do the work.
The natural interview is a social skill test. It rewards confidence, articulateness, and the ability to read a room. Those are useful traits in some roles. They are not the primary trait most roles require, and they are almost never what the hiring decision is supposed to be based on.
Structured evaluation is not cold or clinical. It is preparation: deciding before the interview what you’re trying to learn, then asking questions designed to learn it. The result is an interview that still feels like a conversation but produces data rather than impressions.
The Structured Interview Process That Produces Better Hires in Less Time covers the full implementation. This article explains the gap the structure is designed to close.
What Unstructured Interviews Actually Measure
The obvious problem: you ask questions that come to mind during the conversation. The candidate tells you their experience, shares some war stories, and answers your follow-up questions. You walk away with a feeling about them.
The less visible cost is what that feeling reflects. The feeling correlates with how similar the candidate is to people you’ve liked working with, how confident they were, and how smoothly the conversation moved. It correlates weakly with the ability to produce the role’s primary output, handle its challenges, or do what separates good hires from poor ones.
The deepest problem is that unstructured interviews produce inconsistency across candidates. When each interview goes wherever the conversation goes, you have not compared candidates against a common standard. You have compared impressions of different conversations. Those are not comparable in any meaningful way.
Unstructured interviews do not test the things you need to know. They test the things that are easiest to surface in a social conversation.
The Evaluation Gap
The gap between what most interviews test and what they should test has three dimensions. Closing each one requires a specific kind of preparation.
Dimension 1 is the absence of defined criteria before the interview starts
Most interviewers go into an interview without a written list of what they’re looking for. The success definition, the non-negotiable traits, the scenarios that reveal the difference: none of these are on paper before the first question is asked.
The result is that the interviewer improvises. Questions come from the resume, from curiosity, from whatever the candidate says that seems worth following up on. There is nothing wrong with curiosity or follow-up, but they cannot substitute for a set of criteria the interview is designed to surface.
A twenty-minute preparation session before the first interview changes the quality of what the interview produces. Decide what the role requires and what questions would reveal whether a candidate has it. That’s the preparation.
Dimension 2 is the failure to ask the same questions of every candidate
When each interview goes in a different direction, you cannot compare candidates on the same dimensions. The first candidate was asked about process-building experience. The second was asked about team dynamics. The third spent most of the interview discussing a specific project they mentioned in passing.
You now have impressions of three different conversations, not assessments of three candidates against the same criteria. The comparison is impossible to make cleanly.
Before: After three interviews, you pick the candidate who “seemed strongest” without being able to name why. After: After three interviews, each candidate has answered the same four questions and been rated on the same criteria. The comparison is on paper.
Dimension 3 is the absence of a structured way to record what happened
Most interviewers trust their memory. The interview ends, the next one begins, and the mental notes from the first interview blur into the experience of the second. By the time a decision needs to be made, the impressions are merged into a general sense of each candidate.
The candidate tracking record covered in the broader hiring process captures outcomes. The interview record captures what happened during the evaluation. A brief note on each candidate’s answers, taken right after the interview, preserves the signal before the blur sets in.
How the Conductor Explains Your Interview Practice Gap
Most founders assume their interview process works reasonably well. The Conductor can give that assumption a basis in evidence rather than a feeling.
The Living Library is the part of the Kiluma platform that holds your accumulated hiring records, interview notes, and role definitions. The Conductor, Kiluma’s context-aware AI, reads from those records rather than from a generic framework for good interviewing.
Ask it: “Based on our past interviews and hiring outcomes, where is the gap between what our interviews are testing and what this role actually requires?” It reads the interview records, the role’s success criteria, and the outcomes of prior hires. It returns a specific gap: which criteria past interviews covered, which were skipped, and which signals turned out not to predict success in this role.
That reading is the starting point for closing the gap. The Structured Interview Process That Produces Better Hires in Less Time builds from that starting point.
Write the Three Questions Before the Next Interview
Before the next interview, write down three questions: the things you most need to know about this candidate to decide whether to move forward. Not questions about their resume. Not questions about what kind of work environment they prefer.
The three things this role requires a person to be able to do or be, that you have no way of knowing yet. Those become the questions.
Then ask those three questions of every candidate. The first time you do this, you’ll notice that the interviews feel different: more substantive, less polite. That’s the gap starting to close.
An Interview Is a Test With a Predetermined Answer Key
Not every candidate will pass. Not every interviewer will get it right. But an interview with a predetermined answer key (criteria defined before anyone walks into the room) produces decisions based on evidence rather than comfort. Try Kiluma free for 14 days at kiluma.ai.