There was a stretch, not very long ago, when you could prepare for an AI engineering interview by memorising definitions. What is a transformer. What is RAG. What is a vector embedding. Learn thirty of those and you had a decent afternoon.
That preparation stopped working, and it stopped working for a reason worth understanding rather than just working around.
The definition became free
An interviewer asking “what is retrieval-augmented generation?” is trying to find out whether you have engaged with the subject seriously. That was a reasonable proxy in a world where finding a good explanation took effort. It is no longer a proxy for anything. Every candidate walking into the room has the same access to the same clear explanations, and most of them have read the same three articles.
So the question stopped separating anyone from anyone, and interviewers noticed. What replaced it is a different shape of question entirely: a situation with something already going wrong in it, and an invitation to say what you would do.
Those questions are harder to prepare for and much harder to fake, which is precisely why they spread.
What a situation question is actually measuring
Take the one that comes up constantly in any team running retrieval: a chatbot has just given a customer a confident, completely wrong answer, and you are asked how you would find out why.
Nothing in that question is about the definition of retrieval. It is asking whether you know that the failure has two entirely separate causes that look identical from the outside. Either the right passage never reached the model at all, or it did and the model talked over it. Those are different bugs, in different parts of the system, fixed in different ways — and a candidate who has actually operated one of these systems reaches for that split before diagnosing anything, because it is the only way to stop guessing.
A candidate who has only read about it usually starts describing the architecture instead. The architecture was never the question.
The same pattern shows up in the trade-off questions. When someone asks how you would decide between improving the prompt and fine-tuning the model, they are not checking whether you can define fine-tuning. They are checking whether you have a default and a set of conditions that override it — prompting first because it is fast and reversible, fine-tuning when you have stable labelled data and a formatting problem that instructions keep failing to hold. The answer that lands names the cost you are accepting, not just the technique you picked. Working through that decision properly is a genuinely different skill from being able to describe either option.
And then there is the question that quietly reveals the most: the feature demos perfectly, real users break it in ways nobody saw in testing, what is your process. Every part of a good answer here comes from having been on the wrong side of it once. Real inputs are not test inputs — they are empty strings, five-paragraph pastes, other languages, and people actively trying to get somewhere they should not. You need request and response pairs, retrieved context and tool calls logged before the incident, not after, because after is too late to start collecting them. And you need a way to turn the thing off for affected users while you dig, rather than leaving it broken in front of customers out of pride.
None of that is knowledge. All of it is scar tissue.
Why this is good news, mostly
The uncomfortable version of this shift is that you cannot read your way to a good answer any more. The encouraging version is that the bar is now something you can actually reach on purpose, in a way that memorising thirty definitions never was.
Every one of these questions is asking about a system you could build yourself, at small scale, on a laptop, this month. Build a retrieval pipeline and it will return the wrong passage for some query, and you will have to work out whether that was retrieval or generation — not as a concept, as an afternoon. Ship a prompt change without a way to score it and something will regress quietly. Put anything in front of five real people and at least one of them will send an input that was not in your head when you wrote it.
That is the whole curriculum. It is not long. It just cannot be skipped, because the specificity that makes an answer convincing in an interview is the residue of having been surprised by the system at least once.
The gap this leaves you with
If you read the three questions above and found yourself reaching for a definition each time, that is not a verdict on your ability. It is a fairly precise map. It says you understand the concepts and have not yet operated them, which is a gap with a known and unglamorous fix.
The candidates who sound calm in these interviews are rarely the ones who consumed the most material. They are the ones who finished a small number of real builds and can therefore describe a failure in the past tense.
That is the difference the question is designed to find, and it is the only part of preparation nobody can hand you.