The LLM Wiki: A Pattern for Smarter Organisational Knowledge Bases
Andrej Karpathy's 'LLM Wiki' pattern offers a compelling alternative to retrieval-augmented generation. Instead of re-deriving insights from …
Read insightFive essential questions before any AI project, including how to account for the tacit knowledge your experienced people rely on.
Before starting any AI project, the questions that matter most are not technical; they are foundational. We’ve seen millions spent on AI initiatives that were doomed from the start, not because the technology was wrong, but because nobody paused to think critically about the fundamentals.
Here are five questions we ask at the start of every engagement. They’re simple, but the answers are revealing.
The fundamentals worth resolving before any technical work begins.
Not “we want to use AI” or “we need to be more innovative.” What specific, measurable problem is causing pain today?
Good answer: “Our review team misses 12% of issues because they’re handling 200 cases per hour.”
Bad answer: “We want to leverage AI to transform our operations.”
If you can’t articulate the problem in one sentence, you’re not ready to start.
Understanding the current process, including the humans involved and the expertise they bring, is essential. This tells you:
Pay particular attention to the knowledge that is not written down anywhere. Every organisation has people whose experience makes them disproportionately effective: who can read a situation quickly, who know which edge cases matter, whose judgement has been calibrated by years of exposure to the problem. This tacit knowledge is often the most important input to how the process actually works, and it is the part most likely to be ignored when someone maps current-state workflows on a whiteboard.
Skip this step and you risk automating the wrong things.
Define your metrics before you build anything. Be specific:
Without clear success criteria, you’ll never know whether your project actually delivered value.
AI needs data. But not just any data: the right data, in sufficient quantity, with adequate quality. Before committing to an AI approach, assess:
Data problems are the single most common reason AI projects stall after kickoff.
The best AI system in the world is worthless if nobody uses it. Understanding your end users is critical:
The human element isn’t a nice-to-have; it’s the difference between a successful deployment and expensive shelfware.
Print these questions out. Put them on a whiteboard. Make them the first slide in every AI project kickoff. They won’t guarantee success, but they’ll dramatically reduce the probability of failure.
If after working through them you want a more structured view of your organisation’s readiness, a fixed-fee advisory session can go deeper on your specific situation.
Want help working through these questions for your organisation? We’re here to help.
FAQ
The five most important questions are: what specific problem are you solving (with measurable definition), how are you solving it today, including what tacit knowledge your experienced people apply, what does success look like (defined in advance with clear metrics), do you have the right data, and who will use the system and do they want it. Answering these before any technical work dramatically reduces the risk of failure.
Tacit knowledge is the practical expertise that experienced practitioners carry: the judgements they make instinctively, the patterns they recognise before they can articulate why, the edge cases they know to watch for. It matters before starting an AI project because it often constitutes the most important part of how a process actually works, yet it is invisible on process maps and rarely captured in data. AI designed without accounting for it risks automating around the very thing that makes the current process effective.
An organisation is ready to start AI when it can clearly articulate the specific problem it is solving, has or can access the right data, has defined success criteria in advance, and understands the human expertise the system needs to work with. If any of these are unclear, a structured AI readiness assessment is worth doing before committing to build.
Keep reading
Andrej Karpathy's 'LLM Wiki' pattern offers a compelling alternative to retrieval-augmented generation. Instead of re-deriving insights from …
Read insightAI is widely used to process existing information. Its potential to capture expertise that was never documented in the first place is …
Read insightMost AI is designed to optimise tasks. But when it's built without reference to the expertise behind those tasks, it doesn't just fail to …
Read insightWhere this goes
The ideas in this writing are becoming products. The waitlist is the earliest way in. For a deeper conversation about your situation, we hold a small number of fixed-fee advisory sessions each month.
New insights are shared on LinkedIn as they publish.