Private AI Knowledge Base
Your Firm's Knowledge, Queryable. On Infrastructure You Control.
Quick answer: a private AI knowledge base lets your firm ask questions of decades of its own documents, precedents and project records and get sourced answers — with the material and the answers staying on infrastructure you control, up to and including a fully air-gapped server in your own rack. You do not need to train a model. The asset is your knowledge, not the model — the model is a swappable component.
“We want a private LLM” is never one requirement. It is five separate concerns wearing one phrase — and each has a different, differently-priced answer.
The core principle
The asset is your knowledge, not the model
Models improve every quarter and depreciate like laptops. Your corpus — the precedents, project files, drawings, advices and institutional memory accumulated over decades — appreciates, and nobody else has it. A well-designed private AI environment keeps those two facts separate: the knowledge layer (your material, indexed for retrieval) is permanent and owned; the model layer is a replaceable part you upgrade as better ones appear.
This is why training a model on your firm's data is usually the wrong instinct. Retrieval-augmented generation gives you firm-specific, source-cited answers from a general model — without baking confidential material into weights you can never fully audit or unwind.
Unbundling the ask
The five concerns inside “we want a private LLM”
Boards rarely mean the same thing by “private”. Before any architecture decision, separate the five concerns — because only some of them require self-hosting, and each one you can answer with contract or configuration instead of hardware saves real money.
Data sovereignty
Where does the material physically live, and under whose jurisdiction? For some firms EU hosting settles it; for others nothing short of their own rack will.
Training leakage
Could your material end up inside someone else's model? Enterprise agreements exclude it contractually; self-hosting excludes it physically. Consumer tools exclude it not at all.
Regulatory exposure
Client confidentiality, professional privilege, sectoral rules. The question is what your regulator and your client engagements actually require - not what feels safest.
Competitive moat
Decades of judgement, precedent and project memory are the asset competitors cannot buy. The knowledge base is how that asset compounds instead of retiring with the people who hold it.
Cost predictability
Per-user subscriptions scale linearly with headcount; owned infrastructure is a capital cost with flat running costs. Which is cheaper depends on scale and time horizon.
The spectrum
Five architecture positions, five price points
“Private AI” is a spectrum, not a product. These are the five positions we assess every firm against — the discipline is choosing the cheapest one that genuinely answers your concerns, not the most impressive one.
1. Configured enterprise AI
€20-60 / user / monthClaude, Copilot or equivalent under enterprise terms: no training on your data, admin controls, audit logs. The right answer for most firms most of the time.
2. Enterprise AI + private retrieval
Subscription + buildA retrieval index over your own corpus, queried through an enterprise model. Firm-specific answers, sourced citations, no self-hosting burden.
3. EU-hosted / managed open model
Mid-rangeOpen-weight models run by an EU provider under EU jurisdiction. Sovereignty without owning hardware.
4. Self-hosted open model
€20-60k kit + 8-12 week buildAn open-weight model and retrieval stack on a GPU server in your own rack. Air-gap optional. You own every layer.
5. On-premise bespoke
€300k+Custom environment, dedicated infrastructure, formal verification posture. Justified only when the strictest concerns genuinely apply.
The corpus
Where a firm's knowledge actually lives
In a typical professional practice, only a sliver of the knowledge is in systems designed to be searched. The rest is in documents — and in people. A knowledge base that only ingests the structured sliver misses the point.
Structured
Databases, registers, filing systems. The easy part - and the smallest.
Unstructured
Documents, drawings, reports, correspondence. This is what retrieval unlocks.
Tacit
In senior people's heads. Captured through structured AI-assisted interviews - and lost at retirement if it isn't.
How to de-risk it
Prove it on public material first
- Prove the pipeline on your public corpus first - published work, planning records, regulations. Full architecture, zero confidentiality risk.
- Unbundle the five concerns and pick the cheapest architecture position that actually answers the ones that apply.
- Run a fixed-fee diagnostic before any build: corpus audit, use-case selection, architecture recommendation with costs.
- Pilot with a bounded group on one high-value use case before rolling out to the practice.
- Govern from day one: access controls, source citation on every answer, and a named owner.
Common questions
Private AI knowledge bases: FAQs
What is a private AI knowledge base?
A private AI knowledge base is a system that lets your firm ask questions of its own material - documents, precedents, drawings, project records, policies - and get sourced answers, where both the material and the answers stay on infrastructure you control. It has three separable layers: your corpus, a retrieval index over it, and a language model that drafts answers from what the index returns. The model is swappable; the corpus and index are the asset.
Do we need to train our own AI model?
Almost certainly not. Retrieval-augmented generation (RAG) separates your knowledge from the model: the model reads the passages retrieved from your corpus at question time and answers from them. That means you get firm-specific answers from a general model, you can upgrade the model without losing anything, and none of your material needs to be baked into model weights. Fine-tuning or training a model on firm data is rarely justified for professional firms and creates the very leakage risk most firms are trying to avoid.
Can AI run entirely on our own servers?
Yes. Open-weight models now run well on a single GPU server in your own rack - indicative hardware cost runs from roughly €20k for a capable starting point to €60k for a high-end configuration - and the environment can be fully air-gapped: no internet connection, no telemetry, nothing leaving the building. Whether you need that posture is a separate question; it is the strictest of five architecture positions, not the default.
Will our data be used to train public AI models?
Under consumer AI tools, possibly. Under enterprise agreements from the major providers, training on your data is contractually excluded by default. This is one of five distinct concerns that firms usually bundle into the phrase “private LLM” - data sovereignty, training leakage, regulatory exposure, competitive moat, and cost predictability. Each has a different answer, and only some of them require self-hosting. Unbundling them is the first step of any serious private-AI decision.
What does private AI cost?
It spans a wide spectrum. Configured enterprise AI with contractual data protections runs roughly €20-60 per user per month. Enterprise AI plus a private retrieval layer over your documents adds a build cost but keeps subscription economics. EU-hosted or managed open-model options sit in the middle. A self-hosted open model on your own hardware means an indicative €20-60k of kit plus an 8-12 week build. A fully bespoke on-premise environment can exceed €300k. The right position depends on which of the five concerns actually apply to your firm - most firms need less than they think, and some genuinely need the strictest posture.
Where should a firm start?
Prove the pipeline before touching confidential material. A proof-of-concept built on your public corpus - published projects, planning records, regulations, your own website - demonstrates the full architecture with zero confidentiality risk. If the answers are good on public material, the leap to the private corpus is an infrastructure decision, not a leap of faith. From there: a fixed-fee diagnostic, a bounded pilot with a small user group, then governed scale.
Start with the diagnostic
Find Out Which Position Your Firm Actually Needs
Acuity AI Advisory is vendor-neutral: we sell no hardware, resell no licences, and take no referral fees — so the architecture recommendation is the one that fits, not the one that pays us. The engagement follows the same discipline as all our independent AI consulting: diagnose before prescribing, with governance built in from day one.