Self-Hosted AI: What It Really Costs After the Licence Says Zero

Self-hosting an open-source AI platform costs far more than zero. The licence may be free, but the deployment is a distributed system: a dozen containers, two machine-learning model servers, a search index, a database, a cache and an object store, the architecture we unpack in enterprise search vs an AI context layer, all of which someone has to size, secure, patch and restore at 2am.

We wanted a real number instead of an opinion, so on 4 August 2026 we deployed a leading open-source AI stack the documented way, on default settings, and measured what happened. Twelve services. Twenty-four minutes to install. And the search index was killed by the operating system ten times before we stopped it, because there was not enough memory left for it to start.

None of the vendors involved publishes a minimum system requirement. Not in the quickstart, not in the README, not as a resource limit in the deployment file. Your IT team is sizing this by guesswork, and that guesswork is where the real cost hides.

We used Onyx, the most credible open-source alternative to Glean, with 31,400 GitHub stars and Netflix and Thales Group in production, reviewed in full here. We followed the documented path exactly: clone the repo, copy the environment template, run the standard compose command. No tuning, no tricks, nothing an evaluating engineer would not do on a Tuesday afternoon.

The host had 12 CPU cores and 36 GB of RAM, comfortably above anything a small company would provision. But the container runtime was on its default allocation of 3.83 GB, which is what almost everyone has until the moment they discover they need more.

Ten of the twelve containers came up healthy. OpenSearch, the component that actually indexes and searches your documents, never started once. It was terminated by the kernel out-of-memory killer with exit code 137, restarted, and terminated again, ten times over.

One: the licence you did not know you needed. As we found in our licence audit of seven open-source AI tools, single sign-on, role-based access control and permission synchronisation sit behind a commercial licence in Onyx, Flowise and n8n alike. If you index anything containing personal data, Art. 32 GDPR effectively requires those, so the free tier is an evaluation, not a deployment.

Two: the infrastructure you have to over-provision. A dozen services with two model servers and a search cluster is not a small virtual machine. Because no vendor publishes sizing guidance, most teams discover the real requirement the way we did, by watching something die.

Three: the engineering time, which dwarfs both. Patching, version upgrades across breaking changes, certificate renewal, backup and restore, and incident response when the index falls over during month-end reporting. This is the cost that never appears in a comparison table because it is paid in your own people's hours rather than on an invoice.

None of this means self-hosting is a mistake. It means it is an engineering decision rather than a savings decision, and it wins decisively in one scenario: when data physically must not leave your building.

For defence work, healthcare, works-council-sensitive deployments or any air-gap requirement, self-hosting is not a cheaper way to buy software. It is the only way to satisfy the requirement at all. Running on a server you control in Frankfurt with a local model means there is no international transfer to assess, no subprocessor chain to audit and no adequacy decision to monitor. That is stronger than any data processing agreement, including ours.

What self-hosting is not is a way to avoid paying. You exchange a licence fee for an operations budget, and for most companies under about 200 employees, the operations budget is larger.

The measured result is not an argument against open source. It is an argument against the sentence and it is free, which we traced back to the licences themselves, which appears in almost every comparison article and is true only of the licence file.

A twelve-service distributed system that ships without minimum system requirements, whose access-control layer is commercially licensed, and whose search index dies quietly on a default allocation, is a serious piece of infrastructure. Treat it like one. Budget an engineer, size generously, buy the licence you actually need, and it will serve you well.

Treat it like a free download and you will find out what it costs the same way we did, except in production.

Installation is a one-off. Maintenance is forever, and it decides whether self-hosting still looks like a good idea eighteen months later.

Every one of those twelve services has its own release cadence and its own security advisories. PostgreSQL, Redis, OpenSearch, MinIO and nginx are all widely deployed infrastructure with a steady stream of CVEs. The application containers move faster still. Someone has to track all of it, test upgrades against breaking changes, and roll back when an update takes the index offline during a working day.

That is why the honest comparison is not licence against licence. It is a permanent slice of an engineering headcount against a subscription. At a loaded cost of roughly 500 to 800 Euro per engineering day, two days a month comes to 12,000 to 19,200 Euro a year before a single document has been indexed.