The shape of it
Clients are separated by data, not by hardware. One machine serves all the websites and runs one copy of the language model. Each client brings their own knowledge base, their own system prompt and their own inbox, and the model borrows whichever one it needs for the question in front of it.
Client websites
Business owners
The model is the expensive part, and there is only ever one of it.
What happens when a visitor asks a question
- The widget sends the questionAlong with the client id it was installed with.
- The API checks the request came from that client's domainThe id alone is not proof of anything, so the origin has to match what is on file.
- It loads that client's configurationBusiness name, language, tone, opening hours, where enquiries should go.
- It retrieves the relevant factsA database query filtered by tenant id, returning a few chunks of that client's own content.
- It assembles the promptSystem instructions, the retrieved facts, and the last few turns of this conversation only.
- The model streams the answer backThe visitor sees words appear immediately, which matters far more than raw speed.
- Anything that looks like an enquiry is saved and forwardedStored against that client and pushed to their inbox while the visitor is still interested.
Why one model beats one per client
Memory is spent once
The weights are what fill the machine. Twenty clients need barely more memory than one, because each extra conversation only adds a small amount of context.
Idle servers cost the same as busy ones
A small business generates a handful of chats a day. A dedicated box per client would sit unused almost all the time while still being billed in full.
A better model for the same money
Pooling the cost affords one capable model rather than many cramped ones. Everyone gets better answers, and better Swedish.
One thing to maintain
One deployment to patch, monitor and back up. Improve a guardrail once and every client has it that afternoon.
Concurrent chats batch together
Serving engines process simultaneous requests in a batch, so eight conversations at once cost far less than eight times one conversation.
Flat pricing becomes honest
With a near-zero marginal cost per client, "no per-message fees and no limit on questions" is a promise the arithmetic actually supports.
How many clients fit on one box
Client count is rarely the limit. What matters is how many conversations overlap at the same moment. Move the sliders to sketch your own load.
Comfortable on a single shared GPU box.
Rough planning arithmetic, not a benchmark. It assumes traffic spreads over a ten-hour day and that busy moments run about three times the average. A CPU-only box realistically handles one or two overlapping chats; a 24 GB GPU box handles roughly eight to ten.
Two hardware routes
| Setup | Model | Feel | Overlapping chats | When it fits |
|---|---|---|---|---|
| CPU only | 3–4B, 4-bit, about 2 GB of memory | Workable with short answers and streaming; published 2026 benchmarks put a 7–8B model on a modest four-core machine at only three to four tokens a second, which is why the smaller model matters | One, maybe two | Your first client or two, while traffic is unproven |
| Shared GPU | 7–14B plus an embedding model, on a 24 GB card | Answers feel instant, and language quality is noticeably better | Roughly eight to ten | From two or three paying clients onward |
European hosting prices moved several times during 2026, so treat any figure you remember as stale and re-check before quoting a client.
The rules that keep clients apart
Everything above is an optimisation. This part is not optional: one client's prices surfacing in another client's chat would end both relationships.
Filter in the query, never in the prompt
Retrieval must be scoped by tenant id in the database call itself. Asking the model politely to ignore other clients' documents is not a boundary.
One index per client, or one indexed column
Never load several clients' content into a single unfiltered search. The same applies to conversation logs and captured enquiries.
Verify the origin
A client id in a widget is public. Check it against the domain the request came from before answering anything.
Test for leaks deliberately
Add a distinctive fake fact to one client's knowledge base and try to pull it out through another client's widget. Repeat after every change to retrieval.
When things go wrong
- The model is down or restarting. The widget falls back to a short contact form, so the visitor still becomes a lead. Websites keep serving regardless, because they are static files and do not depend on the assistant.
- Everything is busy. Queue the request, show that the assistant is thinking, and cap the wait. Per-client rate limits stop one busy site starving the others.
- Someone abuses the widget. Limit requests per visitor, cap answer length, and set a hard timeout. Compute is your cost, so treat it like bandwidth.
- The model invents an answer. Ground it in the retrieved facts, instruct it to admit when it does not know, and offer to pass the question to the owner. Wrong answers about allergens, prices or availability are your client's liability and your reputation.
- A client's content changes. Re-index on demand from an admin page, so a new menu takes minutes rather than a rebuild.
Data protection
Chat transcripts are personal data. In each arrangement the client is the controller and you are the processor, which means a written data processing agreement with every client, servers in the EU, a stated retention period with automatic deletion, and a line in the client's privacy policy describing the assistant. Shared infrastructure is perfectly acceptable; unseparated data is not.
Order of work
- Build it multi-tenant from the first clientRetrofitting separation into a single-tenant system is how leaks happen. The tenant id costs nothing to add now.
- Start CPU-only with a small modelProve the flow, the guardrails and the enquiry handover before paying for a GPU.
- Measure real traffic for a monthLog overlapping chats and reply times, not just totals. That is what sizing depends on.
- Move to a shared GPU box when two or three clients justify itSame code, better model, more headroom, and the marginal client stays nearly free.