The short answer.
Local AI runs the model on hardware you control. Cloud AI sends each request to a provider’s servers. A local model can still call online tools, and a cloud GPU rented for testing is still cloud infrastructure.
Local tends to fit when:
- prompts and documents should stay on your premises, for example because a client contract requires it;
- use is heavy and steady, such as several people or agents working every day, so a fixed cost is spread over a lot of work;
- you need answers without an internet connection, or a model that does not change until you change it;
- an open-weight model that fits your hardware passes your tests.
Cloud tends to fit when:
- you need the strongest proprietary models, which are offered only as cloud services;
- use is light or irregular, so paying per request costs less than owning a machine;
- you are still exploring and not ready to commit to hardware;
- nobody on the team can look after a machine.
Many teams mix the two: local for confidential or high-volume work, cloud for the hardest tasks. Either way, if personal data is involved, the PDPA applies wherever the model runs. See PDPA and AI.
| Decision | Local model | Cloud model service |
|---|---|---|
| Where is inference? | On your workstation or server. | On the provider’s infrastructure. |
| Which models? | Open-weight models that fit your hardware’s memory. | Proprietary models, plus hosted open-weight models. |
| Where can data go? | Inspect the interface, retrieval, tools, logs and backups as well as the model. | Review the specific service’s processing, retention and access terms, plus connected tools. |
| What do you maintain? | Hardware, drivers, serving software, access and recovery. | The provider maintains model hosting; you still maintain your data, integrations and permissions. |
| What do you pay for? | Purchase or rental, electricity, setup time, maintenance and replacement. Mostly fixed. | Subscription or usage charges, integration and any storage or retrieval services. Rises with use, at prices the provider sets. |
| What sets capacity? | Model, GPU memory, context length, serving configuration and workload. | The chosen model and service limits, network conditions and workload. |
| If the internet goes down? | The model keeps answering; online tools stop. | Requests fail until the connection returns. |
“Local” is a data-flow question.
Draw the path from your document to the final answer. Include the interface, embedding model, document store, language model, tools and logs. For each step, write where it runs and what it stores.
- Check the selected model endpoint. A desktop interface may connect to either a local server or a cloud service.
- Check retrieval. Document parsing, embeddings and storage can run in different places. Verify each one.
- Check tools and integrations. Search, email and external APIs may send information outside the workstation.
- Check access and recovery. Decide who can read documents, what is logged, where backups live and how access is removed.
For example, Ollama documents separate local and cloud behavior and a local-only configuration option. That setting belongs to that runtime; it does not control every other application connected to it. See the Ollama FAQ. TensorQuay’s supplied runtime and interface are agreed for each deployment.
If personal data is involved, add one more check: where each service processes and stores it. A low-cost cloud API raises the same question as any other provider, and the PDPA’s transfer rules apply when personal data leaves Singapore. Our PDPA and AI guide lists the obligations to check.
What fits on local hardware.
Two numbers decide most of what a local machine can do with a language model: how much memory the model can use, and how fast that memory can be read.
- Memory size decides which models fit. A model’s weights take roughly its parameter count × bits per weight ÷ 8 bytes. A 27-billion-parameter model at 4 bits needs about 14 GB, plus memory for the conversation itself (the KV cache), which grows with context length and the number of users.
- Memory speed sets the pace. To write each token, the machine reads the weights it uses once. Memory bandwidth ÷ weight size therefore gives a rough ceiling on tokens per second for one user. Real speeds come in below it.
| Kind | Example | Memory for models | Memory bandwidth | Paper ceiling, 27B model at 4 bits |
|---|---|---|---|---|
| Desktop GPU | NVIDIA GeForce RTX 5090 | 32 GB on the graphics card | 1,792 GB/s | About 130 tokens/s |
| Unified-memory desktop | NVIDIA DGX Spark | 128 GB shared with the system (a 64 GB version is coming) | 273 GB/s | About 20 tokens/s |
| Unified-memory Windows PC | NVIDIA RTX Spark laptops and desktops | Up to 128 GB shared with the system | Not published | Not known yet |
The paper ceiling is memory bandwidth ÷ 14 GB of weights. The pattern holds beyond this example: a desktop GPU is fast but holds only what fits in its memory, while unified-memory machines fit much larger models but write each token more slowly. Mixture-of-experts (MoE) models narrow that gap, because each token reads only a fraction of the weights. Workstations with several professional GPUs offer both size and speed, at a much higher price.
New this month: on 7 October 2026, NVIDIA and Microsoft announced RTX Spark Windows laptops and compact desktops, with up to 128 GB of unified memory and one petaflop of FP4 AI performance. Acer, ASUS, Dell, HP, Lenovo, Microsoft, MSI and Gigabyte are making them. NVIDIA says the laptops are available from 16 October and the compact desktops in November. Its announcement gives no memory bandwidth or prices, so compare them on your own tasks once independent tests appear.
The largest open models still need servers. DeepSeek V4.1 Flash, released under the MIT licence, is a 510 GB download: far more than a desktop GPU or a 128 GB unified-memory machine holds. In practice it runs on multi-GPU servers or through cloud APIs.
For measured numbers rather than paper ceilings, see our long-session tests on one RTX 5090.
A small evaluation you can actually repeat.
As a starting exercise, choose ten representative tasks with non-sensitive or authorised sample data. Include short and long inputs, a question the documents cannot answer, and one task that failed in a previous trial. Ten tasks are a pilot, not a statistically reliable quality benchmark.
- Write the acceptance criteria first. For a summary, specify the facts it must retain. For a coding task, supply checks. For retrieval, identify the source passages.
- Run the same task set on each candidate. Record the model, prompt, settings and date. Keep the task set fixed for the comparison.
- Record the result and the effort. Capture correctness, missing details, waiting time and the human corrections needed.
- Try realistic simultaneous use. Repeat with the number of active tasks you expect. Count failures and unfinished work, not just successful answers.
- Review the operational cost. Include the work of maintaining the system and reviewing its output.
A simple worksheet: task → expected result → observed result → completion time → corrections → pass/fail. Keep the original outputs so a future model update can be checked against the same examples.
Compare cost per useful result.
For local hardware, spread the purchase and setup cost over the period you expect to use it, then add power and maintenance. For a cloud service, estimate the actual subscription or usage cost for the same work. Include human review on both sides.
Divide by the number of results that meet your acceptance criteria. A system that produces more text can still take more time to turn into useful work. There is no universal break-even point: utilisation, model choice, support and task quality change the calculation.
Worked numbers, 10 October 2026
- Cloud usage. DeepSeek lists its V4.1 Flash model at US$0.30 per million input tokens and US$1.20 per million output tokens at peak times, which cover most of the Singapore working day (09:00–12:00 and 14:00–18:00, Monday to Friday). Off-peak rates are half that. A month with 50 million input and 5 million output tokens would cost about US$21 at peak rates. Agents use far more tokens than chat, because each step sends the conversation and tool results again; repeated input that hits the provider’s cache is billed at a much lower rate. Prices differ widely between models, so use the price list of the model that passes your tests.
- Local power. At SP Group’s regulated electricity tariff for 1 October to 31 December 2026, 31.16 cents per kWh including GST, a machine drawing 1 kW for 8 hours on 22 working days (176 kWh) costs about S$55 a month. Electricity bought from a retailer is priced differently.
Cheap cloud models make it hard for local hardware to win on price alone at low volume. Local hardware is more likely to pay off when use is heavy and steady, and teams often choose it for control over their data rather than for cost.
Common questions.
Is local AI cheaper than cloud AI?
Not by default. Cloud costs rise with use, while local hardware is mostly a fixed cost. At light use, cloud is usually cheaper. With heavy, steady use the comparison can turn, so work it out per useful result, as above.
Is local AI more private?
It can be, if every part of the workflow stays under your control. A local model connected to cloud tools, remote support or online backups still sends data out. Trace the data flow before you rely on it.
Does local AI make us PDPA compliant?
No setup is compliant on its own. The PDPA applies to personal data wherever the model runs. Local AI changes where the data goes, not whose obligations they are. See PDPA and AI.
Can a local model match ChatGPT or Claude?
On some everyday tasks, an open-weight model that fits your hardware may be good enough. On the hardest tasks, the strongest cloud models usually do better. The only reliable answer is a test on your own tasks.
What hardware do I need to run an LLM locally?
Start from the model that passes your tests. Its size at your chosen precision, plus room for context, sets the memory you need. The number of people using it at once sets the speed you need. See What fits on local hardware.
Look at a real test, with its limits.
Our Qwen / RTX 5090 agent test notes show why opening more sessions is different from completing more useful work. The tests used rented cloud hardware; they do not validate a finished workstation.
For a practical starting point, read your first local AI task. If you need help scoping a deployment, see our deployment process.
Sources.
- NVIDIA RTX Blackwell GPU Architecture (PDF; GeForce RTX 5090: 32 GB GDDR7, 1,792 GB/s)
- NVIDIA DGX Spark (specifications: 128 GB or 64 GB unified memory, 273 GB/s)
- NVIDIA: RTX Spark and AI agents on Windows PCs (7 October 2026)
- DeepSeek V4.1 Flash on Hugging Face (MIT licence; 510 GB of weights)
- DeepSeek API: models and pricing (checked 10 October 2026)
- SP Group: tariff information (1 October to 31 December 2026)
- Ollama FAQ
Prices and specifications change. Check the source before relying on a figure.