Local LLM or cloud API: how to compare the real costs

A local LLM can be economical when a suitable model handles a sustained workload on hardware you can operate efficiently. A cloud API can be cheaper when usage is light, demand is unpredictable or a hosted model completes the task more reliably.

Compare the cost of completing the same work to the same standard. Dividing a GPU’s purchase price by its theoretical token output leaves out utilisation, support and whether those tokens produce useful results.

About the figures: every price and workload in the worked example below is an illustrative assumption, not a supplier quote, current API tariff or Northloom benchmark. Replace them with your own measurements and quotations. Documentation was reviewed on 19 October 2026.

Northloom develops the Krasis inference runtime and offers private AI infrastructure planning. Our assessments start with whether a configuration can do the work, then compare capacity and operating costs; local hardware and cloud APIs both belong in that decision. See our inference engineering work.

Define the job before comparing the bill

Start with a representative set of tasks. Establish acceptable answers, completion time and review requirements. Then test both the local configuration and the hosted model against that set.

For a document workflow, record documents completed correctly, rather than tokens alone. Include retries, extraction failures, context truncation and human correction time. A cheaper model that needs frequent rework can produce the more expensive service.

Keep the deployment’s data requirements explicit. If a workload must stay within a particular boundary, compare options that satisfy it. A public API, a dedicated hosted deployment and a server in your office have different operating arrangements. Trace document processing, retrieval, logs and tools as well as the inference endpoint.

Our model suitability guide explains how to establish this baseline.

Calculate the API side from actual requests

For a simple token-priced API, estimate:

Monthly inference cost = input tokens ÷ 1,000,000 × input rate + output tokens ÷ 1,000,000 × output rate.

Add any separately billed caching, tools, storage, embeddings or other services. Agent workflows may make several model calls for one user task; account for all of them. Token counts can also differ between models, so use each candidate’s tokenizer or usage records.

Request patterns matter. Providers can distinguish ordinary input, cache writes, cache reads and output, and may offer batch processing or different service tiers. The Claude API pricing documentation is one example of that structure. Check the selected provider’s current terms and prices when preparing the comparison.

Use one currency and a consistent tax basis. Include currency conversion where relevant, and test a busy month as well as the average. A consumer chat subscription is a separate product; do not assume it pays for application API calls.

Include the cost of owning the local service

For a planning comparison over a defined period, spread the net hardware cost across that period, then add operating costs:

Monthly local cost = hardware allocation + electricity + operations + other local service costs.

Include the complete machine, RAM, storage and installation, rather than the GPU alone. Allow for software licences, monitoring, access management, updates, replacement cover and recovery. Treat any assumed resale value cautiously.

Electricity should use measured wall power, including idle periods. The GPU’s maximum power rating is not the server’s average draw. Include cooling or hosting charges where they apply.

Staff time needs an explicit allowance. Decide who handles upgrades, failed jobs, access changes and capacity problems. API deployments also require application maintenance and security work: include costs on both sides, then separate genuinely shared work from the additional effort each option requires.

A worked comparison

Suppose a business expects 20,000 completed tasks per month. For the initial calculation, assume each task uses one request with 8,000 input tokens and 1,000 output tokens, and both options meet the same quality target without retries.

At assumed API prices of £2 per million input tokens and £8 per million output tokens, the monthly calculation is:

• 160 million input tokens × £2 per million = £320.
• 20 million output tokens × £8 per million = £160.
• Total assumed API inference spend = £480 per month.

Now suppose a local system has the following budget. These are deliberately round planning assumptions, all on the same tax basis.

On these assumptions, the API is about £64 per month cheaper. That conclusion is conditional on the local machine actually handling the workload and the operations allowance being sufficient. It also excludes application costs assumed equal on both sides.

The local option requires £9,000 upfront. Spreading that over three years helps compare costs, but does not remove the cash requirement, financing costs or risk that requirements change before then.

Find where the result changes

The assumed API cost per task is £0.024. Dividing £543.88 by £0.024 gives an approximate crossover at 22,662 tasks per month.

That is a screening calculation. It holds the local monthly budget constant, assumes the same request sizes and ignores extra power or support as usage rises. Check capacity and refine those assumptions before treating it as a buying threshold.

Under the same simplified model:

• At 5,000 tasks per month, API inference costs £120. The local system is difficult to justify on inference savings alone.
• At 50,000 tasks per month, API inference costs £1,200. Local operation could save money if the machine can deliver the workload within the required time.
• If API rates halve, the approximate crossover doubles. If local operating effort increases, the local budget also rises.

A volume total does not establish capacity. Fifty thousand tasks arriving steadily over a month are different from the same number arriving before a daily deadline. Measure input processing, output generation and queueing under the expected arrival pattern. Our performance guide explains the distinction.

Test quality, utilisation and recovery

Three questions often change the result more than electricity does.

How much output is usable? Include repeat requests and human review. If one option fails more often, compare its cost per accepted task and the cost of correcting failures.

How much capacity will you use? A lightly used server still ties up capital and consumes idle power. Existing hardware can reduce the additional cash spend, but account for work it displaces and any upgrade it needs.

What happens when it is unavailable? A single local machine may be sufficient for an overnight job that can wait. A service with strict availability requirements may need spare capacity or a fallback. Include the required recovery arrangements and any API commitments on the corresponding side.

A hybrid deployment may fit: suitable routine or sensitive tasks stay local, while approved tasks use an API. Budget for routing, testing and operating both paths. Fallback must respect the workload’s data boundary.

What to put in a buying decision

Prepare low, expected and high-demand scenarios over the same planning period. Each should contain measured quality, task volume, peak demand, complete costs, ownership responsibilities and a recovery plan.

Local inference becomes a stronger candidate when a tested model meets the work, demand is sustained and operating capacity already exists. An API becomes a stronger candidate when the volume is uncertain, bursts are difficult to provision for or hosted model quality materially reduces rework. Data requirements may decide the shortlist before either financial comparison begins.

Northloom’s infrastructure planning assessment can review those assumptions before a hardware purchase. If you already have a design or supplier proposal, an architecture review examines whether its model, capacity and cost assumptions stand up together.