Counting LLM Tokens Without Surprise Bills

By BoltQuickTools team · May 18, 2026 · Updated November 8, 2026 · 6 min read

LLM API bills scale with a number you do not see until the response comes back. Batch jobs are the easy way to be surprised: a request that costs one cent looks harmless until you multiply it by 50,000 items. The short version of this post is to count tokens before you send, cap output explicitly, and pick the cheapest model that meets your quality bar.

Tokens, briefly

Models read tokens, which are sub-word units produced by a tokenizer. Tokenizers differ between providers and sometimes between generations of one provider's models, so count with the right one:

For a quick estimate, OpenAI's rule of thumb for English is about 4 characters or three quarters of a word per token, so 1,000 words is roughly 1,300 tokens. Code, JSON, and non-English text can differ a lot from that, so use a real tokenizer before you commit to a large run.

Cost arithmetic

The bill for one request is:

cost = (input_tokens / 1_000_000) * input_price + (output_tokens / 1_000_000) * output_price

Output usually costs several times more per token than input. Prices change often and vary by model, so take current numbers from the provider: OpenAI, Anthropic, Google.

Here is a worked example with made-up prices of $3 per million input tokens and $15 per million output tokens. A request with 2,000 input tokens and 500 output tokens costs:

input:  2,000 / 1,000,000 * $3  = $0.0060
output:   500 / 1,000,000 * $15 = $0.0075
total per request               = $0.0135
over 50,000 requests            = $675.00

The token counter gives a side-by-side estimate for pasted text. It estimates from character ratios and its price list is a snapshot, so confirm against the provider's tokenizer and pricing page before a large job.

Ways to control cost

1. Estimate before batches

Make the estimate part of the job, so the script refuses to start when it is over budget. Use your output cap as the output figure, which gives a worst case:

def projected_cost(n_items, avg_input, max_output, input_price, output_price):
    per_request = (avg_input * input_price + max_output * output_price) / 1_000_000
    return per_request * n_items

if projected_cost(len(items), 2000, 500, 3.00, 15.00) > BUDGET_USD:
    raise SystemExit("Projected cost is over budget")

2. Cap output on every request

An output limit sets the worst case for each call. The parameter name varies: max_tokens on Anthropic's Messages API, where it is required; max_completion_tokens on OpenAI Chat Completions and max_output_tokens on its Responses API; maxOutputTokens on Gemini. If you leave it out on OpenAI, the limit is whatever the model allows. On models that reason before answering, reasoning tokens generally count as output, so allow room for them.

Check the stop reason in each response. A reply cut off by the limit is usually unusable, and retrying it means paying twice.

3. Prefer the cheapest model that works

Smaller models are often good enough for summarization, classification, and extraction, but this depends on the task. Build a small set of real inputs with known good outputs, run it across a few model sizes, and use the cheapest one that passes. Run it again when a provider releases new models, since the lineup changes often.

4. Make output parse the first time

Each retry after a malformed response is another billable request. Where the API offers structured outputs or a JSON schema mode, use it instead of parsing free text. Temperature is a weaker control: setting it to 0 reduces variation but does not guarantee identical output, and some current models, including recent Claude models, reject a temperature setting altogether, so check the model's documentation.

5. Use prompt caching for repeated context

If thousands of requests share a long prefix, such as a system prompt, caching reuses it at a discount. The details differ by provider:

// Anthropic request body
{
  "model": "MODEL_ID",
  "max_tokens": 1024,
  "system": [
    {
      "type": "text",
      "text": "You are a support agent for...",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "messages": [{"role": "user", "content": "..."}]
}

Using the made-up $3 input price and Anthropic's multipliers, a 5,000-token prefix sent 1,000 times inside the cache lifetime costs $15.00 uncached. Cached, it is one write (6,250 token-equivalents) plus 999 reads (499,500 token-equivalents), or about $1.52.

Caching works on an exact prefix match. A timestamp or request ID near the start of the prompt means nothing after it is reused, so keep changing content at the end. Confirm hits in the usage fields of the response.

6. Batch where the API supports it

OpenAI's Batch API, Anthropic's Message Batches, and Gemini's batch mode all price asynchronous work at 50% of the standard rate, in exchange for results that can take up to 24 hours. For non-interactive work such as overnight classification, this is a discount that requires no change to your prompts.

7. Limit retries and loops

Code that re-prompts on failure keeps spending until something stops it, and an agent loop can do the same. Set a fixed limit on attempts per item and on total requests per job, and use the provider console's spending limits or alerts as a backstop.

Do not re-embed what has not changed

Embedding endpoints are cheap per call, which makes repeated work easy to overlook. If a retrieval pipeline embeds the same documents on every query, it pays again for identical results. Compute embeddings at ingestion and store them, keyed by a hash of the text plus the model name, then re-embed only when one of those changes.

Quick checklist

Related