What Are AI Tokens and How You Can Reduce AI Costs

Artificial Intelligence has arrived in the German SME (Mittelstand) sector. According to the Bitkom survey of 2026, 41 percent of German companies are now actively using AI. This represents an increase by a factor of 2.4 within a single year. However, this increase also comes with an uncomfortable number: 33 percent of AI users have found that AI has led to significantly higher costs than previously expected.

A central cost driver behind this is tokens. In this article, you will learn what tokens are and how you can reduce your token costs.

What are AI Tokens?

AI language models do not read words or letters. They break every text down into tokens—small building blocks, similar to the syllables of a language. A token might be a whole word, just part of a word, or a punctuation mark. The purpose of tokens is to compress text by replacing frequent character sequences with their own identifiers.

An example using the OpenAI Tokenizer makes this tangible: the English word “hello” is exactly 1 token. In contrast, the German “hallo” is broken down into 2 tokens. And if you write “héllo” with an accent, it is also 2 tokens—the special character costs extra.

Why is this important? Because AI is not billed in hours or licenses, but in tokens. Every token you input, and every token the AI outputs, costs money. Tokens are the AI’s taximeter: it ticks with every input AND every output.

How are Tokens Calculated?

Language models require mathematical vectors to perform their calculations. To do this, texts must first be converted in the following steps:

1. Sentence → Tokens. The example sentence “Tokenisierung macht Sprache berechenbar.” is broken down into individual tokens (building blocks) by a tokenizer. During this process, it was previously analyzed which character sequences occur frequently in the language. Instead of spelling these out individually, frequent text components receive their own identifiers (IDs). Words like “und” or “aber” essentially receive their own identifier in the AI alphabet. The token vocabulary of large language models like ChatGPT, Gemini, Claude, etc., comprises several hundred thousand tokens.

   Through tokenization, the size of the input is significantly compressed. From 40 characters, a list of only 8 tokens is created. But for costs, one thing matters most: the more tokens, the more expensive. You can try out how your own text is broken down, for example, in the OpenAI Tokenizer.

2. Tokens → Embeddings. Behind every token is a unique ID from the respective AI alphabet. Vectors are formed from these IDs. These vectors are called embeddings.

   These embeddings are the actual trick. Imagine a massive map of meaning, where every word has a fixed location. Terms with similar meanings lie close to each other: “king” and “queen” are neighbors; “king” and “refrigerator” are far apart. This is how the machine “understands” that “invoice” and “payment deadline” belong together thematically in your customer inquiry—even though it only processes numbers. Mathematically, even operations are possible, e.g., kitten - cat + dog = puppy.

   Embeddings are trained during AI training through the context of words. The mere position of a word in a sentence or relative to other words provides insight into its meaning. Which word is being sought here?

• “The [___] chases the mailman.”
• “I am going for a walk with my [___].”
• “The [___] barks loudly.”

Why German is More Expensive Than English

German texts cost more than English texts because they consist of more tokens. The reason for this is that tokenizers are predominantly trained on English texts. On top of that, we have our compound words (Komposita). A word like ”Suchmaschinenoptimierung” (search engine optimization) is a single term to a human—but for the tokenizer, it is a chain of several building blocks.

Most models are optimized for the English language. As a rule of thumb, based on a 2026 Tokenizer Comparison:

• English: approx. 1.17 tokens per word
• German: approx. 1.71 tokens per word

Depending on the model, a German text with 1,000 words can quickly reach 1,700 to 3,000 tokens. An extreme example with numbers: ”neunzehnhundertvierundachtzig” (nineteen eighty-four) is broken down into around ten tokens, while the English “nineteen eighty four” is only five.

The practical consequence: The same prompt consumes noticeably more tokens in German than in English.

Price Differences of AI Tokens

There are two things you must know regarding AI billing with tokens.

First: Input Tokens ≠ Output Tokens. You pay separately for what you put in (Input) and what the AI generates (Output). The output is consistently more expensive—typically three to ten times the price, and for large models, usually a factor of five to six. The reason: reading text is significantly easier for the machine than generating new text word by word.

Second: Token prices vary greatly from model to model. One million output tokens cost $1.20 with the OpenAI model gpt-5.6-luna, compared to $50 with the flagship model gpt-6-astra (as of September 2026, current OpenAI pricing overview here). Flagship models are much larger and require significantly more computation. They are used for complex, reasoning tasks.

Therefore, it makes sense to decide individually per task which model is appropriate. For example, a flagship model could also write Christmas cards or translate a text, but that would be absurdly expensive. It is more sensible to use flagship models for complex tasks such as research, software development, and evaluations, and to rely on smaller models for simple tasks. Tools like LiteLLM’s Auto Router can independently classify requests semantically and call the best model for the respective complexity.

Reducing AI Costs: Seven Concrete Ways

1. Model Routing: Not every request needs the flagship. A router checks the request and forwards it to the appropriate model. The Stanford approach ”FrugalGPT” by authors Lingjiao Chen, Matei Zaharia, and James Zou demonstrates that cascade routing can achieve a measured cost savings of up to 98 percent depending on the task.

2. Prompt Caching: Recurring prompt components, such as a fixed system instruction or a reference document, can be cached and then billed much more cheaply. Savings for “Cached Input” tokens are often in the range of 90 percent. The security company ProjectDiscovery operated its agent “Neo” with a 20,000-token system prompt and 20 to 40+ LLM steps per task. A single change improved the cache hit rate from 7 to 74 percent, and further optimizations brought it to 84 percent. In total, ProjectDiscovery served 9.8 billion tokens from the cache and reduced LLM costs by 59 percent. (see Source).

3. Batch Processing: For non-urgent tasks—nightly evaluations, mass classification, document analysis—OpenAI, Anthropic, and Google offer a batch mode with a current 50 percent discount. Batch processing uses the desired model and therefore has no impact on output quality. Results are not delivered immediately, but rather within 24 hours, for example.

4. Keep Context Short. Every unnecessary token in the prompt costs money. For long dialogues: summarize old conversation histories instead of carrying them along in their entirety.

5. Streamline System Prompts and Tools. Use precise, lean system instructions instead of page-long instructions. Every tool definition that an agent sends with every call counts: A complex agent with 20+ tools can consume thousands of tokens per call just for their descriptions. Anthropic has reduced the system prompt of Claude Code for its latest model generation by over 80 percent—without loss of quality (see Source).

6. Avoid Filler Phrases. Instructions like “Could you please summarize the following text for me?” can be reduced to “Summarize:“. Furthermore, it is not necessary to thank AI agents. Otherwise, the entire conversation history is processed again just to generate an unnecessary response.

7. Limit Output Length. The output is the expensive part. Request precise formats—for example, “Answer in a maximum of three bullet points” instead of an open essay. Answers can be hard-truncated for safety using parameters like max_tokens.

Outlook: When the Cost Question is Completely Reimagined

All seven levers optimize the token consumption of cloud AI. However, there is an alternative that changes the calculation from the ground up: Local AI models running on your own hardware—replacing the variable taximeter with calculable fixed costs. What this means for data protection, independence from US corporations, predictability, and your cost structure, you will learn in our next blog articles.

We have invested in a local AI server to process critical company data securely within our intranet. We have automated processes where we do not want to pass data to cloud service providers. In addition, we continue to use flagship models in the cloud to handle complex tasks.

Conclusion

AI is not billed in licenses or working hours, but in tokens—for every input and every output. This is likely why a third of companies are surprised by their own AI bills.

The good news: The cost drivers are known and manageable. In practice, four things make up the bulk of the bill: a model that is too large for a simple task, a bloated context, unused caching, and unlimited output lengths. Addressing these can often reduce AI costs by several tens of percent—without compromising quality.

The decisive factor is the order: measure first, then optimize. Those who do not know how many input and output tokens an application uses are optimizing blindly. Looking at your provider’s usage statistics is the cheapest first step—and usually the most insightful.