What the AI token counter does
The counter shows how many tokens a text takes for the language models you plan to send it to. For OpenAI's models it counts exactly, with the same o200k_base and cl100k_base tokenizers OpenAI publishes, run in your browser. For Claude, Gemini, DeepSeek and Mistral it applies each provider's published average and shows the result as an estimate. It also counts characters, words, lines and bytes, and shows where the text splits into tokens.
How to count tokens
- Paste a prompt, a document or any text, or open a text file of up to 2 MB.
- The counts follow your typing. Text over 1 MB is counted when you select Count or press Ctrl+Enter.
- Read the exact counts for OpenAI's tokenizers and the estimates for the other model families, each with its source.
- Under Tokens in the text, see each token as a colored box. Switch the tokenizer, or turn on Token IDs to see each token's number.
- Select Estimate cost to take the o200k_base count to the LLM Cost Calculator as the input tokens of a request.
Examples
Input
Hello, world!
Output
o200k_base (GPT-5, GPT-4.1, GPT-4o, o1, o3, o4-mini): 4 tokens
cl100k_base (GPT-4, GPT-3.5 Turbo, text-embedding-3): 4 tokens
Claude (estimate): ~4–5 tokens
Gemini (estimate): ~3 tokens
DeepSeek (estimate): ~4 tokens
Mistral (estimate): ~3 tokens
Characters: 13
Words: 2
Lines: 1
UTF-8 bytes: 13
Input
naïve café 🙂
Output
o200k_base (GPT-5, GPT-4.1, GPT-4o, o1, o3, o4-mini): 5 tokens
cl100k_base (GPT-4, GPT-3.5 Turbo, text-embedding-3): 5 tokens
Claude (estimate): ~4–5 tokens
Gemini (estimate): ~3 tokens
DeepSeek (estimate): ~4 tokens
Mistral (estimate): ~3 tokens
Characters: 12
Words: 2
Lines: 1
UTF-8 bytes: 17
Input
<|endoftext|>
Output
o200k_base (GPT-5, GPT-4.1, GPT-4o, o1, o3, o4-mini): 7 tokens
cl100k_base (GPT-4, GPT-3.5 Turbo, text-embedding-3): 7 tokens
Claude (estimate): ~2–5 tokens
Gemini (estimate): ~1–3 tokens
DeepSeek (estimate): ~4 tokens
Mistral (estimate): ~3 tokens
Characters: 13
Words: 1
Lines: 1
UTF-8 bytes: 13
The third text is counted as its 13 characters, not as the special token that marks the end of a text: tokenizers add special tokens themselves, so text that only looks like one counts as ordinary characters.
What a token is
A model does not read letters or words but tokens: pieces of text from a fixed vocabulary, about 200,000 entries in OpenAI's newest tokenizer. A common English word is one token, often with the space before it, as in " world". A rare word, a long number, code and many words in other languages split into several tokens. Punctuation, line breaks and runs of spaces are tokens too.
Providers bill every request by its input and output tokens and limit each request to a number of tokens, the context window. Knowing the count before you send a prompt tells you whether it fits and what it costs.
Exact counts and estimates
OpenAI publishes its tokenizers in its tiktoken library, so this page runs the same vocabulary files in your browser and its counts are exact. In tiktoken 0.14.0, the GPT-5, GPT-4.1, GPT-4o, o1, o3 and o4-mini models use o200k_base, and GPT-4, GPT-3.5 Turbo and the text-embedding-3 models use cl100k_base. OpenAI has not published the tokenizer of its GPT-6 models; for those, only the usage the API reports is exact.
For Claude, Gemini, DeepSeek and Mistral, the page has no tokenizer it can run in your browser: an exact count comes from the provider's own API, which needs a key and receives your text, or its own software. Instead, the page applies the average each provider publishes: a number of characters or words per token, or of tokens per character. Where a provider gives two figures, such as characters and words, the estimate is the range between them. We found no such figure for xAI's Grok or Meta's Llama models, so the page shows no estimate for them.
o200k_base and cl100k_base
cl100k_base has about 100,000 tokens in its vocabulary and came with GPT-4 and GPT-3.5 Turbo. o200k_base, with about 200,000, came with GPT-4o and is used by the newer OpenAI models. The larger vocabulary has more whole words of other languages, so the same text often takes fewer tokens: in the table below, the Turkish sentence takes 37 tokens in cl100k_base and 28 in o200k_base. For English the two often match.
Why other languages use more tokens
A vocabulary holds the pieces that were most common in the text it was built from. English words are often single tokens, while words in many other languages split into more pieces, so the same message can take very different counts:
| Language | Text | Characters | o200k_base | cl100k_base |
|---|---|---|---|---|
| English | The meeting is moved to Thursday at 3 pm. Please bring the signed contract. | 75 | 17 | 17 |
| Turkish | Toplantı perşembe saat 15.00'e alındı. Lütfen imzalı sözleşmeyi getirin. | 72 | 28 | 37 |
| Chinese | 会议改到星期四下午三点。请带上签好的合同。 | 21 | 16 | 22 |
The Turkish message has about as many characters as the English one but takes 65% more tokens with o200k_base, and the Chinese one has under a third of the characters but almost as many tokens. The published averages describe typical text, so estimates for text in other languages, and for code, are rougher than for English prose.
How to use fewer tokens
- Send JSON without indentation. Spaces and line breaks are tokens too; the JSON Minifier removes them without changing a value.
- Remove markup and repetition. HTML tags, attribute names and boilerplate repeated in every request add tokens without adding meaning.
- Ask for shorter answers. Output tokens usually cost several times as much as input tokens.
- Keep the repeated part at the start. It does not shrink the count, but providers bill a cached beginning of a prompt at a fraction of the price.
Options
| Option | Values | Default | When to change it |
|---|---|---|---|
| Tokenizer | o200k_base, cl100k_base | o200k_base | cl100k_base to see how GPT-4, GPT-3.5 Turbo or an embedding model splits the text. Both are counted either way. |
| Token IDs | On, Off | Off | On to show each token's number in the tokenizer's vocabulary, for comparing with an API's logprobs or a tokenizer in code. |
Common errors and fixes
| Code | What it means | How to fix it |
|---|---|---|
TOKENS_Note | The text contains something that looks like a special token, such as <|endoftext|>. It is counted as ordinary characters. | Nothing to fix: the count treats it as text. Remove it if it was pasted by mistake. |
INPUT_Error | The text is over 2 MB, about 500,000 tokens of English. | Count the text in parts and add the counts up. |