Skip to main content

04 - Gemini: 1M-Token Context, Thinking, Caching

One million tokens shared by input, thinking and output: what filling them costs with Gemini 4 and when implicit caching really saves money.

What do you think about this article?

04 - Gemini: 1M-Token Context, Thinking, Caching
Google Gemini for Developersfedericocalo.dev

At a Glance

One million tokens shared by input, thinking and output: what filling them costs with Gemini 4 and when implicit caching really saves money.

The context window of the Gemini 4 models is 1,048,576 tokens, and it is a single one: input, reasoning (thinking) and output all fit inside it together. The earlier models, Gemini 3 and 2.5, have two separate budgets instead, a million in and about 64,000 out. Three practical consequences follow. Filling the window costs money on every request: with gemini-4-argon that is about 2.10 dollars for input alone, and implicit caching brings it down to about 0.10 only if the prefix is identical and exceeds 4,096 tokens. Reasoning is billed as output and is controlled with thinking_level. Finally, if the input and max_output_tokens together exceed the window, the API lowers the output limit on its own to the space that is left.

The figures come from the official pages read on 13 November 2026, and in two weeks the price list has moved: Gemini 4 Carbon arrived on 10 November and Gemini 4 Hydrogen on 12 November, which, next to Argon (stable since 13 October), makes three models of the same generation.

Series "Google Gemini for Developers", article 4 of 6. Prerequisites: models and prices (article 1) and the call with the Interactions API (article 2). I am not affiliated with Google and I used no key: I ran the calculation listings, the one that calls the API is a schema to adapt, not executed, and the text says so.

What fits in a million tokens

The long-context page gives a concrete equivalence: a million tokens are 50,000 lines of code of 80 characters, or eight English novels of average length. The tokens page adds the rule: about 4 characters per token, and 100 tokens correspond to 60-80 English words. For Italian the documentation gives no figure, so your conversion depends on the language and on the format, and to know it you have to count. The method is count_tokens, which counts only the input and is called before you spend it; the tokens actually used then come back in the usage field of the response.

"""What fits in the Gemini 4 context window and how much output is left."""
WINDOW = 1_048_576          # tokens shared by input, thinking and output (Long context page)
CHARS_PER_TOKEN = 4         # rule from the Tokens page: "about 4 characters" per token


def max_output(input_tokens, max_output_tokens):
    """The API reduces the output to the space left in the window."""
    return max(0, min(max_output_tokens, WINDOW - input_tokens))


def estimate_tokens(chars):
    """ESTIMATE, not a count: for the real number you need count_tokens."""
    return -(-chars // CHARS_PER_TOKEN)


# 1) Check against the page: do 50,000 lines of 80 characters come to about a million tokens?
estimate = estimate_tokens(50_000 * 80)
print("50,000 lines x 80 characters ->", estimate, "estimated tokens;",
      "margin in the window:", WINDOW - estimate)

# 2) A hypothetical case file: 400 pages of 1,800 characters (my assumption, not the documentation's)
case_file = estimate_tokens(400 * 1_800)
print("400-page case file ->", case_file, "estimated tokens")

# 3) How much output is left: with the default limit (65,536) and with long decode (1,048,576)
for input_tokens in (100_000, 900_000, 1_000_000, 1_048_576):
    print("input", f"{input_tokens:>9,}",
          "-> output", f"{max_output(input_tokens, 65_536):>7,} (limit 65,536)",
          f"{max_output(input_tokens, WINDOW):>9,} (limit 1,048,576)")

# Output (run twice, identical output):
# 50,000 lines x 80 characters -> 1000000 estimated tokens; margin in the window: 48576
# 400-page case file -> 180000 estimated tokens
# input   100,000 -> output  65,536 (limit 65,536)   948,576 (limit 1,048,576)
# input   900,000 -> output  65,536 (limit 65,536)   148,576 (limit 1,048,576)
# input 1,000,000 -> output  48,576 (limit 65,536)    48,576 (limit 1,048,576)
# input 1,048,576 -> output       0 (limit 65,536)         0 (limit 1,048,576)

The first calculation puts the page in front of its own rule: 50,000 lines of 80 characters are 4 million characters, that is a million estimated tokens, which fits in the window with a margin of 48,576 tokens. The two statements are consistent. The second is an example with an assumption of mine: a case file of 400 pages of 1,800 characters is worth about 180,000 tokens, less than a fifth of the window. The function is called estimate_tokens on purpose: the real count is done by the service.

From here comes the idea to take away: a million tokens holds a folder of contracts or a codebase of medium size, but not an archive. Forty thousand pages of manuals do not fit in one request, however large the window.

The output ceiling: the window is shared

Here is the detail that breaks code written for the earlier models. If the input tokens plus max_output_tokens exceed 1,048,576, the API lowers the output limit to the space that is left. The third part of the output above shows it with max_output: with 900,000 input tokens and the default limit the answer reaches 65,536, but if you asked for long decode (long decode, up to 1,048,576 tokens) the ceiling drops to 148,576. With a million in input there are 48,576 tokens left for everything the model thinks and writes; with a full window, none.

Two clarifications from the thinking page. A synchronous, non-streaming call generates up to 32,768 tokens on Carbon and on Argon, while in streaming and on Hydrogen it reaches 65,536. To go beyond 65,536 you need background=True or a client-side continuation. Everything the model uses to reason takes space away from the answer: it is the same tank.

What it costs to fill the window

The price list of the Pricing page, read on 13 November 2026, in dollars per million tokens: Hydrogen 1.50 for input, 7.50 for output, 0.075 read from the cache; Argon 2.00, 10.00 and 0.10 until 30 November, then 4.00, 20.00 and 0.20; 3.8 Flash 0.75, 3.75 and 0.075 until 31 December, then double. The cache price is a twentieth of the input price on Argon and on Hydrogen, a tenth on 3.8 Flash. The listing does the calculation with exact fractions and repeats it in floating point with a formula written by hand.

"""Cost of a Gemini request, with and without cache, from the Pricing page (2026-11-13)."""
from fractions import Fraction as F

M = 1_000_000
# dollars per million tokens: (input, output with thinking, input read from the cache)
PRICE_LIST = {
    "gemini-4-hydrogen": (F("1.50"), F("7.50"), F("0.075")),
    "gemini-4-argon, until 30 Nov 2026": (F("2.00"), F("10.00"), F("0.10")),
    "gemini-4-argon, from 1 Dec 2026": (F("4.00"), F("20.00"), F("0.20")),
    "gemini-3.8-flash, until 31 Dec 2026": (F("0.75"), F("3.75"), F("0.075")),
}


def cost(prices, new, cached=0, output=0):
    """new = input tokens at full price; cached = tokens read from the cache;
    output = generated tokens, thinking included."""
    p_in, p_out, p_cache = prices
    return (new * p_in + cached * p_cache + output * p_out) / M


def dollars(x):
    return f"{float(x):,.2f}"


WINDOW = 1_048_576
print("== 1) A request that fills the window (input only)")
for name, pr in PRICE_LIST.items():
    print(f"{name:40} no cache {dollars(cost(pr, WINDOW)):>6}  fully cached {dollars(cost(pr, 0, WINDOW)):>6}")

CORPUS, QUESTION, ANSWER, N = 400_000, 200, 1_000, 100
print(f"\n== 2) {N} questions on a fixed corpus of {CORPUS:,} tokens (question {QUESTION}, answer {ANSWER})")
for name, pr in PRICE_LIST.items():
    without = N * cost(pr, CORPUS + QUESTION, 0, ANSWER)
    # best case: the first request is cold, the other 99 read the corpus from the cache
    with_cache = cost(pr, CORPUS + QUESTION, 0, ANSWER) + (N - 1) * cost(pr, QUESTION, CORPUS, ANSWER)
    print(f"{name:40} no cache {dollars(without):>7}  best case {dollars(with_cache):>7}  ratio {float(with_cache / without):.3f}")

REQUESTS, THINK = 10_000, 5_000
print(f"\n== 3) Surcharge of {THINK:,} thinking tokens for {REQUESTS:,} requests")
for name, pr in PRICE_LIST.items():
    print(f"{name:40} {dollars(cost(pr, 0, 0, REQUESTS * THINK)):>8} dollars")

# Second, independent calculation: same formula in floating point, written by hand
for name, (pi, po, pc) in PRICE_LIST.items():
    pi, po, pc = float(pi), float(po), float(pc)
    a = N * ((CORPUS + QUESTION) * pi + ANSWER * po) / M
    b = ((CORPUS + QUESTION) * pi + ANSWER * po + (N - 1) * (QUESTION * pi + CORPUS * pc + ANSWER * po)) / M
    assert abs(a - float(N * cost(PRICE_LIST[name], CORPUS + QUESTION, 0, ANSWER))) < 1e-9
    assert abs(b - float(cost(PRICE_LIST[name], CORPUS + QUESTION, 0, ANSWER) + (N - 1) * cost(PRICE_LIST[name], QUESTION, CORPUS, ANSWER))) < 1e-9
print("\nsecond floating-point calculation: matches")

# Output (run twice, identical output):
# == 1) A request that fills the window (input only)
# gemini-4-hydrogen                        no cache   1.57  fully cached   0.08
# gemini-4-argon, until 30 Nov 2026        no cache   2.10  fully cached   0.10
# gemini-4-argon, from 1 Dec 2026          no cache   4.19  fully cached   0.21
# gemini-3.8-flash, until 31 Dec 2026      no cache   0.79  fully cached   0.08
#
# == 2) 100 questions on a fixed corpus of 400,000 tokens (question 200, answer 1000)
# gemini-4-hydrogen                        no cache   60.78  best case    4.35  ratio 0.072
# gemini-4-argon, until 30 Nov 2026        no cache   81.04  best case    5.80  ratio 0.072
# gemini-4-argon, from 1 Dec 2026          no cache  162.08  best case   11.60  ratio 0.072
# gemini-3.8-flash, until 31 Dec 2026      no cache   30.39  best case    3.66  ratio 0.120
#
# == 3) Surcharge of 5,000 thinking tokens for 10,000 requests
# gemini-4-hydrogen                          375.00 dollars
# gemini-4-argon, until 30 Nov 2026          500.00 dollars
# gemini-4-argon, from 1 Dec 2026          1,000.00 dollars
# gemini-3.8-flash, until 31 Dec 2026        187.50 dollars
#
# second floating-point calculation: matches

Read the three blocks in order. A request that fills the window costs 2.10 dollars with Argon, 1.57 with Hydrogen and 0.79 with 3.8 Flash; with the whole prefix read from the cache it drops to 0.10, 0.08 and 0.08. A hundred questions on the same 400,000-token corpus cost 60.78 dollars with Hydrogen and 81.04 with Argon without cache. In the best case, with the first request cold and the other 99 reading the corpus from the cache, the bill falls to 4.35 and 5.80, about a fourteenth. On 3.8 Flash the ratio is 0.120 and not 0.072, because there the cache costs a tenth.

Watch the word "best": the page says that implicit caching is on and that the savings are passed on automatically, not that every request finds it. The third block is the other side of the bill: 5,000 extra reasoning tokens for 10,000 requests are 50 million output tokens, that is 375 dollars with Hydrogen and 500 with Argon. The listing assumes that total_cached_tokens is the part of the input already in the cache, as the field description in the SDK says: check it against the first invoice.

Long context or selective retrieval

The long-context page says something that marketing material leaves out. On retrieving a single fact in a huge text the accuracy reaches 99% in many cases; with several scattered facts the performance varies a lot with the content. Every query pays for the whole input: for a hundred facts at that 99% you would need a hundred requests, and the cache is the lever that cuts the cost. The page recommends putting the question at the end of the prompt and warns that a longer input lengthens the time to first token.

Everything in the contextSelective retrieval (RAG)
SetupAlmost noneIndex, splitting into chunks and ranking to maintain
Cost per questionHigh (all the input), reduced by the cacheLow (only the passages)
Cross-cutting questions (compare A with B)Strong pointWeak point: you need the right passage
Archive larger than the windowImpossiblePossible
Traceability of the sourceAsk for it in the promptNatural: you have the chunk

The table is my judgement, not a measurement. The rule I draw from it: long context wins when the material is a closed set, queried several times and with questions that cut across several documents; retrieval wins when the archive grows or changes every day. In both cases ask for the verbatim quotation of the passage, because it is the only way to notice an invention.

Thinking: reasoning is billed as output

The control is thinking_level, inside generation_config. The values depend on the model: Argon starts from medium and accepts low, medium, high; Hydrogen starts from high and adds xhigh; 3.8 Flash starts from medium and, like the Gemini 4 models, does not support minimal. On xhigh the page is explicit: reserve it for hard problems where high is not enough, because it increases latency and tokens and on simple tasks (extracting text or JSON) it can lead to "overthinking" or to exhausting max_output_tokens. The criterion is to try all the levels with a high limit and choose the lowest one that reaches your quality threshold.

On cost, the page says that the price is based on the full thinking tokens the model generates, even if the API returns only a summary. You get the summary with thinking_summaries set to "auto", but it can be empty, so the code must handle that case. And max_output_tokens includes the thinking tokens: if the model exhausts them while reasoning, the status is incomplete and the output may be missing, while the tokens already generated are still billed. The page recommends lowering thinking_level, not tightening the limit.

"""SCHEMA TO ADAPT, NOT EXECUTED: it needs a key and consumes tokens. Names taken from the documentation."""
from google import genai

client = genai.Client()                      # reads GEMINI_API_KEY from the environment
MODEL = "gemini-4-argon"
with open("case_file.txt", encoding="utf-8") as f:
    prompt = f.read() + "\n\nQuestion: which clauses of the specification contradict the standard contract?"

# 1) How heavy is the input? It counts only the input, before you spend it.
count = client.models.count_tokens(model=MODEL, contents=prompt)
print("input tokens:", count.total_tokens)

# 2) The call: minimal reasoning and no copy of the conversation on Google's server.
interaction = client.interactions.create(
    model=MODEL,
    input=prompt,
    generation_config={"thinking_level": "low"},
    store=False,
)

# 3) What you actually paid for: the four counters to cross-check against the invoice.
u = interaction.usage
print("input:", u.total_input_tokens, "| from cache:", u.total_cached_tokens,
      "| thinking:", u.total_thought_tokens, "| output:", u.total_output_tokens)

# Schema to adapt, not executed: compiled with py_compile, no call made. The attributes (interactions.create, models.count_tokens, the usage fields) exist in the google-genai 3.3.0 SDK, checked without network.

The block does three things in order. count_tokens tells you how heavy the input is before you spend it. interactions.create calls the model with thinking_level set to low and store=False: the second option avoids the copy of the conversation on the server, but then you cannot use previous_interaction_id in later turns, nor background execution. Finally usage returns the four counters to cross-check against the invoice. For a loop with tools this is not right: there you need state, and the choice between keeping it on the server or sending it back yourself is the one in article 2.

Implicit caching: three conditions and a step

Implicit caching is on for all models from 2.5 onwards and works both in conversations with state (previous_interaction_id) and in those without. For it to kick in, three conditions from the page must hold:

  • Minimum token threshold: 4,096 for Carbon, Argon, Hydrogen, 3.8 Flash, 3.6 Flash and 3.1 Pro Preview; 2,048 for 2.5 Flash and 2.5 Pro.
  • Identical prefix: the large, common content goes at the start of the prompt, the question at the end.
  • Requests close together in time: the page gives no duration, so I do not write one.

The three conditions hold together, and none is a guarantee: a 3,000-token prefix never reaches the threshold, and a single different character at the start breaks the prefix. To know whether it worked there is no way other than reading usage.total_cached_tokens on the response. There is also a rule specific to loops with tools on Gemini 4: the previous request is read in full from the cache only if the new one adds at least 2,050 tokens (output, thoughts and function results count); otherwise the last 2,050 tokens are reprocessed at full price.

"""Model of the implicit-cache rule for multi-turn loops on Gemini 4.
Rule read on the Context caching page: the previous request is read in full from the
cache only if the new one adds at least 2,050 tokens; otherwise the last 2,050 tokens of the
previous request are reprocessed at full price. It is a MODEL of the rule, not a measurement:
it assumes the cache always answers, and the page does not guarantee that."""
THRESHOLD = 2_050
P_IN, P_CACHE = 1.50, 0.075        # gemini-4-hydrogen, dollars per million tokens (Pricing, 2026-11-13)


def loop(initial_context, addition, turns):
    """Returns (tokens at full price, tokens read from the cache) summed over all turns."""
    full, cache = initial_context, 0         # the first turn is cold
    length = initial_context
    for _ in range(turns - 1):
        if addition >= THRESHOLD:
            read, new = length, addition
        else:
            read, new = max(0, length - THRESHOLD), addition + min(THRESHOLD, length)
        full, cache = full + new, cache + read
        length += addition
    return full, cache


print("initial context 30,000 tokens, 40 turns; tokens added at each turn:")
for addition in (500, 2_049, 2_050, 4_000):
    full, cache = loop(30_000, addition, 40)
    without = (full + cache) * P_IN / 1e6
    with_cache = (full * P_IN + cache * P_CACHE) / 1e6
    print(f"  +{addition:>5}: at full price {full:>9,}  from cache {cache:>9,}  "
          f"no cache ${without:.2f}  with cache ${with_cache:.2f}")

# Output (run twice, identical output):
# initial context 30,000 tokens, 40 turns; tokens added at each turn:
#   +  500: at full price   129,450  from cache 1,460,550  no cache $2.38  with cache $0.30
#   + 2049: at full price   189,861  from cache 2,608,359  no cache $4.20  with cache $0.48
#   + 2050: at full price   109,950  from cache 2,689,050  no cache $4.20  with cache $0.37
#   + 4000: at full price   186,000  from cache 4,134,000  no cache $6.48  with cache $0.59

The listing is a model of that rule, not a measurement: it assumes the cache always answers. It starts from a context of 30,000 tokens, runs 40 turns and changes only how much each turn adds. With 500 tokens per turn, 1,460,550 tokens out of 1,590,000 come back from the cache and the bill drops from 2.38 to 0.30 dollars. The step shows when you compare 2,049 and 2,050: one more token per turn brings the full-price tokens from 189,861 down to 109,950, and the bill from 0.48 to 0.37 dollars. In dollars it is little, but it explains why a pipeline that adds very little at each turn can pay more than one that adds a little more.

Explicit caching remains: it is not supported by Argon, by Hydrogen or by Interactions, and it is used only with generateContent on models 3 and 2.5. But from 1 December 2026 gemini-2.5-pro and gemini-2.5-flash are routed to 3.8 Flash and to 3.5 Flash-Lite (changelog of 2 November): it is not a safe harbour.

Three scenarios, where the window opens something new

These are scenarios built by me, with declared assumptions: none is a real case of a company or a person. For each one: what is done by hand today, what changes, what goes wrong.

Technology: an inherited service with no documentation

By hand today. Whoever inherits a service spends days reading modules to understand who calls what. What changes. If the code fits in about 400,000 tokens (assumption), you put it at the top of the prompt and ask cross-cutting questions, such as "which modules write to the orders table?". A hundred questions cost 60.78 dollars with Hydrogen without cache and 4.35 in the best case. What goes wrong. The model may omit one caller among many: the list is a starting point, not proof of absence, and for changes you still need text search and tests.

Small business: specification, bill of quantities and standard contract

By hand today. A technical firm compares a 400-page specification with its own bill of quantities and with the standard contract, looking for conflicting clauses: days of work for a senior technician (assumption). What changes. The three documents fit together in the window (assumption: 300,000 tokens), which is hard with selective retrieval because the conflict lies between documents. One pass costs about 0.45 dollars of input with Hydrogen (300,000 × 1.50 ÷ 1,000,000) and about 0.23 with 3.8 Flash. What goes wrong. An unreported penalty clause is a professional liability: the output is a list of points to verify, with a citation of the paragraph, checked by a technician. And the specification may contain confidential information of the client: first read how the terms treat data (article 5).

Healthcare: the internal protocols of a facility

By hand today. Staff search among hundreds of PDFs for the current version of a protocol. What changes. An internal service loads the current protocols, a closed set that changes on a few dates, and answers citing document, version and page; the fixed prefix takes advantage of implicit caching. What goes wrong. The Gemini API terms, in force since 23 March 2026, say: "You may not use the Services in clinical practice, to provide medical advice, or in any manner that is overseen by or requires clearance or approval from a medical device regulatory agency". Looking up the version of a document is not in itself clinical practice; if the use slides towards "what should I do with this patient", you enter the prohibition. Where you draw the line is not for me to decide: it is a question for whoever answers for compliance, before the code.

What you can do now

You can say in advance how much fits in a request and how much output is left, with count_tokens and with the subtraction that the shared window imposes. You can work out the cost of a full request with and without cache, and tell the best case from the guaranteed case. You can choose thinking_level according to the task and read usage to understand where the tokens went.

In article 3 you saw how the model calls functions, which is why a loop with tools quickly passes the 2,050-token threshold. In article 5 you find the limits that stop you and what the terms say about data; article 6 closes with the command-line tools.

What I did not verify

  • The conversion between tokens and pages for Italian text: the documentation gives only the 4-character rule and the comparison in English words.
  • How many requests really find the implicit cache: the listings are best-case models, not measurements, and I used no key.
  • The time within which two requests count as "close together" for the cache: the page does not give it.
  • Retrieval benchmarks on long contexts for the current models: none read at the source.

These four points have in common that they can only be resolved with a measurement on your own data: the token estimate with count_tokens, the cache share with usage on a real sample of requests, and a retrieval test on your corpus. Until then the figures in the article are orders of magnitude, and the text treats them that way.

Frequently asked questions

Does a million tokens mean I can send everything?

A lot fits, but every request pays for all the input. With Argon, at 2 dollars per million until 30 November 2026, filling the window costs about 2.10 dollars a call; with Hydrogen about 1.57. And retrieval of several scattered facts loses accuracy.

Are reasoning tokens billed?

Yes. The output price includes thinking tokens, and max_output_tokens includes them: if the model exhausts the limit while reasoning, the answer is marked incomplete. To save money lower thinking_level instead of cutting the limit.

Is implicit caching on by itself?

Yes, for 2.5 and newer models. You need a common prefix of at least 4,096 tokens on Gemini 4 models. Explicit caching is supported neither by Argon, by Hydrogen nor by Interactions.

Sources

  • Long context - unified window of the Gemini 4 models, separate budgets of Gemini 3 and 2.5, equivalences of a million tokens, retrieval accuracy, consulted on 2026-11-13
  • Gemini thinking - levels per model, billing of thinking tokens, max_output_tokens and limits of the Gemini 4 models, consulted on 2026-11-13
  • Context caching - minimum thresholds, the 2,050-token rule, explicit caching not supported, consulted on 2026-11-13
  • Gemini Developer API pricing - input, output and cache prices of Hydrogen, Argon and 3.8 Flash, consulted on 2026-11-13
  • Understand and count tokens - the 4-character rule, count_tokens, usage fields, consulted on 2026-11-13
  • Release notes - dates of Argon, Carbon and Hydrogen, routing of the 2.5 models from 1 December 2026, consulted on 2026-11-13
  • Gemini API Additional Terms of Service - ban on use in clinical practice (in force since 23 March 2026), consulted on 2026-11-13
  • google-genai on PyPI - version 3.3.0 installed in a test environment to check the SDK names without calls, consulted on 2026-11-13

Commenti

Caricamento commenti...

Sign in to comment

New articles and tools on Telegram too

The Telegram channel flags the site's new articles and tools.

Did you like this article?

Explore other content on the blog or discover my projects