Loading community...
Loading community...
by ctaxnagomi
I put five dollars on a card and told myself it had to last a week. Not because five dollars is a large amount of money, but because it is a clean unit. If a model can eat five dollars in an afternoon doing work you could have done yourself, you learn something no benchmark will ever tell you.
I had three months and one question: what does GPT 6 Astra actually cost to use well? Here is the answer I ended with. Astra at ten dollars in and fifty dollars out will burn five dollars in about forty minutes on defaults. The same model, same tasks, same quality bar, ran forty five times longer on a configuration that took two months to work out.
Four defaults did it, and none are exotic. The SDK sends full conversation history on every turn with no cache breakpoint. Reasoning effort defaults to medium. Verbosity defaults to medium. There is no output cap. Multiply those together and every turn costs more than the last, because every turn carries every previous turn inside it.
The first afternoon cost a dollar forty across eleven turns of a to do app. By turn nine the request was hauling nine thousand tokens of history to answer a question that needed four hundred. Nothing in my client was wrong. The client did exactly what the documentation says. The default path is not a cheap path. It is a correct path that happens to be expensive.
The four defaults, priced on their own:
No cache breakpoint. Every reused token bills at ten times the cached rate.
Medium effort. Reasoning tokens bill as output tokens, at fifty dollars per million.
Medium verbosity. Formatting and restatement bill at output rates too.
No output cap. Nothing stops a runaway generation except the context window.
Astra bills input at ten dollars per million tokens. A cache read on those same tokens bills at one dollar. A cache write bills at twelve fifty for the thirty minute lifetime, which is one and a quarter times ordinary input, not a surcharge on top. That ratio is the whole game: ten to one between fresh and reused input, fifty to one between output and reused input.
The mechanism is a breakpoint. Everything before the mark caches, everything after bills fresh. The default puts the mark in the worst place, the end of the latest message, which includes the new question, so the next request has a different tail and misses. Move the mark back to the static boundary and hit rates go from near zero to above ninety percent.
Effort none on Astra returns a 400, because Astra has no none and low is the floor. Two days of client code assumed otherwise, so every request failed at the parameter layer after the input had already counted against the tokens per minute budget. The second failure is silent: too low an output cap and the model spends the allowance on reasoning, returns incomplete with reason max output tokens, and no text. You pay input plus reasoning and receive nothing.
Reasoning tokens bill as output tokens. For two weeks I treated low as a compromise. Then I ran the same twelve tasks at low, medium and high and scored them blind. Low matched high on nine of twelve. High beat low on two, both concurrency bugs where extra thinking surfaced a race I had missed. High beat low on nothing else, and cost two point eight times as much.
So effort is a per task decision. Change it with a configuration update item, not the request level parameter. The parameter rewrites hidden instructions and kills the prefix. The item does not.
Low verbosity plus an output contract. The contract is the part that works, because verbosity alone says be brief without saying what brief must still contain: lead with the conclusion, keep every required fact, cut introductions and reassurance. Astra likes lists and tables and will hand you a scannable answer to a yes or no question. At fifty dollars per million output tokens, formatting is not free.
The most expensive thing I wrote in September was a two line prompt. It was vague, so the model explored: six files, a plan, two revisions, code, tests, then a question it could have asked in turn one. Fourteen turns and a dollar sixty for eleven lines. The same change with the path, the expected behaviour and the constraint took one turn and four cents. A precise prompt is not longer, it is earlier.
Total spend, eighteen dollars and forty cents. Most of it was rework rather than generation, because a model at medium effort on a vague prompt will explore, and exploration bills. The three things that bent the curve were all subtraction: fewer history tokens, lower effort, shorter output.
Cost is not in the dashboard. The dashboard is a post mortem. It is in usage, read on every response. Ordinary input is total minus both cache fields. Weight it at one, cached at a tenth, thirty minute writes at one and a quarter, twelve hour writes at two and a half, output at five. If cached tokens is zero across a session that should reuse a prefix, your breakpoint is misplaced.
The number that matters is not cost per token or per turn. It is the read to write ratio of the task. Read forty thousand tokens and write two thousand and the turn costs about fourteen cents. Invert it and the same turn costs two dollars. Fourteen times the price, and the only variable is direction. Debugging reads. Scaffolding writes. Review reads. Migration writes. That distinction predicted my invoices better than every prompt technique combined.
Image input bills at input rates and re-bills every turn the image stays in context, so one high detail screenshot can quietly cost more than the conversation around it. Use low detail unless the pixels carry the answer, and drop images once described.
Batch and Flex bill at half the standard rate. If a job does not need an answer now, it goes to Batch, which is an instant fifty percent cut on the workload hardest to make cheap. Fast doubles the price and Astra fast carries no latency SLA. Interactive uses standard, everything else uses Batch, nobody touches Fast unless someone is waiting.
Pro mode does more model work before returning one answer and bills every token of it at standard rates. It is a multiplier on top of effort, not a quality switch. Three of my four uses were worth it, all deep reviews with clear criteria. The fourth was a routine refactor where the gain was invisible and the invoice was not.
Input above two hundred and seventy two thousand tokens bills the entire request at double input and one and a half times output. Not the portion over the line. The whole request. A mature repository is what pushes you over it, because a mature repository is what tempts you to paste it. Retrieve, do not paste. Compact around two hundred thousand, not two seventy.
A prefix has to clear one thousand and twenty four tokens before it is cacheable at all. Below that nothing caches and you pay full price with extra steps. The trap runs both ways: a short block never caches so a long session re-bills it, and padding costs you a thousand tokens of context. My fix was not padding, it was moving static rules into a developer message that carried the reference text and cleared the floor on its own.
The biggest cut of the three months was not a parameter. It was moving work off the token meter entirely.
A local tool call costs tokens only for the request and the compact result. The work runs on my machine and bills at nothing. Four thousand tokens of written analysis is twenty cents. The same work as a run_tests call returning a hundred and eighty tokens is under a cent. Same answer, twenty times cheaper, and the second one actually ran the tests.
A grep instead of pasting thirty files is the same trick on the input side. Forty thousand tokens of pasted context, forty cents uncached, collapses to six hundred tokens of path and line hits. Retrieval by tool is the only version that scales, because you cannot cache a file you have not read yet.
One rule shaped everything after month two, and it fits in eleven words: RULE IS YOU DONT RELY MUCH ON 3rd party services.
Every hosted tool, every remote MCP server, every vendor SDK is a line in your bill you do not control and a failure mode you cannot debug at three in the morning. Normally 1 local tool call cost 1 token while the standard method requires the model to reason and cached in order to be usable. So the harness split in two. A universal agentic runtime decides what runs and where, cloud, local or enterprise, the same three modes most hybrid products already carry. A universal agentic execution layer actually does it, dispatching local processes, loopback MCP and the sandbox. The runtime never executes. The layer never decides.
Third party becomes a deliberate exception with a price attached. Web search is the one I still allow, at ten dollars per thousand calls, and it stays off unless the task needs the outside world. The audit function below fails the boot if a hosted tool sneaks into a local run, because a rule you cannot enforce is a preference.
Two months produced this. It is unglamorous and it works.
import os, io, json, asyncio, hashlib, zipfile
from openai import OpenAI
from agents import Agent, Runner, function_tool
client = OpenAI()
AGENTIC = True
LOCAL_ONLY = True
RUNTIME = os.getenv("AGENT_RUNTIME", "local") # cloud | local | enterprise
THIRD_PARTY_OK = not LOCAL_ONLY
# ---- instruction pack (CTECX Formatter, ctecx_instruct@1) -------
# Load, order deterministically, hash. The hash is the cache key:
# same pack, same prefix, same cache. owner/ rules stay out of the ship.
RAW = open("ctecx_instruct@1.zip", "rb").read()
PACK_ID = hashlib.sha256(RAW).hexdigest()[:12]
with zipfile.ZipFile(io.BytesIO(RAW)) as z:
INSTRUCT = "\n\n".join(z.read(n).decode() for n in sorted(z.namelist())
if n.endswith(".md") and not n.startswith("owner/"))
STABLE_PREFIX = INSTRUCT + "\n\n" + open("prompts/agent_prefix.md").read()
# ---- local tools. Short descriptions, they are prefix tokens ----
LOCAL_TOOLS = [
{"type": "function", "name": "grep", "strict": True,
"description": "Regex search. Returns path:line only, never bodies.",
"parameters": {"type": "object", "required": ["pattern"],
"properties": {"pattern": {"type": "string"}}}},
{"type": "function", "name": "read", "strict": True,
"description": "Read a line range of one file.",
"parameters": {"type": "object", "required": ["path", "from", "to"],
"properties": {"path": {"type": "string"}, "from": {"type": "integer"},
"to": {"type": "integer"}}}},
{"type": "function", "name": "apply_patch", "strict": True,
"description": "Apply a unified diff.",
"parameters": {"type": "object", "required": ["diff"],
"properties": {"diff": {"type": "string"}}}},
{"type": "function", "name": "run_tests", "strict": True,
"description": "Run the suite. Returns pass/fail and first failure.",
"parameters": {"type": "object", "required": [], "properties": {}},
"async": True},
# agentic browser: local headless over CDP. Returns a11y text, no images.
{"type": "function", "name": "browser", "strict": True,
"description": "Drive local headless browser. Returns a11y snapshot, no images.",
"parameters": {"type": "object", "required": ["url"],
"properties": {"url": {"type": "string"},
"steps": {"type": "array", "items": {"type": "string"}}}},
"async": True},
]
# ---- local MCP over loopback, catalogs deferred ---------------
MCP = [
{"type": "mcp", "server_label": "fs", "server_url": "http://127.0.0.1:7070/mcp",
"require_approval": "never", "defer_loading": True},
{"type": "mcp", "server_label": "dgui-hypermem",
"server_url": "http://127.0.0.1:8787/mcp",
"require_approval": "never", "defer_loading": True},
]
# ---- universal execution layer --------------------------------
EXEC = {"grep": do_grep, "read": do_read, "apply_patch": do_patch,
"run_tests": do_tests, "browser": do_browser}
def audit(tools):
hosted = [t["type"] for t in tools
if t["type"] in ("web_search", "file_search", "code_interpreter")]
if not THIRD_PARTY_OK and hosted:
raise SystemExit("hosted tool in a local run: " + str(hosted))
TOOLS = [*LOCAL_TOOLS, {"type": "programmatic_tool_calling"}, *MCP] if AGENTIC else []
audit(TOOLS)
# ---- router: read the request, then pick. Heuristics are free. --
# General recall goes to my own models on a loopback gateway, never
# to a GPT model. Mature, cached, settled project = recall work.
GENERAL = ("deepseek-v4", "big-pickle")
GW = OpenAI(base_url=os.getenv("LOCAL_GATEWAY", "http://127.0.0.1:11434/v1"),
api_key="local")
TABLE = {"scaffold": ("gpt-6-luna", "low"), "ui": ("gpt-6.1-terra", "low"),
"feature": ("gpt-6.1-sol", "low"), "review": ("gpt-6.1-sol", "medium"),
"debug": ("gpt-6-astra", "high"), "architect": ("gpt-6-astra", "medium"),
"general": (GENERAL[0], "low")}
def route(text, diff=None):
t = text.lower()
if diff or "review" in t: return TABLE["review"]
if any(k in t for k in ("fails", "traceback", "race", "leak", "error")):
return TABLE["debug"]
if any(k in t for k in ("plan", "design", "tradeoff", "architecture")):
return TABLE["architect"]
if any(k in t for k in ("add ", "implement", "wire", "refactor")):
return TABLE["feature"]
if any(k in t for k in ("scaffold", "boilerplate", "cmake", "rename")):
return TABLE["scaffold"]
if any(k in t for k in ("screen", "layout", "css")): return TABLE["ui"]
# ambiguous only: one luna call, capped at 8 tokens
lab = client.responses.create(
model="gpt-6-luna", max_output_tokens=8, reasoning={"effort": "low"},
input="scaffold|ui|feature|review|debug|architect|general\n" + text
).output_text.strip()
return TABLE.get(lab, ("gpt-6.1-sol", "low"))
BASE = dict(
reasoning={"effort": "low", "context": "current_turn"},
text={"verbosity": "low"},
max_output_tokens=1200,
store=True,
tools=TOOLS, # frozen at import, never rebuilt
tool_choice="auto" if AGENTIC else "none",
parallel_tool_calls=True,
prompt_cache_options={"mode": "explicit", "ttl": "30m"},
)
spend = {"usd": 0.0}
CEILING = 4.50
def meter(r):
u, d = r.usage, r.usage.input_tokens_details
ordinary = u.input_tokens - d.cached_tokens - d.cache_write_tokens
spend["usd"] += (ordinary 10 + d.cached_tokens 1
+ d.cache_write_tokens 12.5 + u.output_tokens 50) / 1e6
if spend["usd"] > CEILING:
raise RuntimeError("budget guard")
def dispatch(call):
fn = EXEC.get(call.name)
try:
out = "ERROR unknown tool" if fn is None else fn(**json.loads(call.arguments))
except Exception as e:
out = "ERROR " + type(e).__name__ + ": " + str(e)[:200]
return {"type": "function_call_output", "call_id": call.call_id,
"output": str(out)[:1500]}
def start_on(text, model, effort):
cli = GW if model in GENERAL else client
r = cli.responses.create(input=[
{"role": "developer", "content": [{
"type": "input_text", "text": STABLE_PREFIX,
"prompt_cache_breakpoint": {"mode": "explicit"},
}]},
{"role": "user", "content": text},
], **dict(BASE, model=model, store=(cli is client),
reasoning={"effort": effort, "context": "current_turn"}))
meter(r)
return r
def start(text, diff=None):
model, effort = route(text, diff) # once per session, then hold it
r = start_on(text, model, effort)
if model in GENERAL and r.output_text.lstrip().startswith("ESCALATE"):
r = start_on(text, "gpt-6.1-sol", "low") # local punted, pay once
return r
# ---- dual async agents, each narrow, neither gets Astra --------
reader = Agent(name="reader", model="gpt-6.1-sol",
instructions="Read only. Return path:line and one line of why.",
tools=[function_tool(EXEC[n]) for n in ("grep", "read")])
prober = Agent(name="prober", model="gpt-6.1-sol",
instructions="Drive browser and suite. Return one compact repro: "
"steps, expected, actual, first failing line.",
tools=[function_tool(EXEC[n]) for n in ("browser", "run_tests")])
@function_tool
async def fanout(brief: str) -> str:
"""Run reader and prober concurrently on the same brief."""
a, b = await asyncio.gather(Runner.run(reader, input=brief),
Runner.run(prober, input=brief),
return_exceptions=True)
cut = lambda r: str(getattr(r, "final_output", r))[:1200]
return "READ\n" + cut(a) + "\n\nPROBE\n" + cut(b)
# ---- orchestrator: the only agent that gets Astra --------------
orchestrator = Agent(
name="orchestrator", model="gpt-6-astra", instructions=STABLE_PREFIX,
tools=[function_tool(EXEC[n]) for n in ("grep", "read", "apply_patch")] + [fanout],
handoffs=[reader, prober],
)
# Runner.run(orchestrator, input=text, session=session) # session carries usage
The instruction pack comes from CTECX Formatter as INSTRUCT dot zip. It is loaded, filtered, sorted and hashed before anything else runs. Sorted order matters because a prefix that shifts by one byte costs a full cache write, and the hash is the cache key: same pack, same prefix, same cache.
Three parts carry the caching. The response id chain means you send only the new message. The explicit breakpoint on the static prefix keeps the cache alive. The configuration update escalates effort for one turn without invalidating anything. Programmatic tool calling is the compressor, writing a short program that reduces results before anything reaches context, so a forty thousand token log arrives as a four hundred token summary. Every tool result is hard truncated in the dispatcher, because a tool that can return unlimited text is a tool that can return your whole bill.
Every tool call is another turn, and every turn re-reads the prefix. Cheap when cached, ruinous when not, so a chatty loop is worse than one large write. Cap the rounds at six and let a run end incomplete rather than spiral. Force compact returns in the description itself. Put async true on anything slow so the model keeps working instead of idling.
Everything so far was one agent. The last piece is splitting it, and the counterintuitive part is that splitting is the cheaper arrangement.
The orchestrator is the only agent that gets Astra. It holds the prefix, the patch tool and the decision. It does not read files and does not drive the browser. Two narrow agents do that: reader, with grep and read, and prober, with the browser and the suite. Both run on sol at a fifth of the price.
Fanout runs them concurrently on the same brief. Wall clock halves, and the orchestrator never sees their working, only two twelve hundred character summaries. Two cheap agents with compressed returns cost less than one expensive agent doing both jobs serially, because the expensive model is not paid to read.
Two local servers make this work without breaking the rule.
The browser is headless and driven over CDP on loopback. The important detail is that it returns an accessibility snapshot as text instead of a screenshot. A high detail screenshot is thousands of image tokens re-billed every turn it stays in context. The same page as a snapshot is a few hundred text tokens, cached and diffable, so a second look costs almost nothing.
The memory is dgui-hypermem on loopback, hybrid recall with vector and full text fused behind a rerank layer. It matters for cost because an agent that forgets pays twice: it re-reads the repo every session at full input rate, and re-derives decisions it already made. Recall turns that into a few hundred cached tokens. Both servers are deferred, so their catalogs stay out of the prefix until asked.
Manual routing was a tax on my attention and I got it wrong under pressure. So the router is code now, and it reads the request.
Heuristics first, because they cost nothing. A diff or the word review goes to sol at medium. Fails, traceback, race, leak or error goes to Astra at high. Plan, design or tradeoff goes to Astra at medium. Add, implement or refactor goes to sol. Scaffold, cmake or rename goes to luna. Screen, layout or css goes to terra. Only when none of those fire does one luna call classify, capped at eight tokens.
General queries do not reach a GPT model at all. They go to my own DeepSeek or Big Pickle over a loopback gateway, because on a mature project with settled conventions and a large cached prefix, a general question is recall work, not reasoning work. Those models hold a cached project well and cost nothing per token. If one punts it returns the word ESCALATE, which buys exactly one turn on sol and no more.
Two constraints keep it honest. The model is chosen once per session and held, because every model has its own cache and flipping mid-session throws the prefix away. The local gateway has no server side store, so those sessions replay a trimmed history instead of chaining response ids.
Routing is a floor, not a ceiling. Nothing routes itself to Astra because the task felt big, and nothing stops a person overriding it.
I pointed all of it at a live project: an open TypeScript service, a closed compiled client, and a deadline. The configuration turned five dollars into a working week on the open half.
The closed half was the interesting result. It is a single executable, no source, licence granting no right to reverse engineer. Converting it to C++ is a clean room rewrite, not a port.
The naive version is a trap, and I only saw it because Astra found it. The client runs workloads through a JavaScript sandbox. Node is not only the language there, it is the sandbox host and the transport. Rewrite in C++, drop Node, lose both, then embed a JS engine or keep a Node sidecar and ship a binary that still needs Node on the path. The port that works splits it: C++ for the supervisor, listener and sandbox lifecycle, TypeScript for orchestration as a child process, schema first. That decision cost sixty dollars instead of a hundred and fifty.
The lesson generalises. A rewrite is a shape question before it is a language question, and the cheap model will happily write the wrong shape faster. That is the honest case for the premium: not that Astra writes more code, but that it finds the problem before you spend a month building the wrong one.
Five dollars was never the unit. The unit is whether you know which tasks are write heavy, and whether your breakpoint sits before or after the part that changes.
The rules, in the order they saved me the most money:
Send the new message only. Use the response id chain.
Put the explicit breakpoint after the static prefix, before anything volatile.
Start at low effort. Escalate with a configuration update, one turn at a time.
Low verbosity, an output contract, a capped output.
Give the model local hands. Every output token it can replace with a tool call is a token you do not buy.
Do not rely much on third party services. Local first, hosted as a priced exception, enforced in code.
Route on the request, not on your mood. Classify once, hold the model for the session.
Split the work. Cheap narrow agents in parallel, expensive model only orchestrating.
Truncate every tool result and cap the rounds. Tools multiply turns.
Meter every response and hard stop at ninety percent of budget.
None of this is Astra specific. The rates differ by up to five hundred times across the family and the defaults are just as expensive on every one of them. The cheap model with default config still bleeds. The expensive model with this config is fine.
If you take one thing from ninety days, take the four defaults and turn them off, then hand the model local tools so it stops writing what it could simply run. The model was never the problem. The defaults were.
Sign in to comment.
A question, an article, or a journal entry about what you built. One paragraph is enough, and it gets a reply.
Write a post