Mirza Ishraq
← All issues

Issue #3 · By · · 10 min read

Haiku 5.5 Saved Me 27%

I spent many rounds rewording the instructions for Tradion’s live analyst on 5-minute charts. Across 8 or more tries, two AI graders scored its comments around 6 out of 10 without knowing which model wrote them. Each rewording just moved the same mistakes around. Then I swapped Sonnet 5.5 for Opus 5.5, and it scored 7.5 on the first try.

Prompt engineering has a ceiling, and I found it with my forehead. When rewording stops raising the score, the model is the limit, not the wording. Try a stronger model or a code change instead of another round of edits.

Inside, Haiku 5.5’s 90% price cut against my real bill, Python 3.15, a Claude dashboard prompt, five repos, one number-checking fix and an app idea.

Talk of the Town

Anthropic released Claude Haiku 5.5 on October 7. Haiku is Claude’s small, fast model, built for high-volume work like summaries, classification and sub-agents, the helper models a bigger model hands small jobs to. It can read up to 1 million tokens at once. A token is a small chunk of text, a word or part of one.

For prompts up to 100,000 tokens, Haiku 5.5 costs $0.10 per million tokens in and $0.50 out. Haiku 4.5 cost $1 and $5, so that’s 90% off per token. Above 100,000 tokens it’s $0.50 and $2.50. Anthropic says it costs around 75% less to run on average, and the gap between 90 and 75 is the whole story.

It was the top Hacker News story of the week, past 1,000 points, and on AWS the same day. People love the price, plus the monthly API credits that came with it, $100 a month on Max 5x and $200 on Max 20x, for your own apps but not Claude Code. One commenter wrote, “This is a very big benefit for me.” Others think the 100,000-token cutoff for the cheap price is far too low.

The loudest pushback was about tokens. In one r/ClaudeAI test, the same voxel pagoda cost $24.96 on Haiku 5.5 and $1.96 on GPT-6 Luna, both at the highest effort setting, because Haiku read about four times as many tokens. The poster called it “another token goblin.”

Anthropic’s own migration guide explains where the discount goes. The same text counts as about 30% more tokens. Thinking, where the model reasons before it answers, is now on by default. And in a chat, the thinking from earlier turns stays in the conversation, so you pay for it again every turn.

I moved every Haiku 4.5 call in Tradion to Haiku 5.5 on launch day. It went pretty smooth, and honestly the answers look and feel the same to me.

Then I re-ran the same 20-question thread on Tradion’s chat. It cost $0.28 on Haiku 4.5 with thinking off and $0.205 on Haiku 5.5 at medium effort. That’s about 27% cheaper. Real money, just not 90%.

One thing did bite. Haiku 5.5 checks that nothing before a thinking block changed, and my chat rewrites saved threads when someone resets its memory. Those requests failed with a 400 error until I turned on a beta setting that drops the stale reasoning. Anthropic’s docs call that setting a stopgap, so keep your threads append-only if you can.

It’s one thread, so I need to keep using it a little longer to actually test what Anthropic said about the cost. Since it’s the newer model, I believe this was the right move.

This Week in AI

Python 3.15

Python 3.15 came out on October 9 as a stable release. UTF-8 is now the default text encoding, explicit lazy imports load a module only when you first use it, a built-in frozendict gives you a dictionary nobody can change, and the experimental JIT compiler got a big upgrade.

The first change is the one to check. open() without an encoding argument now reads files as UTF-8 on every machine, so code that relied on a machine’s own setting can read files differently. Lazy imports can make big apps and command-line tools start faster.

Ship it on new projects and scripts. Django 6.1 only supports Python 3.12 to 3.14 so far, so wait for the Django release that adds 3.15 before you move a Django app.

→ Python 3.15.0 release

EmbeddingGemma 2

Google released EmbeddingGemma 2, an open embedding model with 740 million parameters. An embedding model turns content into a list of numbers, so a search can find things by meaning instead of exact words. This one puts text, images, audio and video in one shared space, under the Apache 2.0 license.

It’s small enough to matter. Google says a quantized text-only version needs as little as about 191MB of memory on a Pixel 11 Pro, so search and RAG, answering from your own documents, can run on the device with no API call.

Try it if you want search that runs on a laptop or phone with no API bill. Apache 2.0 means you can put it in a product.

→ EmbeddingGemma 2 announcement

OpenAI Decisions API

OpenAI’s Decisions API is in public beta. You send text or images plus a list of questions, and it returns typed answers, a probability that something is true, one choice from a list, or a score. OpenAI says it answers about 10 times faster than its Responses API, and it costs $0.10 per million input tokens, with nothing charged for output.

Routing, moderation and tagging usually go through a full chat model plus code that parses its reply. This does both in one call.

Try it for yes or no checks like routing or moderation, where a full chat model is overkill. It’s beta and runs only gpt-6-luna, so expect changes.

→ Decisions API guide

Workflows & Hacks (Steal This!)

Steal #1. A live Claude dashboard for your repo

What it is. A Claude dashboard is a page Claude Code builds and publishes to claude.ai that pulls live data every time you open it. Mine shows my Skyborne repo’s open pull requests, the CI runs on main (the tests that run on every push) and the latest deploys, with a short brief Claude writes on top. It took four prompts and about an hour and 40 minutes.

How to install. You need a paid Claude plan and Claude Code 2.1.209 or later. The hard part is GitHub. Skip the GitHub Integration in your connectors list, because it can’t read pull requests.

  1. On github.com, make a fine-grained personal access token with Repository access set to Public repositories and no extra permissions. That makes it read-only.
  2. On claude.ai, open Settings, then Connectors, then Add custom connector. Name it GitHub and paste the server URL below.
  3. Set Authentication to No sign-in first. Then add an authorization header with Bearer and your token.
  4. Start a new Claude Code session and run /mcp to check that GitHub shows up.
https://api.githubcopilot.com/mcp/x/all/readonly

How to use it. Open Claude Code in an empty folder and paste the prompt with OWNER/REPO swapped for your own repo, like ishraq21/skyborne for mine. My first version went live about 33 minutes later. The first time the page opens, claude.ai asks you to allow GitHub. Then check every number against GitHub. My first version had two bugs, and Claude can’t see the page it wrote, so you’re the QA team now.

The artifact.

Build a dashboard artifact for the GitHub repo OWNER/REPO that pulls live
data through my GitHub connector every time the page loads. Show open pull
requests with their review state, the latest GitHub Actions runs on main
with failures first, and the latest deploy from each deploy workflow with
its status. Use only OWNER/REPO and no other repo. Don't bake in sample or
fallback data. If the connector isn't connected, show a message that names
it. Add a refresh button.

Watch out for. Only connectors on your claude.ai account work in the published page. Local MCP servers in Claude Code can feed Claude while it builds, but the page itself can’t call them. My local Netlify server never worked there, not once.

Builder Radar

  • leviathan turns JSONL, CSV and SQLite records into one ranked full-text index your agent can search, in a single Rust binary. Memory for agents with no vector database to run, and its README claims about 450 tokens per answer.
  • talorys deploys a personal AI agent with chat, memory, tasks and reminders into your own Cloudflare account with one command. Single-user, no telemetry, and your data stays in an account you control.
  • Rgpu keeps your Python on your laptop and runs PyTorch operations on a remote NVIDIA GPU, even from a Mac with no CUDA. For everyone whose Mac is great at everything except CUDA.
  • lithos-metal is a new set of LLM inference kernels and serving for Apple silicon. If you run models locally like in my local LLM guide, it’s one to watch.
  • Durable Actors is an open-source alternative to Cloudflare Durable Objects, small stateful servers for chat, collaboration and agent swarms, with Python and TypeScript SDKs. You can self-host it instead of renting the idea from one vendor.

Fix it before you ship

The AI wrote the number, and the code had it right

A lot of AI-built apps let the model write the numbers users read, in a summary, a report or a dashboard brief. The model can garble a figure your code got right, and users read the sentence, not the chart.

I saw it on a dashboard I built this week. The code worked out a 93.5% pass rate, 43 of 46 finished runs, and the figure card showed it correctly. Claude’s brief above it said 43 of 48 runs, 93.5%. But 43 of 48 is 89.6%.

  1. Compute every figure in code, never in the model.
  2. Hand the model the finished figures and tell it to quote them exactly.
  3. Before launch, check every number in the AI’s text against the data.
Audit this app for numbers written by an AI model. List every place where a
model's text shows a figure to users, such as summaries, reports, briefs and
emails. For each one, report: where | which figures | computed in code or by
the model | fix. Change nothing until I approve. The fix should compute
every figure in code, pass the finished figures to the model, tell it to
quote them exactly and never recalculate, and add a test that checks each
figure in the output against the source data.

App Ideas

Here’s an app idea from my week. Point it at your real API logs, pick the model you want to move to, and it replays your actual requests on both models before you switch. Then it tells you what each real task costs, which requests now fail, and how the answers compare side by side.

It’s for anyone who has read “90% cheaper” on a launch page and then opened their own bill.

How I’d build it with Claude. Tradion already replays recorded trading days through the real code before I change anything, which is how I know my chat thread got 27% cheaper and not 90%. Plain code replays the requests and adds up the cost from each response’s token counts, and Claude only compares the answer pairs and writes the report.

Interview & Sketch prompt.

I want to design a "model switch rehearsal" tool. Before I move my app from
one LLM to another (for example Claude Haiku 4.5 to Haiku 5.5), it replays
my real recorded API requests on both models and reports the cost per real
task, any requests that now fail, and the answers side by side. Before
sketching anything, interview me one question at a time about my own setup:
which providers and models I call, where my requests and responses are
logged today, how long my conversations run, whether I use tools, prompt
caching or structured outputs, and what "the same quality" means for my app.
Keep going until you can design around my real stack, not a generic one.
Then sketch an architecture: how to capture and scrub requests, how to
replay them safely under a spend cap, how to price each run in code from the
usage fields in each response and the providers' published prices, how to
flag new errors, where a model compares answer pairs, and what I review
before switching. Give me the answer as a numbered list of implementation
phases.

Demo/Mockup prompt.

Build a dummy-data prototype called "Rehearsal", a tool that replays an
app's real API requests on a new model before you switch, and shows what
changes.

## Design & Theme
- Light theme, off-white background, near-black text
- One accent color for "worse than before" and nothing else colorful
- Clean sans-serif UI font, monospace for prices, token counts and IDs
- Minimal and calm. No gradients, no glow, no emoji, no bubbly cards
- Desktop-first, works on a laptop screen at minimum

## Pages

### 1. Overview (/)
- Top bar: project name, "Current model" and "Candidate model" pickers,
  "Run rehearsal" button
- Four headline cards: cost per task before and after (with the percent
  change), requests replayed, new errors, answers flagged for review
- A table of task types (chat reply, summary, classification, tool call),
  each with average cost before and after, average tokens in and out, and
  error count
- A "List price vs your price" panel: the candidate model's advertised
  per-token discount next to the measured per-task change

### 2. Run Detail (/runs/:id)
- Run date, request count, spend cap and amount spent
- A bar chart of cost per task type, old model vs new model
- A list of every new error with status code, message and request ID,
  grouped by error type

### 3. Request Diff (/requests/:id)
- The original request (system prompt, messages, settings) collapsed by
  default
- Old answer and new answer side by side, with tokens in, tokens out,
  thinking tokens and cost under each
- A reviewer verdict: same, better, worse, with a notes field

### 4. Settings (/settings)
- Log source (file upload or log folder path), date range, sample size,
  spend cap, models to compare, price list per model (editable)

## Important Notes
- Prototype with dummy data only, no real backend, no real API calls
- All data hardcoded to look like a real run (realistic request IDs,
  prices, token counts, dates)
- Make navigation between the four pages work
- Make "will this switch cost me more or less, and what breaks" obvious
  at a glance

Build it? Hit reply.

Build & Experiment Log

  • Building. Tradion’s Discord desk, six scheduled market posts where code computes every number and Claude only writes the one-line reasons. Tradion also works on phones now.
  • Tried. Opus 5.5 for the live analyst on 5-minute charts. It costs about 2.5 times as much as Sonnet 5.5, $0.48 a live hour against $0.20, and on a day neither model had seen it led by half a point, 7 to 6.5.
  • Broke. My free GitHub Actions minutes, early in the month again. Tradion’s CI runs on my own machine now.
  • Learned. A draft answers the question you ask. Claude helped me write a space navigation plan in May, then took it apart in one reply once I asked what it actually thought.
  • Next week. Tradion’s Discord launch, then more Django tutorials, app ideas and Claude experiments.

From the Blog

P.S. Forward this to a friend who builds. Read it on the web at Haiku 5.5 Saved Me 27%.

Thank you for reading! Follow me on X @myspaceio.

Hit reply and tell me, what are you building?

More issues