a live map of what I'm learning, what I'm building, and what I finished.
drag to pan · scroll to zoom · click a node
learn, execute, report. mostly for me.
Building shamirai.ai executing runtime & infra This site. A map of what I am learning rather than a list of what I have finished. in progress Loop engineering reported harness & loops Designing the iteration itself: when to continue, retry, escalate, compact, or stop. Most agent failures are loop failures, not model failures. 2026-08-08queued, not started
Agent execution graphs learning harness & loops Expressing agent control flow as an explicit DAG or state machine instead of a free-running loop. Buys inspectability and resumability, costs flexibility. Agent trajectory eval learning evaluation Scoring the path an agent took rather than just its final answer: redundant calls, recovery from errors, dead ends. Final-answer accuracy hides all of it. Chunking strategies learning retrieval How you split documents before embedding them: size, overlap, and whether to respect structure. The variable that quietly decides retrieval quality. Context compaction learning agent architectures What a long-running agent throws away when the window fills, and how it decides. Summarise, score-and-drop, or offload to storage. Graph RAG learning retrieval Retrieving over a graph of entities and relationships rather than flat chunks, so questions needing two or three hops stop failing. Harness engineering learning harness & loops Designing everything around the model: tool surface, context assembly, permissions, feedback. The scaffolding that decides whether a capable model is actually useful. Durable Objects as agent state learning runtime & infra Cloudflare's single-instance stateful objects as a home for one agent's memory: alive between requests, strongly consistent, one per session. Hybrid search: BM25 + dense learning retrieval Combining keyword search (BM25) with vector search, and the real problem: how to fuse two ranked lists that disagree. Knowledge graphs as memory learning agent architectures A graph the agent writes to as it goes, rather than a static index it only reads. Memory with explicit relationships instead of a pile of embeddings. MCP server authoring learning protocol & tooling Building a Model Context Protocol server so an agent can use your own data and actions as first-class tools rather than pasted context. LLM-as-judge learning evaluation Using a model to grade model output. Cheap and scalable, and quietly vulnerable to position bias and self-preference. Multi-agent orchestration learning agent architectures Splitting work across specialised agents (planner, worker, critic), and whether the quality gain survives the cost of every handoff. Prompt regression suites learning evaluation Test suites for prompts, so a change that fixes one case does not silently break twelve others you were not looking at. ReAct loops learning agent architectures The reason / act / observe cycle underneath most agents. The baseline every fancier architecture gets measured against. Tool descriptions as the API learning protocol & tooling The claim that a tool's prose description, not its type signature, is the real interface the model programs against, and that writing it is design work. Tool-use error recovery learning agent architectures What you hand back when a tool call fails: the raw error, a translated message, or a suggested fix. The choice changes recovery rates a lot. Streaming and backpressure learning runtime & infra Streaming tokens when the consumer is slower than the producer, and what to actually do when the buffer fills. Workers AI at the edge learning runtime & infra Running inference and embeddings inside a Cloudflare Worker: no origin server, no API key, no cold-start infrastructure to manage.Expressing agent control flow as an explicit DAG or state machine instead of a free-running loop. Buys inspectability and resumability, costs flexibility.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Scoring the path an agent took rather than just its final answer: redundant calls, recovery from errors, dead ends. Final-answer accuracy hides all of it.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
This site. A map of what I am learning rather than a list of what I have finished.
Every personal site I’ve seen is a list of finished things. That shape hides the part I actually care about: what I don’t know yet. Can the site itself be the instrument, so that looking at it tells me where my gaps are instead of just advertising what’s already done?
A map instead of a feed. Every topic is a node in a semantic space: dashed outline for things I want to understand, pulsing for things I’m building, solid for things I’ve finished and written up.
Positions come from embeddings. Each entry is embedded with Workers AI, projected from 768 dimensions to 2 with PCA, and laid out by meaning rather than by date. Related work ends up near related work without me arranging anything. Adding an entry is one markdown file; the map rearranges itself.
The whole thing is static. Nothing runs at request time, so the entire site is a folder of HTML on Cloudflare Pages.
How you split documents before embedding them: size, overlap, and whether to respect structure. The variable that quietly decides retrieval quality.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
What a long-running agent throws away when the window fills, and how it decides. Summarise, score-and-drop, or offload to storage.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Retrieving over a graph of entities and relationships rather than flat chunks, so questions needing two or three hops stop failing.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Designing everything around the model: tool surface, context assembly, permissions, feedback. The scaffolding that decides whether a capable model is actually useful.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Cloudflare's single-instance stateful objects as a home for one agent's memory: alive between requests, strongly consistent, one per session.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Combining keyword search (BM25) with vector search, and the real problem: how to fuse two ranked lists that disagree.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
A graph the agent writes to as it goes, rather than a static index it only reads. Memory with explicit relationships instead of a pile of embeddings.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Building a Model Context Protocol server so an agent can use your own data and actions as first-class tools rather than pasted context.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Using a model to grade model output. Cheap and scalable, and quietly vulnerable to position bias and self-preference.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Designing the iteration itself: when to continue, retry, escalate, compact, or stop. Most agent failures are loop failures, not model failures.
The claim I keep repeating is that most agent failures are loop failures, not model failures. I don’t actually know if that’s true, or if it’s just a satisfying thing to say. If it is true, I should be able to take a working agent, change nothing about the model or the prompt, and break it purely by changing the loop around it. Can I?
A rig for breaking an agent on purpose, and the control group to break it against.
The world is a frozen fleet of 400 connected medical devices, one 30-day telemetry window, eight fields per device. It is synthetic, from a seeded generator, because this repo is public and the data has to be mine to publish. That trade has a cost worth naming: real data has one virtue a generator cannot fake, which is that nobody chose the answer. So the generative process is written out in the script, the seed is fixed, and I read the answer off the output afterwards instead of designing it in.
The agent sees the fleet through three deliberately narrow tools: count, rank by one stored field, fetch one device. No tool returns more than ten rows, and no tool will compute a rate for me. Both constraints are load-bearing. A tool that ranked by downtime-per-operating-hour would turn the whole task into two calls, and a higher row cap would make the answer visible in a single query. Because neither exists, the one device that matters is invisible to the obvious search and can only be found by narrowing the range and dividing by hand.
The task has a known right answer and a known wrong one. The wrong one is the interesting half: it is what a run reports if it ranks once and stops, and also what a run reports if it ranks once, pulls every candidate and carefully divides all of them. Both roads lead to the same plausible, well-supported, incorrect device.
Then the loop itself, written by hand rather than with the SDK’s tool runner, because the tool runner is the loop and using it would put the entire subject of the project inside a library I cannot instrument. This first version has no retry, no iteration budget, no compaction, and no error handling: a tool that throws takes the process with it. That is not a first draft on the way to the real one. It is the control. I cannot claim a retry policy helped unless I have watched the thing fail without one.
I could not break it. I ran the task twice, changing nothing but the loop: same model, same prompt, same tools, same question, with parallel tool calls allowed and then forbidden. Both runs got the right answer. The failure modes I had built the task specifically to provoke, wrong candidate set, giving up partway, context blowing out, simply did not happen. My prediction that a naive loop would confidently return the plausible wrong device was written down before the run, and it was wrong.
Forcing serial tool calls made the agent cheaper, not more expensive. This is the part I keep turning over. Serialising took 15 turns instead of 6, exactly as expected, but it used 14 tool calls instead of 19 and less peak context. The trace shows why. When the model can batch, speculation is free, so it fired six queries at once and three turned out to be unnecessary. Forced to see each result before choosing the next, it reasoned instead: it took the leading rate as a benchmark, derived that any device above 200 hours would need more than 179 minutes of downtime to beat it, noticed only one device cleared that bar and that its own rate was lower, and eliminated three fifths of the search space in a single inference. Parallel tool calling is partly a substitute for thinking. Take it away and the model thinks harder per call.
The model out-solved my reference path, twice, differently each time. I had assumed the only route was to split the range into bands and check devices inside each. Run 1 banded, then bounded each band and pruned three of them without looking inside. Run 2 skipped banding and derived one global threshold. Mine was the worst of the three solutions.
Most of the design work was in the tools, not the loop. Whether the task took two hops or fifteen was decided entirely by which fields I made rankable, before the model was ever called. One line, a ten row cap on results, turned out to be the whole difficulty setting: at ten the answer is invisible to any single query, at twenty it falls out of one call and the task collapses. I wrote that line without thinking about it.
Run it on day one. I spent an entire session on the apparatus (data, tools, task, ground truth, loop) before the first execution. The first run then invalidated one of my core measurement assumptions in about ninety seconds: I had been counting hops as a proxy for how much loop there is, and run 1 was 6 turns and 19 tool calls, because the model batches independent work into single turns. Hops and work are different quantities and I would have known that on day one for a dollar.
I picked the wrong variable. Serialising tool calls rearranges when information arrives; it never removes any. That is presumably why the model absorbed it without difficulty. The loop changes worth testing are the ones that destroy information the model cannot reconstruct: dropping tool results, truncating history mid-task, capping iterations before the work is done. I reached for the change that was easiest to implement rather than the one most likely to break something.
I would state the claim more carefully. “Most agent failures are loop failures” is too loose to be tested. What these two runs actually support is something narrower and less quotable: a capable enough model absorbs a merely inefficient loop. Which means if loop failures dominate in practice, it is not because loops are badly arranged, it is because they lose information or stop early. That is a sharper question, and it is the one I should have started with.
Splitting work across specialised agents (planner, worker, critic), and whether the quality gain survives the cost of every handoff.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Test suites for prompts, so a change that fixes one case does not silently break twelve others you were not looking at.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
The reason / act / observe cycle underneath most agents. The baseline every fancier architecture gets measured against.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
The claim that a tool's prose description, not its type signature, is the real interface the model programs against, and that writing it is design work.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
What you hand back when a tool call fails: the raw error, a translated message, or a suggested fix. The choice changes recovery rates a lot.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Streaming tokens when the consumer is slower than the producer, and what to actually do when the buffer fills.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.
Running inference and embeddings inside a Cloudflare Worker: no origin server, no API key, no cold-start infrastructure to manage.
Nothing written yet. It's on the map because I want to understand it. The node exists so I can see the gap.