Technical notes from experiments in language geometry, GPU inference, and agentic AI systems.
Work in progress. All numbers are from actual runs. Code linked where available.
Posts
-
Nothing was delivered: three days of verified motion, one morning of actual speed
I spent most of this week watching an agent fleet work. It reviewed honestly, it rejected fakes, it cited its sources, and after three days the scoreboard read: prefill still slow, correctness still open, nothing you could call a deliverable. The agents weren’t lying — every claim had a receipt. They were just producing verified motion instead of results, and the difference only shows when you force the question “what can I use that I couldn’t use yesterday?”
-
Harness bug or model ceiling? Run the same model through a second harness and find out
Last post covered six real bugs I found and fixed in GKH, my agent harness for SWE-bench-style tasks — a hung index build, an unbounded subprocess, silent zero-token accounting, a skill-synthesis path that had never once actually fired, a hallucinated
cdpath the harness itself handed the model, and a correction message the model was echoing back instead of acting on. After fixing all six, GKH still scored 0/2 on two SWE-bench Verified tasks (sympy__sympy-13031,sympy__sympy-12419). The obvious next question: is that 0/2 a seventh bug I haven’t found yet, or is it the actual ceiling of the model on these two tasks? -
Twelve ways to make a graph write language. All twelve failed, cleanly.
I’ve been growing a graph substrate with no trained weights on the core routing mechanism — no attention, no matmul, no gradient descent on the walk itself. Edges come from demonstration traces, computation is a walk over the graph. Part two of this post (coming next) is about where that substrate turned out to be a genuinely capable arithmetic engine. This one is about the twelve times I tried to make the same substrate generate open-ended language, and every single one failed, for a reason I can point at and a number attached to it.
-
The same graph that couldn't write a sentence solves its own arithmetic exactly
Last post was the negative arc: twelve mechanisms, twelve failures, trying to get a weight-free graph substrate to generate open-ended language. Same substrate, same pre-registration discipline, same independent checker — pointed at arithmetic and symbolic computation instead. This is where it stopped being a curiosity and started being an actual result.
-
GKH: a narrow-waist agent kernel, and the six bugs hiding inside it
GKH (Grounded Kernel Harness) is my agent harness for SWE-bench-style tasks: instead of giving a model a general-purpose shell and hoping, it gets a small, fixed set of verbs —
k.read,k.patch,k.search,k.exec,k.context,k.investigate,k.recall— and every call returns a structured, budgeted response instead of raw text. The bet is that a narrow, well-instrumented interface produces more diagnosable failures than a wide one. This post is about what happened when I actually went looking for those failures instead of assuming the interface was fine. -
The fix that should have worked made things worse
After the reward-dedup fix from a couple days ago, one known problem was still sitting there: the same edge in the graph can get pulled into service by more than one context, and a single weight on that edge can’t represent what it means in each context separately. Push the weight up for one context and you’ve silently pushed it up for all the others too, whether that’s right for them or not.
-
Three milestones said the reward channel was broken. It never was.
For three straight milestones, training a walker on Memoria’s graph made it worse, not better. Every arm that let the contrastive reward channel touch the graph came out behind a walker that never learned anything at all. The obvious read is “the reward signal is broken, throw it out.” I didn’t buy that, mostly because the channel itself was simple enough that I couldn’t see how it would be wrong on its own.
-
What if you cached the model's hidden states instead of running it again?
This started as an experiment. I didn’t expect it to work this well.
-
The Stack I Built While Learning to Use AI as a Tool
I use AI coding assistants every day. Multiple sessions, multiple agents, sometimes parallel subagents working different parts of the same problem. After a few months of this, two things became obvious.
-
Transformer X-Ray: Attention Commitment Depth Across 6 Architectures
Cross-architecture attention analysis using llama.cpp tensor callbacks and JS-divergence.
-
X-raying a Transformer Forward Pass
What does attention actually do, token by token, layer by layer? Not the textbook answer — the actual numbers, on a real prompt, with a real model.
-
Transformer X-Ray, Part II: The BOS Bottleneck
The first x-ray post looked at where the prediction position sends attention mass. That was useful, but it flattened the forward pass into one aggregate graph.
-
rocmforge: GeoGraph Execution Engine and Branch Selection with a 0.5B Model
This post covers two things built in rocmforge over the past week: a graph-based CPU execution engine with temporal rollback, and a first working result using a local 0.5B model to select between branches.
-
Multi-Layer Graphs, Ricci Curvature, and a Hypothesis About How Computation Should Route
This post is about an idea, not a result. The experiment described here has not been run. The hypothesis may be wrong. I’m writing it down because the reasoning is worth making explicit before touching code.
-
Geometric-Only Attention: Linear Scaling from Sparse Neighborhoods
The previous posts documented a ceiling: every geometric attention variant converged to ~50.5 val perplexity on TinyStories, while a plain trigram MLP reached 32. Static geometry was the bottleneck.
-
Geometry as Substrate: What the Failing Results Are Telling Us
The previous post documented a series of negative results: PMI+SVD geometric positions don’t beat a trigram baseline, attention over geometry doesn’t beat a trigram baseline, RoPE doesn’t help, curvature weighting consistently hurts. Every experiment lost to two token IDs fed into a flat MLP.
-
rocmforge: Building an AMD GPU Inference Engine from Scratch
This is a prototype. It produces real throughput numbers on real hardware, it has a real GPU safety protocol because a real GPU page fault happened, and it implements a real stack of inference optimizations. It is also not finished, not stable, and not safe to run without caution. The CLAUDE.md for the project says: “Before ANY new GPU code: Acquire cross-process GPU lock → Run staged preflight → Use timeout-wrapped subprocess.” That rule exists because ignoring it caused a desktop freeze.
-
Training a Geometric Language Model in Pure Rust: First Results
The geometric decoder post described how a corpus-native graph can guide token decoding through Rodrigues rotation and curvature weighting. This post covers what happens when you connect that graph to a training loop and actually try to learn next-token prediction from it.
-
Envoy: The Coordination Server AI Coding Agents Were Missing
I run multiple AI coding agents in parallel. Claude Code sessions, Hermes agents, subagents spawning subagents. After a while I noticed something: none of them know the others exist. They overwrite each other’s files, repeat discoveries, and have no memory of what happened yesterday. There is no infrastructure for this. So I built one.
-
Envoy v0.2.0: Observability, Lock-Free Paths, and Bug Fixes
Three weeks after the initial release, envoy v0.2.0 is out. This isn’t a feature dump – it’s the result of running the server continuously and fixing the things that actually hurt. Three bugs from the original article got fixed, Prometheus metrics landed, and a performance improvement from another project turned out to transfer cleanly.
-
Atheneum: Persistent Memory for AI Coding Agents
Every AI coding session starts from zero. The assistant that helped you trace a bug yesterday has no memory of it today. You explain the same context again, re-answer the same questions, and watch it rediscover the same facts. The tools I’ve built over the last six months — magellan, llmgrep, mirage-analyzer — solve the code structure problem. They make the codebase queryable. But they don’t solve the session continuity problem. An agent still can’t carry decisions, discoveries, or hard-won debugging context from one session into the next.
-
SQLiteGraph: Why I Built One Embedded Library Instead of Four Services
I needed a graph database for a code intelligence project. The typical recommendation would have been Neo4j, or at minimum a graph library backed by something like PostgreSQL. But I was building a CLI tool that runs as a single process on a developer’s laptop. Pulling in a graph database server, a vector search service, a KV cache, and a pub/sub bus — plus connection management and process lifecycle for each — is the wrong architecture for that constraint.
-
Splice: Span-Safe Refactoring That Knows Your Codebase
Splice edits code using byte-accurate span replacements backed by graph algorithms. It replaces function bodies, renames symbols across files, deletes definitions with all their references, and generates machine-checkable refactoring proofs – all with AST validation and automatic rollback on failure.
-
Mirage: Building a Control-Flow Analysis Engine Without a Compiler Backend
Mirage analyzes control-flow graphs from Magellan databases. It enumerates execution paths, computes dominator trees, detects loops, finds dead code, identifies cycles, scores risk, and suggests refactorings – all without being a compiler pass or a language server. It reads what Magellan indexed and answers questions about program structure.
-
Magellan: Six Months of Building a Code Intelligence Indexer
I needed to understand codebases faster than
grepallows and more accurately than tossing files at a model. The answer wasn’t better prompts — it was a proper index. Not an IDE index (those are ephemeral), and not a search engine (those don’t understand structure). Something that records what symbols exist, where they’re defined, who calls whom, and how control flows — then makes all of that queryable from a CLI. -
llmgrep: The Query Layer That Grew With the Stack
In January 2026 I needed a way to search a Magellan database from the command line. Not an IDE plugin, not a language server – just a CLI tool that takes a query and returns structured JSON. Four and a half months later, that tool has 33 published versions, 9 search modes, 12 subcommands, and has evolved from a simple SQL wrapper into the primary query interface for the entire code intelligence stack.
-
Graph-Native Reasoning Experiments: What Worked, What Didn't, and Why
Over three months I ran a series of experiments testing whether graph-based reasoning – not transformers, not neural networks, but pure graph traversal with edge-weight reinforcement – can solve problems that normally require deep learning. The repo is at github.com/oldnordic/ai, 62K lines of Rust, 98 standalone experiment binaries, every result logged with the failures alongside the wins.
-
Ollivier-Ricci Curvature in a Corpus-Native Graph — No Training Required
Kuo et al. (Yale 2024) measured Ollivier-Ricci curvature in trained transformer embeddings (Llama 2, Gemma 2, DeepSeek) and found substantial negative mean curvature — the geometry of the embedding space is hyperbolic, not flat. Their interpretation: the structure emerges from training.
-
A Geometric Decoder for Language Graphs
This post is the decoder half of a larger experiment: can a language model decode by moving through a graph instead of comparing query/key vectors?
subscribe via RSS