The silent drop — three days of PulseAI data the schema said was fine 

PulseAI is the multi-provider LLM orchestration engine I built to route batch workloads across every commercially viable model — Groq, Cerebras, SambaNova, Gemini — whichever has capacity when a job hits the queue. Workflows are JSON graphs. Nodes hand data to each other through a shared context object. Batch jobs run against live customer data.

This is a short story about a production workflow that compiled, passed every validator, shipped on real traffic for three days, and was silently wrong the entire time.

01Chapter I

The bug

A new workflow I had paired with AI to build started returning partial records to its downstream consumer. Not partial in the "missing rows" sense — partial in the "every row has the right shape but half the optional fields are undefined" sense.

No node threw. No validator flagged it. The PulseAI dashboard was green. The downstream consumer's schema check accepted the payload because every required field was present, and the absent ones were optional — "optional and absent" is indistinguishable from "this record didn't have that data."

I found out three days in, from a secondary metric on a dashboard I had built for a different reason. The distribution of one downstream label had flattened. That was the alarm.

“The payload was syntactically perfect and semantically hollow.”

02Chapter II

What the dashboard didn't say

The script node assembling the final record was returning an object with keys named _raw, _debug, and _latency_ms. The AI writing the node had adopted underscore prefixes as a convention for "internal fields you probably don't want to expose" and then spread them into the return value anyway.

PulseAI's engine has a rule about underscore-prefixed keys. When a node returns an object and the engine merges it into the workflow's shared context, any key starting with _ is silently dropped. By design. The underscore is the engine's convention for "this is scratch, do not persist."

So every _raw, _debug, _latency_ms key the AI had put in the return was being stripped at the merge step. Downstream nodes read undefined where they expected a value. The script node was returning what I had asked for. The engine was stripping what I had told it to strip. Neither component had enough information to flag the mismatch.

No test caught it because the schema validator only checked required fields. The dropped fields were optional. The payload was syntactically perfect and semantically hollow.

03Chapter III

The fix, the sweep, and the rules that did the work

The fix was six characters: rename _raw → raw, and so on. One node, three keys.

The sweep was the work. I grepped every workflow definition in production for object keys starting with an underscore. Seven other nodes from the same week had the same pattern — the same AI pair applying the same convention to the same engine. I fixed them in one pass.

Then I added a warning to the engine itself. When a script node returns an object containing _-prefixed keys, PulseAI now logs a deprecation notice with the key names and a link to the principle in the docs. The next AI pair that reaches for that convention gets caught inside sixty seconds instead of three days.

Two of the eight rules from the manifesto did the real work here.

Rule III — correct by inversion. The fix was not "write a better prompt for the script node." It was naming the axis the output was wrong on: your keys are being dropped at the merge step. Once the axis was named, the correction was six characters. The correction is always shorter than the re-specification.

Rule VIII — use memory as leverage. The engine warning is not a bug report. It is a memory asset. Every AI pair that touches a PulseAI script node from now on reads the lesson before it can repeat the bug. The first time I paid for that lesson it was three days of silent drift. The hundredth time is free.

If you read the 8 rules and wanted to know what leading AI on a real production system actually looks like on a Wednesday — this is what it looks like.