jesganaud.devaudio engineering field notes
> Notes.log/.sh/.env

Hooks start things — they don't do them

Two OpenClaw failures, one ownership mistake: a skill and a hook both started the same agent, then a blocking execSync froze the gateway.

logged 2026-08-06tags system-ops · openclaw

Setup, short: cron ingest jobs fill a shared inbox. When a job finishes, a hook starts a separate wiki-writer agent (research-kb) by shelling out from the OpenClaw gateway. That writer run is multi-minute. That duration is load-bearing for both bugs below.

Research wiki 1 — Agents as jobs already names the rules. This is what they cost before they were rules.

I never named a single owner for starting the writer. Then I fixed one path and skipped the other.

The same start twice

ingest job done
    ├─ skill  →  wiki-writer   ← remove
    └─ hook   →  wiki-writer   ← sole starter

On two ingest paths the skill and the hook both started the writer. The skill called the agent and waited; the hook fired on the same “job done” message. Found first on paper ingest, then again on mail triage — still there because the first fix never checked the sibling. Visible when the writer showed up three times in one morning in the session list (two starters on one path, and the other path still double-firing).

Every run double-billed the wiki pass. One day logged 6.5M tokens / 158 API requests; the duplicate was part of that bill. Fix: remove the skill’s call. The hook is the sole starter.

Not subtle. A fix on one path should have meant auditing every other way that agent can start. I treated it as a one-skill bug. The duplicate kept billing until the next path bit me.

The hook that stopped the gateway

First — a job misbehaved. The paper-ingest hook used execSync, so the cron timeout (600s) had to cover ingest and the whole wiki pass. Past budget → timeout → OpenClaw retries (~17 min, up to ) → expensive chain all day. Fix: async exec; cron timeout 900s. Runs finished in tens of seconds.

Then — the gateway was down. Same execSync still on the mail-triage hook. I had patched one watcher and not walked the sibling. execSync freezes the entire gateway event loop for the full writer run — not just the cron job.

Diagnosed live (samples reconstructed; verbatim logs not archived):

# webchat / dashboard
error: websocket timeout after 30000ms

# journalctl --user -u openclaw-gateway.service -f
# …then nothing for 5+ minutes
# no periodic "long-running-session" lines (~every 2.5 min when healthy)

# ps -o pid,etime,pcpu,cmd -p "$(pgrep -f openclaw-gateway | head -1)"
# PID  ELAPSED  %CPU  CMD
# …   …        40+   … openclaw-gateway …

Alive and burning CPU. Could not log. Not a crash — a frozen event loop. Every triage run since that hook existed had been doing this.

Cron --timeout-seconds does not cover agents spawned from hooks. Fix: async exec on the still-blocking mail-triage hook, and { timeout: 900000, killSignal: "SIGTERM" } on both (15 min).

The code shape

Broken:

import { execSync } from "child_process";

execSync('openclaw agent --agent research-kb -m "…"', { stdio: "inherit" });

Fixed:

import { exec } from "child_process";

exec(
  'openclaw agent --agent research-kb -m "…"',
  { timeout: 900000, killSignal: "SIGTERM" },
  (err) => {
    if (err) console.error("research-kb trigger failed or timed out:", err.message);
  }
);

What I check now

One starter per downstream action. If a hook starts agent B when A finishes, A’s skill must not also call B. Draw the ownership line once and enforce it in both places.

Hooks launch; they don’t run the work. Prefer async exec (fire-and-forget) over execSync. A sync shell from a hook freezes the whole gateway event loop, not just that cron job — webchat, WebSocket, logging included. OpenClaw’s hooks docs say keep handlers fast and fire-and-forget heavy work; they frame the risk as blocking command processing. They do not spell out that a sync shell blocks the whole event loop, not just that command.

Timeout the child in the hook. A cron timeout on the ingest job does not bound the writer the hook starts. Put { timeout, killSignal } on the exec so a stuck model call dies without taking the gateway with it.

A fix is a sibling audit. When you find double-trigger or execSync on one path, walk every other skill/hook that starts the same agent the same day. One-skill patches leave the bug alive next door.

Watch the session list and gateway health, not only the cron job. “Agent ran 3× this morning” and “gateway stopped logging while ps still shows CPU” are the signals; a green ingest log can hide both.

Hooks start things. They do not do things. Each downstream action has exactly one starter.

Pointers