Long-running agents beyond prompt engineering

Y'all relying too much on prompting your way through agent design. Call it context engineering or god damn loop engineering, you still ask the LLM to make sure it only generates factual information and that it reviews information itself.

The more you use an LLM to evaluate itself or perform analysis, the more points of hallucination and drift you introduce.

When you use text generators to automate the stuff you don’t want to do, you still need to treat it as any other piece of software, and design its operating logic.

Before talking about long running agents, remind yourself of the following:

• It’s fine to use short-running agents. Use the simplest tool for the job.
• Differentiate between models and agents. A model takes a text input and generates a text output, but it does not do anything else. The agent is the execution part. Agents rely on models, for example a model hitting its token limit mid-JSON-object is a model-level event (truncated output) with an agent-level consequence (malformed tool call, broken state write, corrupted ledger entry).

This series is about the harness. An LLM call can be used as a tool (e.g., summarize these 40 tool results into 200 words) but it should be part of a deterministic function call the harness decides to make, on its own schedule, with its own validation of the output. It is not the agent deciding, mid-task "oh man, I should really compress my context now."

Part 1: Context and Memory

Every prompt you send to an LLM resends the full conversation history. The chat-like experience is a UI. A user illusion™. This means that in longer conversations, you can max out the context window and the model starts truncating it, and/or you experience context rot and drift where the model gets off-point.

The fact you always send the full conversation makes context management easier. It means you can massage the context inside the window to get rid of knots behind the shoulderblade and keep it as semantically packed as possible. Even if you get a one million dollar- sorry, token window, you will still experience rot and drift.

It also means that you can pick up the context from one agent and dump it into another and carry on the session. You can therefore get rid of the thinking tokens and tool call overhead to only keep what is semantically relevant.

Context has its own lifecycle. You start with the system prompt, tool definitions, user prompt, conversation, reasoning, tool calls, etc. So when thinking about managing context for these long-running sessions, we should think about what to do throughout the lifecycle.

Create (and understand) context

Context is built naturally as you use the LLM. What you need to be intentional about is visibility of what goes in the window. Gumloop has this cool Context Usage Meter that gives you real-time visibility into how much of the LLMs model’s context window is being used during a conversation. It shows token breakdown with categories like:

• System
• AI Instructions
• Abilities
• Tools
• Skills
• Subagents
• Conversation

In long-running agents, you will see that conversation portion increases while the system prompt tool definitions remain stable.

Compress context

As context accrues, you can trim semantically irrelevant tokens and summarize larger chunks into semantically equivalent smaller chunks.

Google’s ADK Context Compaction reduces the size of context by summarizing older parts of the agent workflow event history. It uses a sliding window approach for collecting and summarizing agent workflow event data within a session. It summarizes data from older events once it reaches a threshold of a specific number of workflow events, or invocations, with the current session.

You can’t get away with constantly summarizing previous context. At some point, the harness should perform a full context reset. This means it will tear down the session and rebuild the next request from durable artifacts, explored in the next session.

Store context

LLMs are stateless, so if your context is ephemeral, you will lose the whole agent session. You can, however, write context in persistent storage. Bonus points for architecting these as immutable ledgers, where agents can write, and read, but not modify or delete.

These storage services must be provisioned by you, the developer, where you can define permissions deterministically. Asking the LLM “pls never update ledger pls” is not software engineering.

What goes in the storage is up to you. Dump the whole window, summarize it, write H1s only, etc.

In Google’s EAP, memory generation bundles several of these decisions together.

• Extraction pulls only the most meaningful information out of the source data to persist as memories, rather than dumping everything in.
• Consolidation merges newly extracted information with what already exists, letting memories evolve as new information is ingested.
• Generation runs asynchronously in the background, so the agent doesn't have to wait for it to complete.
• Event ingestion streams and manages conversation events continuously, automatically triggering generation based on batching rules you configure.
• And extraction is customizable. You tell Memory Bank what counts as meaningful by giving it specific topics and few-shot examples.

Recall context

With stored context, you can use the ledger as a reconstruction mechanism. On a fresh session, an agent can read the durable plan, the progress notes, and the append-only record of what has already happened to reconstruct "where am I" without replaying the full conversation history.

As Cloudflare put it, you can fire 20% of your employees to replace them with AI and use the Agent’s plan as context. If the agent has a structured plan, the plan itself provides sufficient context: "I am on step 3 of 7, the step was 'await the sanctions-check result', and the result just arrived."

Once again in Google's EAP, managed storage and retrieval brings its own set of features.

• Consolidation and retrieval are isolated per identity, so one entity's memories don't bleed into another's.
• Storage is persistent and accessible from multiple environments: Agent Runtime, your local environment, or other deployment options. Retrieval can use similarity search scoped to a specific identity, pulling only what's relevant rather than everything.
• A time-to-live can be set so stale information expires automatically, with the TTL applied to inserted or generated memories.
• Revisions are maintained automatically, letting you inspect how a memory transforms as new information is ingested.
• IAM conditions restrict which principals can read or write a given scope's memories.

Identity as a context-management concern

You can retrieve all the memories scoped to a particular user identity. A memory's scope is defined when the memory is generated or created and is immutable.

That makes identity-scoped memory durable across separate jobs and sessions for a given user or entity. In Google's Agent Memory Bank, memories are extracted and consolidated via an LLM from session events, scoped to a user or agent identity, retrieved via similarity search, and managed with TTL-based expiry and revision history.

In Cloudflare, Identity is a durable, addressable property of the agent. Durable Object identity persists across hibernation/restart without re-establishment; this is the anchor that lets task state and identity memory be found again after a crash.

This durability quality requires additional mechanism for ensuring continuity across instances and through failure points, which we address in the following part.

Part 2: Durable Execution

As part of your context management, you now have some persistent storage. Cool, you can use it to pull the context back into an agent if you lose the session.

“Hey Claude, retrieve this and pick up fro-”

No!

We look at deterministic ways of resuming agents in case of failure.

A long-running agent doesn’t run for long. It runs when needed, keeping track of tasks, context, and previous context. Expect the agent to wait quite a lot. But as we discussed in the first part, LLMs always get the whole conversation request with every prompt, so the whole concept of durability comes to “how do I structure this data effectively such that task-to-task the LLM behaves consistently”.

What persists, and what doesn’t need to

Referring back to Cloudflare’s piece, their long-running agent pattern ensures the following persists across calls:

• Agent state, the persistent data the agent requires to continue a session
• All SQLite tables created, including abstractions built on SQLite
• Scheduled tasks, which are stored in SQLite, trigger alarms to wake the agent
• Connection states for each WebSocket client

The following don’t need to survive:

• In-memory variables
• Running timers
• Open HTTP calls
• Callbacks and promise chains

An agent has an identity and persistent state. It does not need an active compute instance. It can be configured to wake on an event and go back to sleep upon completion. Wake sources are the agent's entire interface to the outside world; you don't need an always-on loop required to keep an agent "running" over weeks.

To wake an agent up, you can use a webhook callback, where the agent kicks off external work, registers its own callback URL, and hibernates, waking only when the callback arrives. You can also define polling with backoff for services that don't support callbacks. The agent schedules a poll, reschedules it with an increasing delay, and caps the interval.

Lastly, you can define wide automation workflows with, independently-retryable multi-step pipeline, and hand it to a dedicated workflow engine rather than managing the step sequencing inside the agent.

Sub-agents get this same durability on their own terms. Each child has its own state, schedules, durable fibers, and lifecycle, and stores its own data colocated under the parent.

The property that matters for durability is that the parent doesn't have to stay active while the child works. It can start the work, hibernate, and be woken when the child's schedule or recovery check fires. A crash doesn't take down the whole family at once; each identity is recoverable on its own.

Token and rate-limit monitoring as a predictive signal

With per-session and per-tenant token consumption monitoring, you can define continuation and retry policies when hitting rate-limits or errors. If cumulative usage trajectory projects a rate-limit hit before the next checkpoint, the harness can proactively checkpoint, throttle, or switch model/provider.

Recovery mechanics

Recovery at the coarsest level can be a session that reads the ledger on startup. A simple "where am I" check can resume the task even without finer policies. You can then persist a row for the duration of a piece of work, stash intermediate state at defined points, and recover from the last stash on restart.

Idempotency matters here for durable acceptance, particularly for webhook-driven agents where callers retry delivery and duplicate side effects must be avoided. The general pattern underneath all of this is event-sourcing or journal-based recovery:

We described an immutable task ledger consisting of the agent’s plan: what should happen, in what order, etc. You also need a separate append-only execution log containing what actually happened, containing every tool call, every model response, every state transition. It can be replayed deterministically to reconstruct the state, regardless of which process or container resumes the work.

This is the underlying idea in Restate's journal feature and DBOS's workflow/step annotations.

Restate persists every step before proceeding and replays deterministically to reconstruct pre-crash state, while DBOS checkpoints into Postgres.

These tools also offer compensation and rollback mechanisms for agents that need to undo partial work when failures occur. When agents perform multiple actions and something goes wrong, you need to systematically undo the changes to maintain consistency.

Durable execution vendor landscape

You can look at different implementation ideas by evaluating the vendors such as DBOS, Restate, Inngest. Some general examples include:

• Suspend-and-resume on external events — first-class primitives for pausing a workflow until a signal arrives (a webhook, an approval, a human decision) and resuming it later, potentially across redeploys, without holding compute open while waiting.
• Flow control and concurrency limits — durable queues with per-tenant or per-workflow concurrency caps and rate limits, so retries and fan-out don't overwhelm downstream systems.
• Durability expressed in application code vs. a separate orchestration layer: some make durability transparent to ordinary control flow (a library or annotations in your language), others offer it as a managed platform primitive with its own execution model.

You can also design your own durable execution logic in n8n by implementing its deterministic workflow-based features to define retries, writing data in persistent storage, deterministic triggers such as schedules and webhooks.

Identity as a durability property

In Cloudflare's model the agent's name is the routing key, and that identity persists across hibernation, restart, and redeploy. Credentials are scoped per agent or sub-agent identity. The payoff for durability is attribution: because identity is durable, every entry in the execution log ties back to a persistent agent identity.

At the platform level, Google's Agent Identity and Registry are the productized version of this, tracking which identity, at which version, is running which task.

Part 3: Task Progression & Evaluation

You’ll notice your agent slowly slipping into mistakes and hallucinations. To determine whether the agent is still on task without asking it, most AI engineers found the perfect solution: ask another LLM.

Huh…

LLM-as-judge is the easiest fix and the least reliable one. It's the same model class making the same kind of error, one level removed. This article is about minimizing how much progress validation depends on any LLM's judgment at all, and where an LLM is still useful, constraining it to narrow, checkable roles.

The checklist as the unit of progress

In a task ledger or checklist, each entry needs a completion criterion defined before an agent executes against it. Writing down the done condition before the agent starts is the single highest-leverage move, precisely because it stops the agent from redefining "done" mid-run.

The companion rule is to work one entry at a time. Working a single entry per pass keeps the agent from trying to do everything at once and leaving things half-done, and it makes validation tractable, since each pass has exactly one claimed state transition to check.

Deterministic validation gates

The alternative to "ask the model if it's done" is to run a check against the execution log or live system state that returns a boolean, independent of any model call.

A taxonomy of gate types, from cheapest/most reliable to more involved:

• Status/response codes: did the API call return 200, not 4xx/5xx.
• Schema validation: does the response parse as valid JSON/XML and match an expected shape (required fields present, correct types).
• Cross-field consistency checks: does the username in the response payload match the identity of the requesting user; does a returned ID match the one requested.
• State-diff checks: did the thing the step claimed to create/update/delete actually appear/change/disappear in the target system (re-query after the action).
• Test execution: unit/integration tests run against code changes.

Other mechanism for validating activities include:

State machines for valid progress - A Finite-state-machine approach to semantic parsing defines states as tool-call/step types and transitions as the allowable sequences between them; the observed action sequence is parsed through the FSM at runtime. States outside the machine are violations; unexpected transitions are anomalies. With a small fixed set of legal states, each transition can be validated against the execution log before it's committed to the ledger.

Sandboxing and behavioral baselines - you can also run the agent in a controlled environment first (the sandbox) and observe what it actually does: which tools it calls, what data volumes it moves, what destinations it reaches, what system calls it makes. You can then:

• Graduate specific actions to production once observed-safe, ideally as deterministic allowlist entries rather than re-invoking an agent's judgment;
• Use the profile as the reference baseline for least-privilege permissions and deviation alerting.

Non-generative checks for agent behavior - Some useful checks don't generate text at all. Encoder-only classifiers (BERT-family models such as DeBERTa, RoBERTa, or ModernBERT) can be fine-tuned on labeled aligned/misaligned or benign/malicious examples and output a scalar against a threshold, giving you a verdict without a generative model in the loop.

Anomaly detection on tool-call patterns - monitoring recursive loops (same tool called repeatedly with minor argument variations), token count spikes, out-of-order executions and kills the run before they accrue.

Acceptable LLM-as-judge usage

Intent evaluation is only possible if intent was explicitly defined before execution

‘Explicit’ means an enumerated tool allowlist, a defined sequence of steps or states the task must cycle through, defined data sources, defined rules-based mechanisms for spawning sub-agents, and defined API endpoints and methods.

Once that exists, evaluation collapses into a set of Yes/No questions asked against the execution log: did this step execute, were the fields filled in correctly, did the call return the expected code, was the retrieved data validated against its schema, etc.

Where an LLM is used for evaluation at all, it should be constrained to a narrow, checkable judgment. For example "does this trace match taxonomy category X." That's acceptable precisely because the model is acting as a fuzzy compiler mapping observed behavior onto a deterministic category, not inventing "good" on the fly.

Implementation

Everything described in this article will span across multiple products and require ongoing engineering. Do not treat this as a step-by-step blueprint or framework for how to construct reliable long-running agents, but rather an exploration of the deterministic components which are overlooked by all self-titled loop engineering experts.

I will continue exploring these topics and how they apply for n8n and in the wide market. I welcome improvements, feedback and corrections, so please reach out to me on LinkedIn.