The layers of an agent research stack, from acquisition to approval

Developers building a research agent often start by asking which search API to use. It is a reasonable question, but it concerns one layer of several, and picking components before you know the layers usually ends in glue code that nobody can audit. A research and monitoring stack has to acquire material, find the right passages in it, turn passages into structured values, repeat questions on a schedule, keep the evidence behind every claim and stop for a person before anything consequential happens. This page maps those layers, what to look for in each, and what changes when the agent plans and runs research on its own.

The stack at a glance

The orchestration loop (the agent itself, its planner and its tool calls) runs across all of these. Tools are increasingly exposed through MCP or as command-line skills; the trade-offs are covered in MCP, skills or CLI tools.

Acquisition and retrieval

Acquisition

There are four common ways to bring web material in, and most stacks use more than one. A web search API answers a query with ranked results and snippets; you still fetch and read the pages. A crawler walks known sites on a schedule. A scraping or extraction API turns a known page into clean text or fields. A headless browser handles pages that need JavaScript or interaction. Licensed feeds and your own documents cover what the open web does not.

Whatever you choose, record the address and retrieval time at the moment material enters the system, and respect robots rules and site terms. Provenance cannot be added later. For choosing among scraping options, see the scraping API guide.

Retrieval

Once material is inside, retrieval finds the passages that matter. Full-text search is precise for names, codes and exact phrases; semantic search helps when wording varies. Most production systems combine the two and rerank. Two properties matter more than any leaderboard: retrieval must be scoped, so that one project's documents never answer another project's question, and it must distinguish “nothing found” from “the index was unavailable”. An agent that cannot tell those apart will confidently report absence when the system was simply down.

Extraction and monitoring

Extraction turns passages into structured values: a plan limit, a registration number, a filing date. Define the schema first, require the extractor to return the passage that supports each value, and accept unknown as a first-class answer. A plausible guess in a cell is harder to catch than an empty one. The structured extraction guide covers schemas and validation at scale.

Monitoring is extraction or research repeated on a schedule, plus a comparison. The pieces are a baseline snapshot, normalization that removes noise, a written definition of a material change and one record per run that a person can review. A monitor should notify; it should not act. If a detected change needs to trigger work, route it to a reviewer first.

The evidence store is the layer people skip

Most prototypes keep only the final answer and perhaps a list of URLs. That is not enough to defend a claim a month later, when the page has changed. Keep evidence as its own record:

{
  "claim_id":   "c-0192",
  "claim":      "Vendor lists SSO on the Team plan",
  "source":     "https://vendor.example/pricing",
  "retrieved":  "YYYY-MM-DDTHH:MM:SSZ",
  "snapshot":   "sha256 of the normalized page text",
  "quote":      "Single sign-on (Team and above)",
  "method":     "extracted",
  "status":     "supported"
}

A few rules make the store trustworthy. Snapshots and quotations are immutable; corrections create a new record. A claim without a supporting quotation is stored as an open question, not dropped. And a citation is a pointer, not proof: someone still has to check that the quoted passage means what the claim says. A link the model produced without ever retrieving it is not evidence at all.

Approvals, budgets and the trail

Research agents are mostly read-only until they are not. The moment an agent sends an email, exports a dataset, schedules a recurring job or edits its own instructions, it has stepped into actions that deserve a person's decision. Build that layer in from the start:

• Classify actions, and make the consequential classes pause with the exact request shown to the approver.
• Scope each approval to one attempt, and re-check it when the action actually executes.
• Bound every run by wall time, tool calls and spend, and stop visibly at the limit instead of degrading silently.
• Write requests, decisions, tool runs and outcomes to one trail that a person can read end to end.

The note on what an agent audit trail should record goes through it field by field.

What an autonomous research agent adds

An autonomous research agent decides its own next step: which query to run, which page to read, when to stop. That adds three components on top of the stack. A planner turns a question into ordered steps. A loop controller decides when the evidence is sufficient or the budget is spent. And a source policy limits where the agent may look.

Treat the plan as an artifact, not a hidden intermediate. Write it down as the question, the steps and an allowlist of sources, have a person approve it before it runs, and do not let execution widen it. If the question changes halfway through, revise the plan rather than letting the agent drift. The common ways such loops fail are described in research agents that hold up. Whether to build this loop yourself or buy it as a deep research API is a separate decision; build or buy deep research walks through it.

Where GoCux fits

GoCux is not a search API, a crawler or a model, so it does not replace your acquisition layer. It covers the governed layers above it. A work order holds a goal, the scope the work may touch, acceptance criteria and bounds such as wall time and tool calls, and it moves from queued to running to awaiting decision to done, with each step written to its record as it happens. Research plans carry a source allowlist and need approval before they run. Knowledge search answers from one project's documents with citations; Extraction ties each value to the passage behind it; Pulse turns recurring questions into occurrences someone reviews. Consequential actions pause for a person, spending draws on visible credits with a daily ceiling, and the trail can be read in the workspace. MCP tools run in your own compatible client, and having a server configured does not mean every task may use it.

Choosing components: a checklist

• Each acquisition tool returns, or lets you record, the source address and retrieval time.
• Retrieval is scoped per project and reports “unavailable” separately from “nothing found”.
• Extraction returns a supporting passage per value and allows unknown.
• Monitors have baselines, written materiality rules and one reviewable record per run.
• Evidence is stored as immutable claim records with quotations and snapshots.
• Consequential actions pause with the exact request, and approvals are re-checked at execution.
• Every run has bounds on time, tool calls and spend.
• Autonomous research runs from an approved plan with a source allowlist.