Skip to main content
Back to all essays
System Design6 min read

AI tool search: scope the catalog before discovery

A freight assistant example separates catalog eligibility, search relevance and execution permission. Test missing tools and ambiguous names before optimizing context size.

Proposed tool-discovery design: filter a fictional 200-tool registry into 24 eligible session tools before sending definitions. Search returns three candidates; shipment evidence selects two reads. Every call still passes independent authorization. A missing tool leads to a bounded handoff, not broader access.
Open full-size workflow diagram
Read the workflow steps
  1. The fictional registry has 200 tools. Application policy permits 24 definitions for this actor, task and tenant. Filter before constructing the request; deferred definitions are still sent to the API.
  2. A search for the reason a shipment is held returns three relevant-looking tools. Read the shipment first; its customs-document hold determines the next read. A billing-hold result remains unused.
  3. The executor checks current actor, tenant, resource, action and tool version on every call. Finding or loading a tool never grants permission to run it.
  4. If no eligible tool can answer, preserve the unresolved question and assign the handoff. Evaluate selection and business completion separately from token use.
Cover graphic for AI tool search: scope the catalog before discovery, System Design

A freight assistant with access to many integrations has two distinct questions to answer: which tool could help, and which tool may this operator use for this shipment? A search feature can improve the first decision. The application still owns the second.

Consider a fictional request: “Why is S-204 held? Draft an internal note.” The operator may read this customer’s shipment and relevant operating instructions. The request does not authorize clearing a hold, booking transport or sending a customer message. We will use a constructed tool catalog to show where those limits belong. No tool-search benchmark or carrier integration was run.

Build an eligible catalog before sending the request

Claude’s tool search discovers definitions on demand. Its current guide describes regex and BM25 variants, and a defer_loading flag that keeps a definition out of the initial model context. The full deferred definition still goes in the API request. Deferred loading changes context assembly; it does not hide the catalog from the API. Claude: tool search and deferred definitions

For the example, I would start with the operator’s authenticated identity, tenant membership and task scope. Application policy reduces a registry of 200 tools to 24 eligible definitions before constructing the request. Those counts are illustrative. A tool that changes a customs hold is absent from this session rather than merely deferred or discouraged in a prompt.

Treat names and descriptions as information that also needs an appropriate audience. Do not put credentials, private record examples or another customer’s identifiers into a catalog description. Discovery needs enough detail to distinguish operations, but it does not need the data the tool will retrieve. Fetch that data only after the relevant access checks.

OWASP recommends task-specific tool privileges and authorization enforced by the execution component outside the model’s context. That supports two separate controls here: reduce what the model can select, then verify the actual requested operation at execution. OWASP: least privilege and tool authorization

Write descriptions that resolve the business ambiguity

Illustrative search results for “why is shipment held?”
Candidate toolWhat it answersWhat it does not do
shipment_getCurrent shipment, hold code and source versionChange the hold
customs_requirements_getRequired documents for the stated hold codeConfirm customs release
billing_hold_getWhether this customer has a billing holdExplain every operational delay

All three names look relevant to “held.” The assistant should first read shipment_get for S-204. In this example, the response says hold_code=CUSTOMS_DOC_MISSING and source_version=42. That evidence selects customs_requirements_get as the next read. A high-ranked billing result is not a reason to invent a billing problem.

A useful description states the resource, the operation and the boundaries. For example: “Read the current customs document checklist for a supplied hold code; does not change shipment state or confirm clearance.” Keep argument meanings explicit too. A customer account ID and a shipment ID should not be interchangeable strings just because both endpoints accept text.

Claude’s guide says tool search matches names, descriptions and argument metadata, and that discovered references expand into full definitions. For a small, frequently used tool set, ordinary tool calling may be simpler. Choose the discovery mechanism after checking the catalog and the task, rather than adding search to every agent by default. Claude: discovery behavior and when search is useful

Recheck the call when it reaches the executor

Proposed execution boundary, not a complete API implementation
Proposed call: customs_requirements_get
Actor: operator-17
Tenant: customer-A
Shipment: S-204
Hold code: CUSTOMS_DOC_MISSING

Executor verifies:
  current actor membership and resource access
  allowed operation for this task
  registered tool identity and schema version
  validated arguments and required approval, if any

Only then call the connector. Record its actual result.

The executor should derive tenant context from the authenticated session. A model-supplied tenant field cannot establish membership. If the model changes S-204 to a shipment belonging to another customer, the same tool name must still fail the resource check. Tool visibility is useful defense, but it cannot cover every parameter a model might propose.

Permissions can change after a definition loads. An operator might lose access while a long conversation remains open. Revalidate against current policy at each call and retire any cached discovery entries according to the session’s access rules. Rebuilding a prompt alone would not revoke credentials already available to a connector.

If a tool definition changes, keep the registered identity and schema version visible in the trace. A worker should not silently dispatch yesterday’s arguments to a different operation with the same friendly name. Validate the call at the connector boundary and give the agent a clear failure it can report or recover from within the original task.

Give missing and misleading results a bounded exit

Suppose the checklist tool is unavailable. The assistant can report the verified hold code and create the internal draft with an unresolved checklist question, if that draft operation is allowed. It should not claim that a document is missing by guessing from memory. Record the connector error separately from a legitimate response saying no checklist exists.

A catalog search that finds nothing should not widen the operator’s access. Rephrasing the search within the eligible catalog is reasonable when terminology differs. Searching unrestricted tools, fabricating an endpoint or switching to a broadly privileged shell changes the security model and needs a separate authorized design.

The internal note should preserve what was learned: shipment version 42 reports a customs-document hold, the relevant checklist was retrieved or unavailable, and no hold was cleared. That is a useful outcome even when the assistant cannot finish the investigation. It leaves the next operator a specific question rather than a vague “agent failed” status.

Evaluate completion before reporting context savings

Proposed evaluation cases for the same assistant
CaseWhat to inspect
Customs hold versus billing holdFirst read establishes the cause; irrelevant candidate stays unused
No relevant eligible toolUnresolved question is handed off without access expansion
Cross-customer shipment IDExecutor rejects the resource even if discovery succeeded
Revoked access after discoveryPreviously loaded definition cannot bypass current policy
Changed schema or unavailable connectorClear error and bounded recovery; no invented result
Read task proposes a writeExecutor enforces the task boundary

Measure whether the right evidence was obtained and whether the resulting note is accurate. Record unnecessary reads, unauthorized attempts and unresolved tasks alongside discovery rounds and input tokens. Fewer definitions in context can reduce overhead while extra searches add steps. The end-to-end task trace is needed to decide whether the tradeoff helps this workflow.

The earlier Skills and MCP article explains who owns the procedure and the connection. Tool search adds a selection layer within that design. It should make the right permitted capability easier to find while keeping the original task and execution checks intact. Read: Skills, MCP and workflow ownership

Putting this into practice?