Skip to content

fix(agent): self-correct when the model calls an unknown tool#481

Open
walcz-de wants to merge 1 commit into
mudler:mainfrom
walcz-de:fix/unknown-tool-self-correction
Open

fix(agent): self-correct when the model calls an unknown tool#481
walcz-de wants to merge 1 commit into
mudler:mainfrom
walcz-de:fix/unknown-tool-self-correction

Conversation

@walcz-de

Copy link
Copy Markdown

What

When the LLM selects a tool name that matches no available action, the agent now feeds the valid tool list back to the model so it can re-select an existing tool, instead of silently proceeding with a nil action.

Why

In consumeJob's WithToolCallBack, the chosen tool is resolved with allActions.Find(tc.Name). If the model emits a name that was never offered — common with local models, which sometimes hallucinate tool names — Find returns nil and the code falls through:

  • execution proceeds with a nil action → the model gets an empty/unhelpful result, no signal to recover, and typically repeats the bad call or gives up;
  • if an observer is attached, the decision path dereferences chosenAction.Definition() on a nil action → nil-pointer panic.

There was no path that told the model "that tool doesn't exist; here are the ones that do."

How

cogito already has the right primitive: ToolCallDecision.Adjustment re-runs tool selection with feedback (see tools.go). This change uses it:

  • On an unknown tool name, return {Approved: true, Adjustment: "<name> doesn't exist; available tools are […]; re-issue using one of these"} so the model self-corrects.
  • The correction is bounded (max 3 attempts per tool name); past that the call is Skipped, so a model that keeps hallucinating can't drive an unbounded re-selection loop (cogito's goto reprocessCallbacks).
  • Built-in control verbs (stop / send_message / update_state) are dispatched by name and are explicitly excluded.
  • The observer creation path is guarded against a nil action (fixes the latent nil-deref).

The decision logic is factored into a small pure helper, correctUnknownToolCall, with a hermetic unit test (no live LLM required).

Notes

Test

go test ./core/agent/ -run TestCorrectUnknownToolCall -v   # passes, no LLM needed
go build ./core/agent/... && go vet ./core/agent/...        # clean

Assisted-by: Claude:claude-opus-4-8 [Claude Code]

When the LLM selects a tool name that matches no available action
(common with local models, which sometimes hallucinate tool names),
the tool-call callback looked the name up via allActions.Find, got nil
and fell through: execution proceeded with a nil action, yielding an
empty result and -- when an observer is attached -- a nil-pointer
dereference at chosenAction.Definition(). The model received no signal
to recover and would typically repeat the invalid call or give up.

Use cogito's existing ToolCallDecision.Adjustment mechanism to feed the
valid tool list back so the model re-selects an existing tool. The
correction is bounded (max 3 attempts per tool name); past that the
call is skipped to avoid an unbounded re-selection loop. Built-in
control verbs (stop/send_message/update_state) are dispatched by name
and are excluded. The observer creation path is also guarded against a
nil action.

Adds a hermetic unit test for the decision logic (no live LLM required).

Assisted-by: Claude:claude-opus-4-8 [Claude Code]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant