fix(agent): self-correct when the model calls an unknown tool#481
Open
walcz-de wants to merge 1 commit into
Open
fix(agent): self-correct when the model calls an unknown tool#481walcz-de wants to merge 1 commit into
walcz-de wants to merge 1 commit into
Conversation
When the LLM selects a tool name that matches no available action (common with local models, which sometimes hallucinate tool names), the tool-call callback looked the name up via allActions.Find, got nil and fell through: execution proceeded with a nil action, yielding an empty result and -- when an observer is attached -- a nil-pointer dereference at chosenAction.Definition(). The model received no signal to recover and would typically repeat the invalid call or give up. Use cogito's existing ToolCallDecision.Adjustment mechanism to feed the valid tool list back so the model re-selects an existing tool. The correction is bounded (max 3 attempts per tool name); past that the call is skipped to avoid an unbounded re-selection loop. Built-in control verbs (stop/send_message/update_state) are dispatched by name and are excluded. The observer creation path is also guarded against a nil action. Adds a hermetic unit test for the decision logic (no live LLM required). Assisted-by: Claude:claude-opus-4-8 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
When the LLM selects a tool name that matches no available action, the agent now feeds the valid tool list back to the model so it can re-select an existing tool, instead of silently proceeding with a
nilaction.Why
In
consumeJob'sWithToolCallBack, the chosen tool is resolved withallActions.Find(tc.Name). If the model emits a name that was never offered — common with local models, which sometimes hallucinate tool names —Findreturnsniland the code falls through:nilaction → the model gets an empty/unhelpful result, no signal to recover, and typically repeats the bad call or gives up;chosenAction.Definition()on anilaction → nil-pointer panic.There was no path that told the model "that tool doesn't exist; here are the ones that do."
How
cogito already has the right primitive:
ToolCallDecision.Adjustmentre-runs tool selection with feedback (seetools.go). This change uses it:{Approved: true, Adjustment: "<name> doesn't exist; available tools are […]; re-issue using one of these"}so the model self-corrects.Skipped, so a model that keeps hallucinating can't drive an unbounded re-selection loop (cogito'sgoto reprocessCallbacks).stop/send_message/update_state) are dispatched by name and are explicitly excluded.nilaction (fixes the latent nil-deref).The decision logic is factored into a small pure helper,
correctUnknownToolCall, with a hermetic unit test (no live LLM required).Notes
Approved: falseis deliberately not used for this case — it raisesErrToolCallCallbackInterruptedand aborts the whole run;Adjustmentis the mechanism designed for "let the model try again."Test
Assisted-by: Claude:claude-opus-4-8 [Claude Code]