Skip to content

Cache AttributeError member-name suggestions#130

Open
jhonabreul wants to merge 1 commit into
QuantConnect:masterfrom
jhonabreul:bug-cache-attribute-error-suggestions
Open

Cache AttributeError member-name suggestions#130
jhonabreul wants to merge 1 commit into
QuantConnect:masterfrom
jhonabreul:bug-cache-attribute-error-suggestions

Conversation

@jhonabreul

Copy link
Copy Markdown
Collaborator

What does this implement/fix? Explain your changes.

Caches the AttributeError "Did you mean ...?" member-name suggestions so they are not recomputed on every attribute miss.

Building the hint (added in #124 / #129) reflects over the managed type's entire member set (GetMembers with FlattenHierarchy), snake_cases every member, and runs a Levenshtein scan. This ran on every miss with no caching — and getattr(obj, name, default) / hasattr trigger it too, because the work happens on the miss path before CPython suppresses the error. A workload that probes the same missing names repeatedly — e.g. a per-bar getattr(self, "_optional", None) on a .NET-derived object — pays the full O(members) reflection + ranking cost on every call.

This adds two caches in ClassBase:

  • _candidateNameCache (Type -> string[]): the deduplicated snake_case member names, computed once per type.
  • _suggestionCache ((Type, name) -> string[]): the ranked suggestion list, memoized per (type, missing-name).

The reflection/ranking moved into ComputeSimilarMemberNames / GetCandidateMemberNames; GetSimilarMemberNames now returns from the memo. Suggestions and error-message text are unchanged — only the repeated computation is removed.

Measured on a real Lean multi-symbol minute backtest (a Python QCAlgorithm subclass doing many per-bar optional-attribute probes), profiled with dotnet-trace: the suggestion path (AttributeErrorHint.GetAttrHook -> GetSimilarMemberNames -> GetMembers / LevenshteinDistance / ToSnakeCase) accounted for ~93% of OnData CPU. With this change the same backtest goes from not finishing (aborted, >15x slower) to ~93s, on par with the last build without the suggestion feature (~90s), with identical results.

Does this close any currently open issues?

No pythonnet issue is open. This fixes the backtest performance regression introduced with the AttributeError suggestion feature (#124 / #129).

Any other comments?

Behavior-preserving: the emitted error messages and suggestions are byte-for-byte identical, so there are no user-visible output changes — only the redundant per-miss work is eliminated.

Checklist

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Ensure you have signed the .NET Foundation CLA
  • Add yourself to AUTHORS
  • Updated the CHANGELOG

Building the "Did you mean ...?" hint for a missing attribute reflected over the
managed type's full member set (GetMembers with FlattenHierarchy), snake_cased
every member, and ran a Levenshtein scan -- on every miss, with no caching.
getattr(obj, name, default) and hasattr trigger it too, since the work happens on
the miss path before CPython suppresses the error. A workload that probes the
same missing names repeatedly (e.g. a per-bar getattr(self, "_optional", None) on
a .NET-derived object) therefore paid the full O(members) reflection + ranking
cost on every access.

Add two caches in ClassBase:
- _candidateNameCache (Type -> string[]): the reflected, deduplicated snake_case
  member names, computed once per type.
- _suggestionCache ((Type, name) -> string[]): the ranked suggestion list,
  memoized so repeated misses of the same name are a dictionary lookup.

Suggestions and error messages are unchanged; only the repeated computation is
removed. On a real Lean multi-symbol minute backtest the suggestion path
dominated ~93% of OnData CPU; with this change the backtest goes from not
finishing (aborted, >15x slower) to ~93s, on par with the last build without the
suggestion feature (~90s), with identical results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants