Skip to content

fix: compare review diagrams against target baseline#39

Merged
ivanmilevtues merged 8 commits into
mainfrom
fix/review-target-baseline
Jun 18, 2026
Merged

fix: compare review diagrams against target baseline#39
ivanmilevtues merged 8 commits into
mainfrom
fix/review-target-baseline

Conversation

@ivanmilevtues

@ivanmilevtues ivanmilevtues commented Jun 16, 2026

Copy link
Copy Markdown
Member

Ivan's comment:
There was an issue with the mermaid renders, that is fixed. I stumbled across another issue which was related to forks, we used to check with the real repo instead of our fork (fixed).

Example PR:
ivanmilevtues/optimizerDuck#1

Summary

  • compare review-mode diagrams against .codeboarding/analysis.json from the PR target/base commit, not PR-head history
  • re-copy the committed base analysis after cache restore so cached generated base artifacts cannot replace the versioned baseline used for the diff
  • fix Mermaid label escaping from invalid #amp;/#quot; forms to GitHub-compatible numeric refs (#38;, #34;, etc.)

Root cause

The deployed action searched backwards through the PR head branch for .codeboarding/analysis.json. If the target branch had a synced baseline that was not present in the PR branch history, review mode missed it and regenerated a fresh base analysis. That made the diagram compare fresh-base output against fresh-head output instead of comparing against the versioned target-branch baseline.

For CodeBoarding/MORT PR #2, the workflow log showed exactly that fallback and produced n_changed: 0, even though comparing main:.codeboarding/analysis.json against the generated PR artifact produced changes.

Validation

Local:

python3 -m unittest tests.test_diff_to_mermaid
python3 -m unittest discover -s tests

GitHub checks on this PR:

  • Tests / unittest
  • Tests / lint
  • CodeBoarding review / review

MORT integration test:

Using committed baseline at PR base 7402b58c4d5713a29081f855d01ec934a3603710; analysis metadata source is 9ec8f3906a5b055fb39c9d8fa8600cff3d756e26.

That run later stopped because the hosted/free-tier quota returned 402 Resource exhausted, so it did not complete a final MORT diagram, but it did verify the critical corrected path: the action now reads the target/base branch baseline.

When depth_level input is empty (new default), the action reads
metadata.depth_level from the committed .codeboarding/analysis.json:
- sync mode: from the working-tree checkout
- review mode: from the PR base SHA via git show

Falls back to 2 when no baseline exists or the field is missing.
An explicit depth_level input always overrides auto-detection.
@codeboarding-review

codeboarding-review Bot commented Jun 16, 2026

Copy link
Copy Markdown

Architecture review · 14 components changed

graph LR
    n_Analysis_Orchestrator["Analysis Orchestrator"]
    n_Structural_Diffing_Engine["Structural Diffing Engine"]
    n_Mermaid_Visualization_Engine["Mermaid Visualization Engine"]
    n_UX_Integration_Layer["UX #38; Integration Layer"]
    n_Analysis_Orchestrator -- "triggers health checks and status reporting" --> n_UX_Integration_Layer
    n_Structural_Diffing_Engine -- "provides structural change data for rendering" --> n_Mermaid_Visualization_Engine
    n_Structural_Diffing_Engine -- "supplies change detection flags and diff summar…" --> n_UX_Integration_Layer
    n_Mermaid_Visualization_Engine -- "queries specific file changes for component map…" --> n_Structural_Diffing_Engine
    n_UX_Integration_Layer -- "filters visual noise for final presentation" --> n_Mermaid_Visualization_Engine
    n_Mermaid_Visualization_Engine -- "provides component change counts for summary re…" --> n_UX_Integration_Layer
    classDef added fill:#1f883d,stroke:#0b5d23,color:#ffffff;
    classDef modified fill:#bf8700,stroke:#7d4e00,color:#ffffff;
    classDef deleted fill:#cf222e,stroke:#82071e,color:#ffffff,stroke-dasharray:5 3;
    class n_Structural_Diffing_Engine,n_Mermaid_Visualization_Engine modified;
    linkStyle 1,2,3,4 stroke:#7d4e00,stroke-width:2px;
    linkStyle 5 stroke:#82071e,stroke-width:2px,stroke-dasharray:5 3;
Loading

Colors indicate component changes compared to main: 🟩 Added · 🟨 Modified · 🟥 Removed

Download the PR analysis artifacts from this workflow artifact.

Structural Diffing Engine : 1 file changed
  • scripts/diff_to_mermaid.py
Mermaid Visualization Engine : 1 file changed
  • scripts/diff_to_mermaid.py

⚠️ 1 architecture issue found — open CodeBoarding to explore them.

Explore this PR’s architecture in your browser or VS Code.

codeboarding-action · run 27653739150

@ivanmilevtues ivanmilevtues force-pushed the fix/review-target-baseline branch from 84d24a3 to b3fecba Compare June 17, 2026 16:23
Comment on lines +233 to +236
"&": "#38;",
'"': "#34;",
"<": "#60;",
">": "#62;",

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were causing friction and were the reason why we wound't render properly the diff.

Comment thread action.yml
HEAD_REPO="$PULL_HEAD_REPO"
EMPTY_BASE="false"
FORK_COMPARE="false"
if [ "$HEAD_REPO" != "$BASE_REPO" ]; then

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This for some really odd reason was comparing with the main of the real repo and not the main of the fork.

Hopefully this is resolved now.

This action.yml is getting to be quite big. I would flag it for refactor, slim-down, weight loss program.

@Svilen-Stefanov Svilen-Stefanov left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering, how did you even verify this is working? I suppose you saw that the high level components between the action and the webview are the same?
I'm looking at 14 components changed, then opening the webview, where I see 2 high level component changed and 7 lower lvl components modified.

The webview is still quite off with the actual modifications, I'm wondering if you've seen these and if you have an assumption where these are coming from?

Comment thread scripts/engine_adapter.py Outdated
Comment thread scripts/engine_adapter.py

@ivanmilevtues ivanmilevtues left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am merging this so I can continue with the investigation and will do a new PR for that.

@ivanmilevtues ivanmilevtues merged commit 7339fbf into main Jun 18, 2026
2 checks passed
@ivanmilevtues ivanmilevtues deleted the fix/review-target-baseline branch June 18, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants