fix(sources/filesystem): order resume comparison by path component#5041
Open
genisis0x wants to merge 1 commit into
Open
fix(sources/filesystem): order resume comparison by path component#5041genisis0x wants to merge 1 commit into
genisis0x wants to merge 1 commit into
Conversation
When resuming a filesystem scan, scanDir compared the directory path
against the stored resume point with a raw string comparison
(`path < resumeAfter`). Raw comparison does not match the depth-first
traversal order produced by os.ReadDir, because the separator '/' (0x2F)
sorts after characters that are valid inside a path component such as
'-' (0x2D) and '.' (0x2E).
So with sibling directories where one name is a prefix of the other,
e.g. `blue-team` and `blue-team-deprecated`, resuming inside `blue-team`
made scanDir evaluate `"/root/blue-team-deprecated" < "/root/blue-team/file"`
as true and skip `blue-team-deprecated` entirely, silently dropping every
file beneath it.
Compare the paths component by component instead, which matches the order
os.ReadDir returns entries in ("blue-team" before "blue-team-deprecated").
Adds an integration test for the prefix-sibling case and a unit test for
the comparison helper.
Closes trufflesecurity#5039
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description:
Resuming a filesystem scan could silently skip an entire sibling directory when one directory name is a prefix of another (e.g.
blue-teamandblue-team-deprecated).scanDirdecided whether a directory was already scanned by comparing its path against the stored resume point with a raw string comparison (path < resumeAfter). Raw string order does not match the depth-first orderos.ReadDirproduces, because the separator/(0x2F) sorts after characters that are valid inside a path component such as-(0x2D) and.(0x2E).So when resuming inside
blue-teamwith resume point/root/blue-team/project-notes.txt, the check evaluated:and skipped
blue-team-deprecatedentirely — every file beneath it (e.g.AWSCredentials.txt) was never enumerated or scanned. Renamingblue-teamtoblue-team2made the problem disappear, matching the report.The fix compares the two paths component by component, which matches
os.ReadDir's ordering (blue-teamsorts beforeblue-team-deprecated). The in-subtreeHasPrefixchecks and the per-entry loop were already correct and are unchanged.Closes #5039
Tests:
TestResumptionWithPrefixSiblingDirectories— integration test reproducing the report (resume insideblue-team, assert the siblingblue-team-deprecated/AWSCredentials.txtis still scanned).TestComparePathsForResume— unit test for the comparison helper, including the prefix-sibling case and the existing before/after/ancestor cases.TestResumption*tests still pass.Checklist:
make test-community)?make lint)?Note
Medium Risk
Changes interrupted-scan resume logic in the filesystem source; a bug here could skip files and miss secrets, but the change is narrow and covered by new regression tests.
Overview
Fixes silent skipped directories when a filesystem scan resumes and sibling folder names share a prefix (e.g.
blue-teamvsblue-team-deprecated).scanDirused to treat a directory as already scanned if its full path was lexicographically before the resume point (path < resumeAfter). That order disagrees withos.ReadDirdepth-first traversal, because/sorts after characters like-inside a name—so…/blue-team-deprecatedcould look “before”…/blue-team/notes.txtand the whole sibling tree was never scanned.The PR adds
comparePathsForResume(component-by-component, then shorter path wins for ancestors) and uses it for the out-of-subtree skip decision. Integration and unit tests cover the prefix-sibling regression and existing before/after/ancestor cases.Reviewed by Cursor Bugbot for commit d39582d. Bugbot is set up for automated code reviews on this repo. Configure here.