Skip to content

compiler: Faster compilation time#2949

Open
FabioLuporini wants to merge 45 commits into
mainfrom
faster-python-rebased-final
Open

compiler: Faster compilation time#2949
FabioLuporini wants to merge 45 commits into
mainfrom
faster-python-rebased-final

Conversation

@FabioLuporini

Copy link
Copy Markdown
Contributor

the key ingredients are:

  • A lot of "avoided, useless reconstructions" -- across all layers (equations, Clusters, IET)
  • More caching (which needed the above), again at all levels
  • More memoization
  • New loop fusion heuristics to avoid useless and expensive analysis

Bonus: a lot of, IMHO, good refactoring. And several new tests.

Compilation time improvements vary between 1.2x and 5x depending on the complexity of the Operator

@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.67161% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.02%. Comparing base (52dddd9) to head (1f261eb).

Files with missing lines Patch % Lines
devito/passes/clusters/fusion.py 88.82% 16 Missing and 4 partials ⚠️
devito/ir/support/guards.py 27.27% 8 Missing ⚠️
devito/passes/iet/engine.py 81.57% 4 Missing and 3 partials ⚠️
tests/test_gpu_common.py 56.25% 7 Missing ⚠️
devito/symbolics/manipulation.py 84.21% 3 Missing and 3 partials ⚠️
devito/ir/clusters/cluster.py 94.31% 5 Missing ⚠️
devito/tools/memoization.py 93.58% 2 Missing and 3 partials ⚠️
devito/ir/iet/visitors.py 92.85% 4 Missing ⚠️
devito/tools/utils.py 71.42% 3 Missing and 1 partial ⚠️
devito/ir/support/basic.py 96.73% 1 Missing and 2 partials ⚠️
... and 6 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2949      +/-   ##
==========================================
+ Coverage   82.85%   83.02%   +0.16%     
==========================================
  Files         249      250       +1     
  Lines       52279    52919     +640     
  Branches     4503     4568      +65     
==========================================
+ Hits        43318    43937     +619     
- Misses       8188     8189       +1     
- Partials      773      793      +20     
Flag Coverage Δ
pytest-gpu-aomp-amdgpuX 68.85% <87.12%> (+0.25%) ⬆️
pytest-gpu-gcc- 78.41% <89.98%> (+0.19%) ⬆️
pytest-gpu-icx- 78.33% <89.98%> (+0.19%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@mloubout mloubout left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I like some of those intricate caching methods that could use better implementation rather than a new caching method every place it needs something a bit different.

Comment thread devito/ir/cgen/printer.py Outdated
Comment thread devito/ir/clusters/cluster.py
Comment thread devito/ir/clusters/cluster.py
Comment thread devito/ir/clusters/cluster.py
Comment thread devito/ir/iet/nodes.py
Comment thread devito/ir/iet/nodes.py
Comment thread devito/ir/iet/visitors.py Outdated
Comment thread devito/symbolics/manipulation.py Outdated

def pow_to_mul(expr):
if q_leaf(expr) or isinstance(expr, Basic):
if type(expr) in (tuple, list):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should check if it's Iterable to handle generators and such

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking if Iterable introduces other issues, since str/dict/... are also Iterable

I agree that using type(expr) is ugly though, I'll switch it to isinstance

Comment thread devito/symbolics/manipulation.py Outdated
Comment thread tests/test_iet.py Outdated
body = iet.body._rebuild(body=iet.body.body + (DummyExpr(x, x),))
return iet._rebuild(body=body), {}

monkeypatch.setattr(iet_engine, '_update_args',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude is driving me crazy with these monkeypatch 😅 it's always what it tries first for no reason. Like it had 80 of those or something in horizon

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah codex likes that too

@FabioLuporini FabioLuporini force-pushed the faster-python-rebased-final branch 3 times, most recently from 683a23c to 8fc9ca5 Compare June 17, 2026 12:39

@mloubout mloubout left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more comments 😬

Comment thread devito/ir/iet/nodes.py
Comment thread devito/ir/iet/nodes.py Outdated
Comment thread devito/ir/iet/visitors.py Outdated
Comment thread devito/ir/iet/visitors.py Outdated
Comment thread devito/ir/support/basic.py Outdated
Comment thread devito/passes/clusters/fusion.py Outdated
else:
return [ClusterGroup(processed, prefix)]

class Key(tuple):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't need to be local

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, will change this

return exprs
return replaced
else:
return replaced[0]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if replaced[0] == exprs?

@FabioLuporini FabioLuporini force-pushed the faster-python-rebased-final branch from 91b2d83 to 1f261eb Compare June 19, 2026 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants