FE-629: Compile Petrinaut user code through an HIR to buffer-native programs#8955
FE-629: Compile Petrinaut user code through an HIR to buffer-native programs#8955kube wants to merge 3 commits into
Conversation
Introduce a typed, source-spanned, JSON-serializable HIR as the single compiler for dynamics/lambda/kernel user code: - TS->HIR lowering (const destructuring, guard if/early-return, .map comprehensions, first-class distributions) with positioned diagnostics - Schema-checked typing against Color/parameter definitions; analyses for dependency sets, distribution DAGs (nodes/edges/sinks/shared draws), binding usage and constant folding - Buffer-ABI emitters: lambdas read token attributes at statically resolved offsets (tokenValues[slotBases[slot] + attr]); kernels write place-major staging floats with deferred distribution sampling ordered by output index (exact legacy RNG-stream parity); dynamics compile to allocation-free Float64Array loops - Engine hot loops (single-run + Monte Carlo) prefer buffer programs and fall back to object-convention HIR programs per item - HirArtifacts v2 compiled in the LSP worker (sdcpn/compileHirArtifacts request) and threaded through simulation/experiment configs; the engine no longer bundles any compiler (simulation workers: ~3MB -> ~40kB) - HIR semantic lints in the LSP checker (source "hir", codes 99001+); out-of-subset code is an error and Play gates on error severity only - Remove @babel/standalone and compile-user-code.ts; fix H-6519 violations in the supply-chain example; shipped examples are the buffer-ABI coverage gate Design docs: src/hir/README.md (architecture) and src/hir/dsl-sketch.md (planned OCaml-flavoured DSL lowering to the same HIR). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Interactive playground for the HIR pipeline: type dynamics/lambda/kernel TypeScript on the left, and inspect everything it compiles to on the right — the spanned HIR tree, diagnostics with exact ranges, the inferred return type, dependency sets, the distribution DAG (derivation edges, output sinks, shared draws), and both emitted programs (buffer-ABI fast path and object-convention fallback). The model schema (parameters, places, attribute types) is editable JSON and re-checks live. Exposes the compiler via new `@hashintel/petrinaut-core/hir` and `/hir-runtime` entries (typescript declared as an optional peer — only the ./hir entry needs it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
PR SummaryHigh Risk Overview
Also tightens discrete token attribute rules (H-6519): dynamics derivatives only on reals, distributions only on real kernel outputs, typed defaults/AI cheatsheet, and a supply-chain example fix ( Reviewed by Cursor Bugbot for commit 1cecf1b. Bugbot is set up for automated code reviews on this repo. Configure here. |
| case "let": | ||
| // `let` only appears as a function/callback body; those emit blocks. | ||
| // eslint-disable-next-line no-use-before-define -- mutual recursion | ||
| return `(() => ${emitBody(expr, scope)})()`; |
|
|
||
| function instantiate(source: string): unknown { | ||
| // eslint-disable-next-line no-new-func, @typescript-eslint/no-implied-eval, @typescript-eslint/no-unsafe-call | ||
| return new Function("__dist", `"use strict"; return (${source});`)( |
| return new Function( | ||
| "__dist", | ||
| "__params", | ||
| `"use strict"; return (${source});`, |
There was a problem hiding this comment.
Pull request overview
This PR replaces Petrinaut’s Babel + new Function compilation for dynamics/lambdas/kernels with a typed, source-spanned HIR pipeline, enabling static analysis and emitting buffer-ABI programs that operate directly on packed frame buffers. It also introduces discrete token attribute types (integer/boolean), updates UI/editor flows to reflect typed attributes, and wires simulation/Monte-Carlo runs to require precompiled HIR artifacts produced by the LSP worker.
Changes:
- Add Petrinaut-core HIR compiler + runtime instantiation entrypoints, with LSP integration for compilation and diagnostics (including HIR semantic lints).
- Update simulation and Monte-Carlo engines/workers to execute buffer-ABI programs (with object-convention fallback) and to require
hirArtifactsprovided by the UI via the language worker. - Introduce typed token attributes (real/integer/boolean) end-to-end (schema/types, encoding/decoding, scenario/metric typing, UI token type editor + spreadsheet boolean/integer handling) and update docs accordingly.
Reviewed changes
Copilot reviewed 65 out of 66 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Removes Babel standalone from petrinaut-core dependency set; adds optional TS peer metadata. |
| libs/@hashintel/petrinaut/src/ui/views/shared/place-state-visualization.tsx | Uses frame reader token decoding (getPlaceTokens) and TokenRecord typing for visualizer inputs. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/scenarios/view-scenario-drawer.tsx | Updates scenario initial-state typing to TokenAttributeValue. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/scenarios/scenario-mapping.ts | Updates comments/wording for typed token rows. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/panels/SimulateView/scenarios/scenario-form.tsx | Threads spreadsheet typed cell values + element types through scenario form state. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/panels/PropertiesPanel/type-properties/subviews/main.tsx | Adds per-dimension type dropdown (real/integer/boolean) and related styling. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/panels/PropertiesPanel/place-properties/subviews/place-initial-state/subview.tsx | Passes full place into initial-state editor; supports read-only scenario display. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/panels/PropertiesPanel/place-properties/subviews/place-initial-state/initial-state-editor.tsx | Supports typed spreadsheet cells and defaults based on element type; uses decoded frame tokens. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/components/BottomBar/diagnostics-indicator.tsx | Adds warning state (amber) and separates error count from total count. |
| libs/@hashintel/petrinaut/src/ui/views/Editor/components/BottomBar/bottom-bar.tsx | Gates simulation on error-severity diagnostics only. |
| libs/@hashintel/petrinaut/src/ui/lib/compile-visualizer.ts | Expands visualizer token typing to include booleans. |
| libs/@hashintel/petrinaut/src/ui/components/spreadsheet.tsx | Adds typed columns and boolean cell UX; supports integer rounding and defaults per type. |
| libs/@hashintel/petrinaut/src/ui/components/spreadsheet.stories.tsx | Updates story types for SpreadsheetCellValue. |
| libs/@hashintel/petrinaut/src/react/simulation/provider.tsx | Requests HIR artifacts from language worker before starting simulation; passes hirArtifacts to engine. |
| libs/@hashintel/petrinaut/src/react/lsp/provider.tsx | Exposes errorDiagnosticsCount and requestHirArtifacts via context. |
| libs/@hashintel/petrinaut/src/react/lsp/context.ts | Adds requestHirArtifacts and errorDiagnosticsCount to context contract/defaults. |
| libs/@hashintel/petrinaut/src/react/experiments/provider.tsx | Requests HIR artifacts before starting Monte-Carlo experiments; passes hirArtifacts. |
| libs/@hashintel/petrinaut/docs/simulation.md | Documents error-only Play gating (warnings/hints don’t block). |
| libs/@hashintel/petrinaut/docs/petri-net-extensions.md | Documents typed dimensions, discrete semantics, diagnostics severity split, and supported subset. |
| libs/@hashintel/petrinaut-core/vite.config.ts | Adds hir and hir-runtime entries; externalizes typescript as optional peer for compiler entry. |
| libs/@hashintel/petrinaut-core/src/types/sdcpn.ts | Introduces ColorElementType, TokenAttributeValue, TokenRecord; updates scenario typing. |
| libs/@hashintel/petrinaut-core/src/simulation/worker/simulation.worker.ts | Passes hirArtifacts through init message into runtime. |
| libs/@hashintel/petrinaut-core/src/simulation/worker/messages.ts | Adds optional hirArtifacts to simulation worker init payload. |
| libs/@hashintel/petrinaut-core/src/simulation/runtime/simulation.ts | Forwards hirArtifacts into worker initialization. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/worker/monte-carlo.worker.ts | Passes hirArtifacts through Monte-Carlo init. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/worker/messages.ts | Adds optional hirArtifacts to Monte-Carlo worker init payload. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/types.ts | Threads optional hirArtifacts into Monte-Carlo simulator config. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/transition-effect.ts | Adds buffer-ABI fast path for lambdas/kernels; typed token decode/encode; lazy object decoding. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/runtime/experiment.ts | Threads hirArtifacts through experiment creation and worker init. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/run-state.ts | Passes hirArtifacts into simulation created for each run. |
| libs/@hashintel/petrinaut-core/src/simulation/monte-carlo/monte-carlo-simulator.test.ts | Compiles/threads HIR artifacts in tests now that engine no longer compiles user code. |
| libs/@hashintel/petrinaut-core/src/simulation/frames/frame-reader.ts | Decodes typed token attribute values when producing token records. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/types.ts | Adds buffer-program scratch/type plumbing and hirArtifacts to simulation input. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/token-values.ts | New helpers to coerce/encode/decode typed token attributes and records. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/execute-transitions.test.ts | Updates compiled transition test scaffolding to include element metadata + buffer field. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/compute-possible-transition.ts | Adds buffer-ABI fast path; typed token decode/encode; lazy object decoding. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/compute-possible-transition.test.ts | Adds typed token decode/encode test; updates scaffolding for new transition shape. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/compute-next-frame.test.ts | Compiles/threads HIR artifacts in tests now that engine no longer compiles user code. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/build-simulation.test.ts | Updates tests for typed attributes and new “missing compiled code” error behavior; threads artifacts in tests. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/build-simulation-hir.test.ts | New end-to-end equivalence tests for buffer vs object programs and missing/stale artifacts behavior. |
| libs/@hashintel/petrinaut-core/src/simulation/engine/buffer-transition.ts | New shared helpers for buffer-ABI kernel execution, slot-base fill, and staging→add-map conversion. |
| libs/@hashintel/petrinaut-core/src/simulation/authoring/user-code/distribution.ts | Removes injected Distribution runtime source string; documents new HIR runtime injection approach. |
| libs/@hashintel/petrinaut-core/src/simulation/authoring/user-code/compile-user-code.ts | Deletes legacy Babel + new Function compiler for dynamics/lambdas/kernels. |
| libs/@hashintel/petrinaut-core/src/simulation/authoring/user-code/compile-user-code.test.ts | Deletes tests for the removed legacy compiler. |
| libs/@hashintel/petrinaut-core/src/simulation/authoring/scenario/compile-scenario.ts | Coerces typed token row/record values; reports coercion errors as compile errors instead of throwing. |
| libs/@hashintel/petrinaut-core/src/simulation/authoring/scenario/compile-scenario.test.ts | Adds coverage for typed token row coercion behavior. |
| libs/@hashintel/petrinaut-core/src/simulation/authoring/metric/compile-metric.ts | Updates metric token typing to TokenRecord. |
| libs/@hashintel/petrinaut-core/src/simulation/ARCHITECTURE.md | Updates architecture notes to reflect HIR as sole compiler for dynamics/lambdas/kernels. |
| libs/@hashintel/petrinaut-core/src/simulation/api.ts | Threads hirArtifacts into public config; updates initial marking typing and frame-reader token typing. |
| libs/@hashintel/petrinaut-core/src/schemas/scenario-schema.ts | Allows boolean cell values in per-place token rows. |
| libs/@hashintel/petrinaut-core/src/schemas/metric-schema.ts | Updates metric docs to reflect typed token objects. |
| libs/@hashintel/petrinaut-core/src/schemas/entity-schemas.ts | Updates schema docs for discrete attributes and distribution constraints for outputs/dynamics. |
| libs/@hashintel/petrinaut-core/src/lsp/worker/protocol.ts | Adds sdcpn/compileHirArtifacts request type. |
| libs/@hashintel/petrinaut-core/src/lsp/worker/language-server.worker.ts | Implements sdcpn/compileHirArtifacts by calling compileHirArtifacts. |
| libs/@hashintel/petrinaut-core/src/lsp/lib/ts-to-lsp.ts | Preserves TS diagnostic source when present (so HIR lints can be source: "hir"). |
| libs/@hashintel/petrinaut-core/src/lsp/lib/generate-virtual-files.ts | Updates generated TS types for discrete dynamics derivatives and kernel outputs (Distribution only for real). |
| libs/@hashintel/petrinaut-core/src/lsp/lib/checker.ts | Integrates HIR semantic lints into diagnostics pipeline when TS has no errors; adjusts validity logic to error-only. |
| libs/@hashintel/petrinaut-core/src/lsp/lib/check-hir.ts | New bridge from HIR diagnostics to TS-shaped diagnostics with stable numeric codes and source: "hir". |
| libs/@hashintel/petrinaut-core/src/lsp/language-client.ts | Tracks error-only diagnostic count; adds requestHirArtifacts client method and wire protocol call. |
| libs/@hashintel/petrinaut-core/src/index.ts | Exposes HIR-related types (type-only) from main entry. |
| libs/@hashintel/petrinaut-core/src/hir/README.md | New design doc for HIR pipeline, subset, ABI, and integration. |
| libs/@hashintel/petrinaut-core/src/hir/lint.ts | New HIR semantic linting combining lowering/typecheck/analysis. |
| libs/@hashintel/petrinaut-core/src/hir/lint.test.ts | Adds unit tests for HIR linting rules and subset handling. |
| libs/@hashintel/petrinaut-core/src/hir/instantiate.ts | New dependency-free artifact instantiation and distribution runtime injection. |
| libs/@hashintel/petrinaut-core/src/hir/emit-js.test.ts | Adds tests for object-convention HIR emit helpers. |
| libs/@hashintel/petrinaut-core/src/hir/emit-buffer-js.test.ts | Adds tests for buffer-ABI HIR emission and runtime behavior. |
| libs/@hashintel/petrinaut-core/src/hir/dsl-sketch.md | Adds planned DSL sketch that lowers to HIR. |
| libs/@hashintel/petrinaut-core/src/hir/compile.ts | New batch compilation to HirArtifacts for root net + subnets, with buffer/object emission choices. |
| libs/@hashintel/petrinaut-core/src/hir/compile.test.ts | Adds compilation coverage gates for all shipped example models + buffer ABI coverage. |
| libs/@hashintel/petrinaut-core/src/hir/analyze.test.ts | Adds tests for dependency analysis, distribution DAG extraction, and constant folding. |
| libs/@hashintel/petrinaut-core/src/hir.ts | New public compiler entrypoint aggregating HIR APIs. |
| libs/@hashintel/petrinaut-core/src/hir-runtime.ts | New runtime-only entrypoint (no typescript) for worker/engine consumption. |
| libs/@hashintel/petrinaut-core/src/examples/supply-chain-with-disruption.ts | Fixes discrete-attribute violations uncovered by HIR typechecking (and adjusts dynamics accordingly). |
| libs/@hashintel/petrinaut-core/src/default-codes.ts | Updates default code templates to respect discrete vs real attributes (dynamics derivatives, kernel defaults). |
| libs/@hashintel/petrinaut-core/src/ai.ts | Updates AI prompt “code-surface” contract notes to reflect typed attributes and discrete semantics. |
| libs/@hashintel/petrinaut-core/package.json | Adds ./hir and ./hir-runtime exports; removes Babel standalone; declares optional TS peer. |
| .changeset/petrinaut-hir-compiler.md | Changeset for HIR compiler + runtime export additions and behavior changes. |
| .changeset/h-6519-discrete-token-attribute-types.md | Changeset for discrete token attribute types. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| requestHirArtifacts: () => | ||
| Promise.resolve({ | ||
| artifacts: { version: 2, dynamics: {}, lambdas: {}, kernels: {} }, | ||
| failures: [], | ||
| }), |
| let currentRngState = rngState; | ||
| for (const pendingIndex of order) { | ||
| const dist: RuntimeDistribution = pendingDists[pendingIndex]!; | ||
| const [sampled, nextRngState] = sampleDistribution(dist, currentRngState); | ||
| currentRngState = nextRngState; | ||
| kernelStaging[pendingSlots[pendingIndex]!] = sampled; | ||
| } | ||
| // Clear per-call sample caches so the next firing draws fresh values: | ||
| // emitted kernels construct fresh distribution objects per call, so this | ||
| // is defensive only. | ||
| pendingSlots.length = 0; | ||
| pendingDists.length = 0; |
🌟 What is the purpose of this PR?
Introduces an HIR (high-level intermediate representation) as the single compiler for Petrinaut user code (dynamics, lambdas, transition kernels), replacing the Babel +
new Functionpipeline. The HIR is a typed, source-spanned, JSON-serializable expression tree that supports:.mapderivation edges, output sinks, shared draws), dependency sets (which parameters/token attributes a lambda reads), determinism classification, and semantic lints surfaced in the LSP with exact source ranges.tokenValues[slotBases[slot] + attr]) — no per-combination record decode/encode, no allocation in the hot loop.The design doc lives at
libs/@hashintel/petrinaut-core/src/hir/README.md; the planned OCaml-flavoured DSL sketch atsrc/hir/dsl-sketch.md.🔗 Related links
🔍 What does this change?
HIR pipeline (
petrinaut-core/src/hir/, new./hir+./hir-runtimepackage entries):const(incl. destructuring likeconst { a, b } = parametersandconst [a, b] = input.Place), guard-clauseif/early returns, ternaries,Math.*, token/parameter access,.map(...), and first-classDistribution.*nodes(tokenValues, slotBases) => rateand kernel(tokenValues, slotBases, out, distSink) => void(deferred distribution sampling ordered by output float index → exact legacy RNG-stream parity), buffer-native dynamicsFloat64Arrayloop, plus an object-convention fallback for shapes that don't scalarizeEngine (
simulation/engine,simulation/monte-carlo):slotBasesInt32Array, stagingFloat64Array); token records are built lazily only for object-fallback programsbuildSimulationconsumesSimulationInput.hirArtifactsonly — the engine bundles no compiler; missing artifacts fail per item with a pointer to the Diagnostics tab@babel/standaloneandcompile-user-code.tsare deleted; simulation/monte-carlo worker bundles shrink from ~3 MB to ~40 kB eachLSP + UI:
sdcpn/compileHirArtifactsworker request /LanguageClient.requestHirArtifacts; the simulation and experiments providers compile artifacts before starting runssource: "hir", codes 99001+):Math.randomreproducibility warning, transition-never-fires, shared-sample notes, unused bindings; out-of-subset code is an errorDiagnosticsSnapshot.errorCount); the diagnostics indicator shows amber for warnings-only vs red for errorsHIR/Playgroundin the petrinaut Storybook): type user code + edit the model schema, inspect the HIR tree, diagnostics, inferred types, dependencies, distribution DAG, and both emitted programs livePre-Merge Checklist 🚀
🚢 Has this modified a publishable library?
This PR:
📜 Does this require a change to the docs?
The changes in this PR:
petri-net-extensions.mddocuments the supported code subset and the error/warning split;simulation.mdupdated for error-only Play gating. Screenshots showing the diagnostics indicator may want refreshing (new amber warnings-only state).🕸️ Does this require a change to the Turbo Graph?
The changes in this PR:
new Functioncompilation (they never used Babel); visualizer JSX keeps@babel/standalonein the UI package.storybook buildoutput fails to render for all stories in this package (pre-existingreact/jsx-runtimeresolution issue); useyarn dev.🐾 Next steps
materializeEngineFramein the single-run engine (buffer path only needs offsets/counts + token floats)reduce/filtercomprehension nodes)src/hir/dsl-sketch.md)🛡 What tests cover this?
build-simulation-hir.test.ts— buffer vs object programs produce bit-identical frames over 50 stochastic steps (incl. RNG stream through distribution sampling); stale-artifact fallback; missing-artifact errorshir/compile.test.ts— coverage gate: every example model compiles fully, and every dynamics/lambda/kernel reaches the buffer ABI❓ How to test this?
cd libs/@hashintel/petrinaut && yarn dev→ open HIR → Playground: type code, break it (add aforloop), edit the schema (make an attribute"integer"and feed it a Distribution)yarn workspace @hashintel/petrinaut-core test:unit run🤖 Generated with Claude Code