diff --git a/README.md b/README.md index 82a24d5..4301954 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,7 @@ Release drift note: current `main` is ahead of `v0.6.0` and contains the five-de - [`docs/reviewer-pack.md`](docs/reviewer-pack.md): demo matrix, artifact contract, and v1 readiness gate - [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note - [`docs/v1-readiness-gate.md`](docs/v1-readiness-gate.md): fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass requirements +- [`docs/v0.6-to-v1-artifact-diff.md`](docs/v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff - [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md): JSON schema contracts for reviewer-facing evidence artifacts - [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md): release artifact diff contract for reviewer-facing outputs - [`docs/vocabulary.md`](docs/vocabulary.md): cross-demo vocabulary for events, hits, signals, bounded correlation, findings, summaries, reports, and audit traces @@ -173,6 +174,7 @@ Cooldown behavior: - [`docs/reviewer-pack.md`](docs/reviewer-pack.md) is the top-level no-guessing reviewer pack and artifact naming contract - [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md) defines the v1.0 five-demo contract freeze gate - [`docs/v1-readiness-gate.md`](docs/v1-readiness-gate.md) defines the fixed-input, fixed-output, schema-validation, artifact-regeneration, and test-pass readiness gate +- [`docs/v0.6-to-v1-artifact-diff.md`](docs/v0.6-to-v1-artifact-diff.md) explains the additive artifact contract from the fourth demo to the fifth - [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md) maps v1 evidence schemas to committed JSON artifacts - [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md) defines the release diff format for reviewer-facing artifact changes - [`docs/reviewer-brief.md`](docs/reviewer-brief.md) gives the short problem, value, evidence, and boundary summary diff --git a/docs/README.md b/docs/README.md index c98f942..bebd256 100644 --- a/docs/README.md +++ b/docs/README.md @@ -9,6 +9,7 @@ This directory separates the current reviewer route from supporting design notes - [`reviewer-brief.md`](reviewer-brief.md): short problem, value, evidence, and boundary summary - [`v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note - [`v1-readiness-gate.md`](v1-readiness-gate.md): fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass requirements +- [`v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): additive artifact contract and compatibility diff from the fourth demo to the fifth - [`evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON schema contracts for reviewer-facing evidence artifacts - [`reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes - [`vocabulary.md`](vocabulary.md): cross-demo vocabulary for evidence workflow terms and bounded correlation diff --git a/docs/reviewer-pack.md b/docs/reviewer-pack.md index c886431..39ebfbb 100644 --- a/docs/reviewer-pack.md +++ b/docs/reviewer-pack.md @@ -34,6 +34,7 @@ The current artifact names are reviewer-facing contracts for the v1 reviewer con - Keep the demo matrix stable unless a demo is intentionally retired. - Prefer additive artifacts over renaming existing reviewer-visible outputs. +- Use [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) for the concrete fourth-to-fifth-demo compatibility baseline. - If an artifact must be renamed, update the README, reviewer path, this reviewer pack, demo README, tests, and committed sample outputs in the same change. - For each release, add the diff described in [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md), using `no-artifact-change` when reviewer-facing artifacts are unchanged. - Do not introduce platform-style names that imply alert routing, case management, real-time ingestion, or autonomous response. @@ -107,6 +108,7 @@ Use the same Python interpreter for install, tests, and demo commands. - [`docs/reviewer-path.md`](reviewer-path.md): demo choice by review question - [`docs/v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note - [`docs/v1-readiness-gate.md`](v1-readiness-gate.md): v1.0 readiness gate for fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass +- [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff - [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON schema contracts for five-demo evidence artifacts - [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes - [`docs/vocabulary.md`](vocabulary.md): cross-demo evidence workflow vocabulary diff --git a/docs/roadmap.md b/docs/roadmap.md index c103419..4077b91 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -19,6 +19,7 @@ Recently added: - [`docs/architecture.md`](architecture.md) describes the local file-based workflow shape. - [`docs/vocabulary.md`](vocabulary.md) defines cross-demo evidence workflow terms. - [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md) defines the release diff contract for reviewer-facing artifact changes. +- [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) records the additive fourth-to-fifth-demo artifact contract. ## v1 Reviewer Contract Stabilization diff --git a/docs/v0.6-to-v1-artifact-diff.md b/docs/v0.6-to-v1-artifact-diff.md new file mode 100644 index 0000000..0d0c806 --- /dev/null +++ b/docs/v0.6-to-v1-artifact-diff.md @@ -0,0 +1,142 @@ +# v0.6.0 to v1 Artifact Contract Diff + +This document explains the reviewer-facing artifact contract change from the +tagged `v0.6.0` baseline to the current v1.0 five-demo contract freeze target. + +`v0.6.0` introduced the fourth demo, +`config-change-investigation-demo`. The v1 target adds the fifth and final demo, +`cloud-iam-change-investigation-demo`, then closes demo expansion. This is an +additive contract change, not a migration from the fourth demo to the fifth. + +## Compatibility Summary + +- Overall compatibility: `additive-compatible`. +- No fourth-demo artifact path was removed or renamed. +- The committed fourth-demo artifacts are unchanged from the `v0.6.0` tag. +- The fifth demo adds four stable reviewer-visible artifact paths under its own + demo directory. +- v1 adds JSON Schema validation, artifact regeneration, and reviewer-doc checks + around the five-demo matrix. +- Consumers that inspect only the `v0.6.0` fourth-demo paths do not need to + change. +- Consumers that inspect both demos must dispatch by full artifact path and + schema, not by basename. The two `investigation_summary.json` files have + intentionally different shapes. + +## Baseline and Target + +| Contract surface | `v0.6.0` fourth demo | v1 fifth demo | Contract effect | +| --- | --- | --- | --- | +| Source evidence | Three synthetic JSONL files for config changes, policy denials, and follow-on events | One synthetic CloudTrail-like JSONL file | Adds a source-specific cloud event skeleton; does not change the config-change inputs. | +| Normalized artifact | `change_events_normalized.json` | `normalized_cloudtrail_events.json` | Adds cloud identity, request, response, Region, source IP, and event-time fields under a new path. | +| Deterministic investigation artifact | `investigation_hits.json` | `investigation_signals.json` | Adds rule-local cloud correlation outputs; does not redefine config-change hits. | +| Summary artifact | Array of per-investigation summaries | One run-level object with counts, time-model metadata, and boundaries | Same basename, different demo-local contract and separate schemas. | +| Human-readable artifact | `investigation_report.md` | `investigation_report.md` | Adds a cloud-specific report under a new demo path; both reports remain review aids rather than canonical machine contracts. | + +## Fourth-Demo Artifacts + +These `v0.6.0` artifacts remain stable in the v1 target: + +| Artifact | Added fields | Removed fields | Semantic changes | Compatibility notes | +| --- | --- | --- | --- | --- | +| `demos/config-change-investigation-demo/artifacts/change_events_normalized.json` | none | none | none | `no-artifact-change` | +| `demos/config-change-investigation-demo/artifacts/investigation_hits.json` | none | none | none | `no-artifact-change` | +| `demos/config-change-investigation-demo/artifacts/investigation_summary.json` | none | none | none | `no-artifact-change`; v1 adds schema validation without changing the committed artifact. | +| `demos/config-change-investigation-demo/artifacts/investigation_report.md` | none | none | none | `no-artifact-change` | + +The fourth-demo summary is now locked by +`schemas/investigation_summary.schema.json`. It remains an array of reduced +investigation records with: + +- `investigation_id` +- `severity` +- `target_system` +- `triggering_change_id` +- `summary` +- `evidence_counts` +- `bounded_correlation_reason` + +## Fifth-Demo Artifacts + +All fifth-demo artifact paths are new relative to `v0.6.0`: + +| Artifact | Added fields or sections | Removed fields | Semantic changes | Compatibility notes | +| --- | --- | --- | --- | --- | +| `demos/cloud-iam-change-investigation-demo/artifacts/normalized_cloudtrail_events.json` | `eventID`, `event_time`, `observed_time`, `eventTime`, `actor`, `identityType`, `eventSource`, `eventName`, `awsRegion`, `sourceIPAddress`, `userAgent`, `errorCode`, `requestParameters`, `responseElements`, `userIdentity`, `outcome` | none | New CloudTrail-like normalization contract. `eventTime` drives ordering; optional `observedTime` is preserved as `observed_time`. | `non-contract-artifact`; stable reviewer-visible path, but not currently covered by a JSON Schema. | +| `demos/cloud-iam-change-investigation-demo/artifacts/investigation_signals.json` | `signal_id`, rule identity, severity, `signal_time`, actor, primary/evidence event IDs, source IPs, embedded evidence, ATT&CK mappings, bounded-correlation reason, and review scope | none | Introduces `signal` for bounded multi-event cloud correlation. A signal is reviewer evidence, not a production alert or incident verdict. | `additive-compatible`; validated by `schemas/cloud_iam_findings.schema.json`. | +| `demos/cloud-iam-change-investigation-demo/artifacts/investigation_summary.json` | `schema_version`, `source_type`, event/signal counts, per-rule counts, ATT&CK mapping count, time-model metadata, and explicit boundaries | none | Adds a run-level summary object. It is not shape-compatible with the fourth demo's per-investigation summary array. | `additive-compatible`; validated by `schemas/cloud_iam_summary.schema.json`. Dispatch by full path or schema. | +| `demos/cloud-iam-change-investigation-demo/artifacts/investigation_report.md` | `Run Summary`, `Signals`, per-signal evidence and ATT&CK context, and `Boundaries` sections | none | Adds a cloud-specific reviewer narrative tied to signal and event IDs. | `non-contract-artifact`; stable reviewer-visible Markdown, not a JSON Schema contract. | + +## Semantic Differences + +The fifth demo reuses the evidence workflow vocabulary without forcing the two +investigation demos into one generic object model. + +| Dimension | Fourth demo: config change | Fifth demo: Cloud IAM change | +| --- | --- | --- | +| Review unit | `hit`: one risky config change plus attached evidence | `signal`: one bounded rule result over one or more CloudTrail-like events | +| Correlation scope | Same `target_system`, after the trigger, inside a fixed window, and from configured evidence families | Rule-local time window, actor or entity key, and event family set | +| Evidence identity | Nested triggering change, policy denials, and follow-on event records | `primary_event_id`, `evidence_event_ids`, and embedded normalized events | +| Time contract | Source `timestamp` values and a configured correlation window | `eventTime` becomes `event_time`; optional `observedTime` is retained but does not drive ordering | +| ATT&CK context | Not part of the fourth-demo artifact contract | Four to five configured cloud-oriented mappings, attached for reviewer orientation only | +| Summary root | JSON array of per-investigation summaries | JSON object describing the complete deterministic run | +| Boundary visibility | Bounded-correlation reason per investigation | Bounded-correlation reason per signal plus run-level scope boundaries | + +`hit` and `signal` are therefore related workflow stages, not interchangeable +field names. See [`docs/vocabulary.md`](vocabulary.md) for the cross-demo term +contract. + +## v1 Contract Additions + +The artifact matrix is not the only change after `v0.6.0`. The v1 freeze adds +reviewer-verifiable controls around both demos: + +1. **Stable paths:** all eight investigation-demo artifacts above are listed in + [`docs/reviewer-pack.md`](reviewer-pack.md#stable-reviewer-visible-artifacts). +2. **Schema validation:** the fourth-demo summary and the fifth-demo signals and + summary validate against separate JSON Schema Draft 2020-12 contracts. +3. **Regeneration:** committed artifacts must match fresh deterministic output + through `python scripts/regenerate_artifacts.py --check`. +4. **Regression checks:** tests keep the demo matrix, artifact links, schema + matrix, terminology, and release-diff requirements aligned. +5. **Release discipline:** future reviewer-facing changes use + [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md), including an + explicit `no-artifact-change` statement when applicable. + +## Consumer Guidance + +- Keep existing `v0.6.0` config-change readers unchanged. +- Add fifth-demo readers only for the new cloud artifact paths. +- Do not select a parser using `investigation_summary.json` alone. Use the full + path and the matching schema; the cloud summary also exposes + `schema_version` as `cloud-iam-change-investigation-demo/v1`. +- Treat open `requestParameters` objects as preserved source-specific evidence. + Contracted wrapper objects reject unknown properties. +- Treat ATT&CK mappings as reviewer context, not as attribution or a verdict. + +## Verification + +From the repository root: + +```bash +git diff --name-status v0.6.0..HEAD -- demos/config-change-investigation-demo/artifacts +python scripts/regenerate_artifacts.py --check +python -m pytest tests/test_evidence_pipeline_schemas.py tests/test_reviewer_docs.py tests/test_markdown_links.py +``` + +The first command should produce no fourth-demo artifact changes. The remaining +commands verify regenerated output, schema conformance, reviewer-contract +language, and local Markdown links. + +## Boundaries + +This diff covers synthetic, local, file-based reviewer artifacts only: + +- Synthetic CloudTrail-like events only. +- No live AWS account. +- No real account ID. +- No production detection claim. +- No real-time ingestion, SIEM, dashboard, alert routing, or case-management + service. +- No autonomous response. +- No final incident verdict. diff --git a/docs/v1-contract-freeze.md b/docs/v1-contract-freeze.md index 1e35bc5..7321f25 100644 --- a/docs/v1-contract-freeze.md +++ b/docs/v1-contract-freeze.md @@ -19,6 +19,10 @@ this document, [`docs/reviewer-pack.md`](reviewer-pack.md), and [`docs/reviewer-path.md`](reviewer-path.md) instead of treating `v0.6.0` release notes as the complete current capability set. +Use [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) for the +concrete artifact and compatibility change from the fourth demo in `v0.6.0` to +the fifth demo in the v1 freeze target. + ## Freeze Scope The v1.0 freeze scope is the existing five-demo matrix: diff --git a/docs/v1-readiness-gate.md b/docs/v1-readiness-gate.md index 0380a23..653bc24 100644 --- a/docs/v1-readiness-gate.md +++ b/docs/v1-readiness-gate.md @@ -35,6 +35,7 @@ Before v1.0: - keep stable artifact paths listed in [`docs/reviewer-pack.md`](reviewer-pack.md#stable-reviewer-visible-artifacts) - keep output semantics aligned with [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md) +- verify the `v0.6.0` fourth-demo compatibility baseline in [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) - document any intentional artifact shape change in [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md) - use `no-artifact-change` when committed reviewer artifacts are unchanged for a release diff --git a/tests/test_reviewer_docs.py b/tests/test_reviewer_docs.py index 20e92e8..0f327ff 100644 --- a/tests/test_reviewer_docs.py +++ b/tests/test_reviewer_docs.py @@ -156,6 +156,7 @@ def test_docs_index_separates_current_route_from_history() -> None: "reviewer-brief.md", "v1-contract-freeze.md", "v1-readiness-gate.md", + "v0.6-to-v1-artifact-diff.md", "evidence-pipeline-contract.md", "reviewer-artifact-diff.md", "vocabulary.md", @@ -193,6 +194,7 @@ def test_top_level_reviewer_pack_covers_matrix_and_artifact_contract() -> None: assert "[`docs/reviewer-path.md`](reviewer-path.md)" in reviewer_pack assert "[`docs/v1-contract-freeze.md`](v1-contract-freeze.md)" in reviewer_pack assert "[`docs/v1-readiness-gate.md`](v1-readiness-gate.md)" in reviewer_pack + assert "[`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md)" in reviewer_pack assert "[`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md)" in reviewer_pack assert "[`docs/vocabulary.md`](vocabulary.md)" in reviewer_pack assert "[`docs/architecture.md`](architecture.md)" in reviewer_pack @@ -393,6 +395,45 @@ def test_v1_readiness_gate_defines_required_release_conditions() -> None: assert "v1-readiness-gate.md" in text +def test_v06_to_v1_artifact_diff_documents_additive_fifth_demo_contract() -> None: + artifact_diff = _read_repo_file("docs/v0.6-to-v1-artifact-diff.md") + docs_index = _read_repo_file("docs/README.md") + reviewer_pack = _read_repo_file("docs/reviewer-pack.md") + readme = _read_repo_file("README.md") + freeze_doc = _read_repo_file("docs/v1-contract-freeze.md") + readiness_gate = _read_repo_file("docs/v1-readiness-gate.md") + roadmap = _read_repo_file("docs/roadmap.md") + + assert "# v0.6.0 to v1 Artifact Contract Diff" in artifact_diff + assert "additive-compatible" in artifact_diff + assert "No fourth-demo artifact path was removed or renamed." in artifact_diff + assert "The committed fourth-demo artifacts are unchanged" in artifact_diff + assert "## Fourth-Demo Artifacts" in artifact_diff + assert "## Fifth-Demo Artifacts" in artifact_diff + assert "## Semantic Differences" in artifact_diff + assert "## v1 Contract Additions" in artifact_diff + assert "## Consumer Guidance" in artifact_diff + assert "## Verification" in artifact_diff + assert "`investigation_hits.json`" in artifact_diff + assert "`investigation_signals.json`" in artifact_diff + assert "`schemas/investigation_summary.schema.json`" in artifact_diff + assert "`schemas/cloud_iam_findings.schema.json`" in artifact_diff + assert "`schemas/cloud_iam_summary.schema.json`" in artifact_diff + assert "Same basename, different demo-local contract" in artifact_diff + assert "No live AWS account" in artifact_diff + assert "final incident verdict" in artifact_diff + + for text in [ + docs_index, + reviewer_pack, + readme, + freeze_doc, + readiness_gate, + roadmap, + ]: + assert "v0.6-to-v1-artifact-diff.md" in text + + def test_bounded_correlation_boundaries_are_documented() -> None: architecture = _read_repo_file("docs/architecture.md") reviewer_pack = _read_repo_file("docs/reviewer-pack.md")