Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Release drift note: current `main` is ahead of `v0.6.0` and contains the five-de
- [`docs/reviewer-pack.md`](docs/reviewer-pack.md): demo matrix, artifact contract, and v1 readiness gate
- [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note
- [`docs/v1-readiness-gate.md`](docs/v1-readiness-gate.md): fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass requirements
- [`docs/v0.6-to-v1-artifact-diff.md`](docs/v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff
- [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md): JSON schema contracts for reviewer-facing evidence artifacts
- [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md): release artifact diff contract for reviewer-facing outputs
- [`docs/vocabulary.md`](docs/vocabulary.md): cross-demo vocabulary for events, hits, signals, bounded correlation, findings, summaries, reports, and audit traces
Expand Down Expand Up @@ -173,6 +174,7 @@ Cooldown behavior:
- [`docs/reviewer-pack.md`](docs/reviewer-pack.md) is the top-level no-guessing reviewer pack and artifact naming contract
- [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md) defines the v1.0 five-demo contract freeze gate
- [`docs/v1-readiness-gate.md`](docs/v1-readiness-gate.md) defines the fixed-input, fixed-output, schema-validation, artifact-regeneration, and test-pass readiness gate
- [`docs/v0.6-to-v1-artifact-diff.md`](docs/v0.6-to-v1-artifact-diff.md) explains the additive artifact contract from the fourth demo to the fifth
- [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md) maps v1 evidence schemas to committed JSON artifacts
- [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md) defines the release diff format for reviewer-facing artifact changes
- [`docs/reviewer-brief.md`](docs/reviewer-brief.md) gives the short problem, value, evidence, and boundary summary
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ This directory separates the current reviewer route from supporting design notes
- [`reviewer-brief.md`](reviewer-brief.md): short problem, value, evidence, and boundary summary
- [`v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note
- [`v1-readiness-gate.md`](v1-readiness-gate.md): fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass requirements
- [`v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): additive artifact contract and compatibility diff from the fourth demo to the fifth
- [`evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON schema contracts for reviewer-facing evidence artifacts
- [`reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes
- [`vocabulary.md`](vocabulary.md): cross-demo vocabulary for evidence workflow terms and bounded correlation
Expand Down
2 changes: 2 additions & 0 deletions docs/reviewer-pack.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ The current artifact names are reviewer-facing contracts for the v1 reviewer con

- Keep the demo matrix stable unless a demo is intentionally retired.
- Prefer additive artifacts over renaming existing reviewer-visible outputs.
- Use [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) for the concrete fourth-to-fifth-demo compatibility baseline.
- If an artifact must be renamed, update the README, reviewer path, this reviewer pack, demo README, tests, and committed sample outputs in the same change.
- For each release, add the diff described in [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md), using `no-artifact-change` when reviewer-facing artifacts are unchanged.
- Do not introduce platform-style names that imply alert routing, case management, real-time ingestion, or autonomous response.
Expand Down Expand Up @@ -107,6 +108,7 @@ Use the same Python interpreter for install, tests, and demo commands.
- [`docs/reviewer-path.md`](reviewer-path.md): demo choice by review question
- [`docs/v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note
- [`docs/v1-readiness-gate.md`](v1-readiness-gate.md): v1.0 readiness gate for fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass
- [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff
- [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON schema contracts for five-demo evidence artifacts
- [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes
- [`docs/vocabulary.md`](vocabulary.md): cross-demo evidence workflow vocabulary
Expand Down
1 change: 1 addition & 0 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Recently added:
- [`docs/architecture.md`](architecture.md) describes the local file-based workflow shape.
- [`docs/vocabulary.md`](vocabulary.md) defines cross-demo evidence workflow terms.
- [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md) defines the release diff contract for reviewer-facing artifact changes.
- [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) records the additive fourth-to-fifth-demo artifact contract.

## v1 Reviewer Contract Stabilization

Expand Down
142 changes: 142 additions & 0 deletions docs/v0.6-to-v1-artifact-diff.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,142 @@
# v0.6.0 to v1 Artifact Contract Diff

This document explains the reviewer-facing artifact contract change from the
tagged `v0.6.0` baseline to the current v1.0 five-demo contract freeze target.

`v0.6.0` introduced the fourth demo,
`config-change-investigation-demo`. The v1 target adds the fifth and final demo,
`cloud-iam-change-investigation-demo`, then closes demo expansion. This is an
additive contract change, not a migration from the fourth demo to the fifth.

## Compatibility Summary

- Overall compatibility: `additive-compatible`.
- No fourth-demo artifact path was removed or renamed.
- The committed fourth-demo artifacts are unchanged from the `v0.6.0` tag.
- The fifth demo adds four stable reviewer-visible artifact paths under its own
demo directory.
- v1 adds JSON Schema validation, artifact regeneration, and reviewer-doc checks
around the five-demo matrix.
- Consumers that inspect only the `v0.6.0` fourth-demo paths do not need to
change.
- Consumers that inspect both demos must dispatch by full artifact path and
schema, not by basename. The two `investigation_summary.json` files have
intentionally different shapes.

## Baseline and Target

| Contract surface | `v0.6.0` fourth demo | v1 fifth demo | Contract effect |
| --- | --- | --- | --- |
| Source evidence | Three synthetic JSONL files for config changes, policy denials, and follow-on events | One synthetic CloudTrail-like JSONL file | Adds a source-specific cloud event skeleton; does not change the config-change inputs. |
| Normalized artifact | `change_events_normalized.json` | `normalized_cloudtrail_events.json` | Adds cloud identity, request, response, Region, source IP, and event-time fields under a new path. |
| Deterministic investigation artifact | `investigation_hits.json` | `investigation_signals.json` | Adds rule-local cloud correlation outputs; does not redefine config-change hits. |
| Summary artifact | Array of per-investigation summaries | One run-level object with counts, time-model metadata, and boundaries | Same basename, different demo-local contract and separate schemas. |
| Human-readable artifact | `investigation_report.md` | `investigation_report.md` | Adds a cloud-specific report under a new demo path; both reports remain review aids rather than canonical machine contracts. |

## Fourth-Demo Artifacts

These `v0.6.0` artifacts remain stable in the v1 target:

| Artifact | Added fields | Removed fields | Semantic changes | Compatibility notes |
| --- | --- | --- | --- | --- |
| `demos/config-change-investigation-demo/artifacts/change_events_normalized.json` | none | none | none | `no-artifact-change` |
| `demos/config-change-investigation-demo/artifacts/investigation_hits.json` | none | none | none | `no-artifact-change` |
| `demos/config-change-investigation-demo/artifacts/investigation_summary.json` | none | none | none | `no-artifact-change`; v1 adds schema validation without changing the committed artifact. |
| `demos/config-change-investigation-demo/artifacts/investigation_report.md` | none | none | none | `no-artifact-change` |

The fourth-demo summary is now locked by
`schemas/investigation_summary.schema.json`. It remains an array of reduced
investigation records with:

- `investigation_id`
- `severity`
- `target_system`
- `triggering_change_id`
- `summary`
- `evidence_counts`
- `bounded_correlation_reason`

## Fifth-Demo Artifacts

All fifth-demo artifact paths are new relative to `v0.6.0`:

| Artifact | Added fields or sections | Removed fields | Semantic changes | Compatibility notes |
| --- | --- | --- | --- | --- |
| `demos/cloud-iam-change-investigation-demo/artifacts/normalized_cloudtrail_events.json` | `eventID`, `event_time`, `observed_time`, `eventTime`, `actor`, `identityType`, `eventSource`, `eventName`, `awsRegion`, `sourceIPAddress`, `userAgent`, `errorCode`, `requestParameters`, `responseElements`, `userIdentity`, `outcome` | none | New CloudTrail-like normalization contract. `eventTime` drives ordering; optional `observedTime` is preserved as `observed_time`. | `non-contract-artifact`; stable reviewer-visible path, but not currently covered by a JSON Schema. |
| `demos/cloud-iam-change-investigation-demo/artifacts/investigation_signals.json` | `signal_id`, rule identity, severity, `signal_time`, actor, primary/evidence event IDs, source IPs, embedded evidence, ATT&CK mappings, bounded-correlation reason, and review scope | none | Introduces `signal` for bounded multi-event cloud correlation. A signal is reviewer evidence, not a production alert or incident verdict. | `additive-compatible`; validated by `schemas/cloud_iam_findings.schema.json`. |
| `demos/cloud-iam-change-investigation-demo/artifacts/investigation_summary.json` | `schema_version`, `source_type`, event/signal counts, per-rule counts, ATT&CK mapping count, time-model metadata, and explicit boundaries | none | Adds a run-level summary object. It is not shape-compatible with the fourth demo's per-investigation summary array. | `additive-compatible`; validated by `schemas/cloud_iam_summary.schema.json`. Dispatch by full path or schema. |
| `demos/cloud-iam-change-investigation-demo/artifacts/investigation_report.md` | `Run Summary`, `Signals`, per-signal evidence and ATT&CK context, and `Boundaries` sections | none | Adds a cloud-specific reviewer narrative tied to signal and event IDs. | `non-contract-artifact`; stable reviewer-visible Markdown, not a JSON Schema contract. |

## Semantic Differences

The fifth demo reuses the evidence workflow vocabulary without forcing the two
investigation demos into one generic object model.

| Dimension | Fourth demo: config change | Fifth demo: Cloud IAM change |
| --- | --- | --- |
| Review unit | `hit`: one risky config change plus attached evidence | `signal`: one bounded rule result over one or more CloudTrail-like events |
| Correlation scope | Same `target_system`, after the trigger, inside a fixed window, and from configured evidence families | Rule-local time window, actor or entity key, and event family set |
| Evidence identity | Nested triggering change, policy denials, and follow-on event records | `primary_event_id`, `evidence_event_ids`, and embedded normalized events |
| Time contract | Source `timestamp` values and a configured correlation window | `eventTime` becomes `event_time`; optional `observedTime` is retained but does not drive ordering |
| ATT&CK context | Not part of the fourth-demo artifact contract | Four to five configured cloud-oriented mappings, attached for reviewer orientation only |
| Summary root | JSON array of per-investigation summaries | JSON object describing the complete deterministic run |
| Boundary visibility | Bounded-correlation reason per investigation | Bounded-correlation reason per signal plus run-level scope boundaries |

`hit` and `signal` are therefore related workflow stages, not interchangeable
field names. See [`docs/vocabulary.md`](vocabulary.md) for the cross-demo term
contract.

## v1 Contract Additions

The artifact matrix is not the only change after `v0.6.0`. The v1 freeze adds
reviewer-verifiable controls around both demos:

1. **Stable paths:** all eight investigation-demo artifacts above are listed in
[`docs/reviewer-pack.md`](reviewer-pack.md#stable-reviewer-visible-artifacts).
2. **Schema validation:** the fourth-demo summary and the fifth-demo signals and
summary validate against separate JSON Schema Draft 2020-12 contracts.
3. **Regeneration:** committed artifacts must match fresh deterministic output
through `python scripts/regenerate_artifacts.py --check`.
4. **Regression checks:** tests keep the demo matrix, artifact links, schema
matrix, terminology, and release-diff requirements aligned.
5. **Release discipline:** future reviewer-facing changes use
[`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md), including an
explicit `no-artifact-change` statement when applicable.

## Consumer Guidance

- Keep existing `v0.6.0` config-change readers unchanged.
- Add fifth-demo readers only for the new cloud artifact paths.
- Do not select a parser using `investigation_summary.json` alone. Use the full
path and the matching schema; the cloud summary also exposes
`schema_version` as `cloud-iam-change-investigation-demo/v1`.
- Treat open `requestParameters` objects as preserved source-specific evidence.
Contracted wrapper objects reject unknown properties.
- Treat ATT&CK mappings as reviewer context, not as attribution or a verdict.

## Verification

From the repository root:

```bash
git diff --name-status v0.6.0..HEAD -- demos/config-change-investigation-demo/artifacts
python scripts/regenerate_artifacts.py --check
python -m pytest tests/test_evidence_pipeline_schemas.py tests/test_reviewer_docs.py tests/test_markdown_links.py
```

The first command should produce no fourth-demo artifact changes. The remaining
commands verify regenerated output, schema conformance, reviewer-contract
language, and local Markdown links.

## Boundaries

This diff covers synthetic, local, file-based reviewer artifacts only:

- Synthetic CloudTrail-like events only.
- No live AWS account.
- No real account ID.
- No production detection claim.
- No real-time ingestion, SIEM, dashboard, alert routing, or case-management
service.
- No autonomous response.
- No final incident verdict.
4 changes: 4 additions & 0 deletions docs/v1-contract-freeze.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,10 @@ this document, [`docs/reviewer-pack.md`](reviewer-pack.md), and
[`docs/reviewer-path.md`](reviewer-path.md) instead of treating `v0.6.0` release
notes as the complete current capability set.

Use [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md) for the
concrete artifact and compatibility change from the fourth demo in `v0.6.0` to
the fifth demo in the v1 freeze target.

## Freeze Scope

The v1.0 freeze scope is the existing five-demo matrix:
Expand Down
1 change: 1 addition & 0 deletions docs/v1-readiness-gate.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Before v1.0:

- keep stable artifact paths listed in [`docs/reviewer-pack.md`](reviewer-pack.md#stable-reviewer-visible-artifacts)
- keep output semantics aligned with [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md)
- verify the `v0.6.0` fourth-demo compatibility baseline in [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md)
- document any intentional artifact shape change in [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md)
- use `no-artifact-change` when committed reviewer artifacts are unchanged for a release

Expand Down
41 changes: 41 additions & 0 deletions tests/test_reviewer_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@ def test_docs_index_separates_current_route_from_history() -> None:
"reviewer-brief.md",
"v1-contract-freeze.md",
"v1-readiness-gate.md",
"v0.6-to-v1-artifact-diff.md",
"evidence-pipeline-contract.md",
"reviewer-artifact-diff.md",
"vocabulary.md",
Expand Down Expand Up @@ -193,6 +194,7 @@ def test_top_level_reviewer_pack_covers_matrix_and_artifact_contract() -> None:
assert "[`docs/reviewer-path.md`](reviewer-path.md)" in reviewer_pack
assert "[`docs/v1-contract-freeze.md`](v1-contract-freeze.md)" in reviewer_pack
assert "[`docs/v1-readiness-gate.md`](v1-readiness-gate.md)" in reviewer_pack
assert "[`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md)" in reviewer_pack
assert "[`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md)" in reviewer_pack
assert "[`docs/vocabulary.md`](vocabulary.md)" in reviewer_pack
assert "[`docs/architecture.md`](architecture.md)" in reviewer_pack
Expand Down Expand Up @@ -393,6 +395,45 @@ def test_v1_readiness_gate_defines_required_release_conditions() -> None:
assert "v1-readiness-gate.md" in text


def test_v06_to_v1_artifact_diff_documents_additive_fifth_demo_contract() -> None:
artifact_diff = _read_repo_file("docs/v0.6-to-v1-artifact-diff.md")
docs_index = _read_repo_file("docs/README.md")
reviewer_pack = _read_repo_file("docs/reviewer-pack.md")
readme = _read_repo_file("README.md")
freeze_doc = _read_repo_file("docs/v1-contract-freeze.md")
readiness_gate = _read_repo_file("docs/v1-readiness-gate.md")
roadmap = _read_repo_file("docs/roadmap.md")

assert "# v0.6.0 to v1 Artifact Contract Diff" in artifact_diff
assert "additive-compatible" in artifact_diff
assert "No fourth-demo artifact path was removed or renamed." in artifact_diff
assert "The committed fourth-demo artifacts are unchanged" in artifact_diff
assert "## Fourth-Demo Artifacts" in artifact_diff
assert "## Fifth-Demo Artifacts" in artifact_diff
assert "## Semantic Differences" in artifact_diff
assert "## v1 Contract Additions" in artifact_diff
assert "## Consumer Guidance" in artifact_diff
assert "## Verification" in artifact_diff
assert "`investigation_hits.json`" in artifact_diff
assert "`investigation_signals.json`" in artifact_diff
assert "`schemas/investigation_summary.schema.json`" in artifact_diff
assert "`schemas/cloud_iam_findings.schema.json`" in artifact_diff
assert "`schemas/cloud_iam_summary.schema.json`" in artifact_diff
assert "Same basename, different demo-local contract" in artifact_diff
assert "No live AWS account" in artifact_diff
assert "final incident verdict" in artifact_diff

for text in [
docs_index,
reviewer_pack,
readme,
freeze_doc,
readiness_gate,
roadmap,
]:
assert "v0.6-to-v1-artifact-diff.md" in text


def test_bounded_correlation_boundaries_are_documented() -> None:
architecture = _read_repo_file("docs/architecture.md")
reviewer_pack = _read_repo_file("docs/reviewer-pack.md")
Expand Down
Loading