Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,18 @@ A local, file-based detection workflow lab for reviewer-verifiable telemetry and

Current focus: v1 reviewer contract stabilization for the five-demo matrix, with v1.0 treated as a five-demo contract freeze.

Latest tagged release: [v0.6.0 — fourth demo and config-change investigation](https://github.com/stacknil/telemetry-lab/releases/latest).

Release drift note: current `main` is ahead of `v0.6.0` and contains the five-demo matrix plus v1 contract-freeze docs. Until a `v1.0` tag exists, use [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md) and [`docs/reviewer-pack.md`](docs/reviewer-pack.md) for current main-branch review.
Latest tagged release: [v1.0 — reviewer contract release](https://github.com/stacknil/telemetry-lab/releases/tag/v1.0).

## Reviewer Start

- [`docs/reviewer-brief.md`](docs/reviewer-brief.md): scope, value, evidence, and boundaries
- [`docs/reviewer-path.md`](docs/reviewer-path.md): choose the right demo by review question
- [`docs/reviewer-pack.md`](docs/reviewer-pack.md): demo matrix, artifact contract, and v1 readiness gate
- [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note
- [`docs/v1-contract-freeze.md`](docs/v1-contract-freeze.md): v1.0 five-demo contract freeze, release status, and contract scope
- [`docs/v1-readiness-gate.md`](docs/v1-readiness-gate.md): fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass requirements
- [`docs/release-v1.0.md`](docs/release-v1.0.md): draft v1.0 reviewer-contract release notes and explicit non-SIEM boundary
- [`docs/release-v1.0.md`](docs/release-v1.0.md): v1.0 reviewer-contract release notes and explicit non-SIEM boundary
- [`docs/v0.6-to-v1-artifact-diff.md`](docs/v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff
- [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md): JSON schema contracts for reviewer-facing evidence artifacts
- [`docs/evidence-pipeline-contract.md`](docs/evidence-pipeline-contract.md): JSON/JSONL schema contracts for reviewer-facing evidence artifacts
- [`docs/reviewer-artifact-diff.md`](docs/reviewer-artifact-diff.md): release artifact diff contract for reviewer-facing outputs
- [`docs/vocabulary.md`](docs/vocabulary.md): cross-demo vocabulary for events, hits, signals, bounded correlation, findings, summaries, reports, and audit traces
- [`docs/README.md`](docs/README.md): current route, supporting docs, and historical release evidence
Expand Down Expand Up @@ -195,7 +193,7 @@ Cooldown behavior:
- treat v1.0 as a five-demo contract freeze, not a feature expansion
- stabilize the five-demo matrix and avoid broad platform expansion
- freeze reviewer-visible artifact names unless a rename is intentionally coordinated across docs, tests, and sample outputs
- keep JSON schema contracts aligned with reviewer-facing JSON evidence artifacts across the five-demo matrix
- keep JSON schema contracts aligned with reviewer-facing JSON and JSONL evidence artifacts across the five-demo matrix
- keep committed artifacts aligned with regenerated pipeline output through `python scripts/regenerate_artifacts.py --check`
- add a reviewer-facing artifact diff for each release, using `no-artifact-change` when committed reviewer artifacts are unchanged
- use [`docs/reviewer-pack.md`](docs/reviewer-pack.md) and [`docs/architecture.md`](docs/architecture.md) as the consolidation entrypoints
Expand Down
6 changes: 3 additions & 3 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ This directory separates the current reviewer route from supporting design notes
- [`reviewer-pack.md`](reviewer-pack.md): top-level reviewer pack, demo matrix, artifact naming contract, and v1 readiness gate
- [`reviewer-path.md`](reviewer-path.md): choose a demo by review question
- [`reviewer-brief.md`](reviewer-brief.md): short problem, value, evidence, and boundary summary
- [`v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note
- [`v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze, release status, and contract scope
- [`v1-readiness-gate.md`](v1-readiness-gate.md): fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass requirements
- [`release-v1.0.md`](release-v1.0.md): draft reviewer-contract release notes with the explicit non-SIEM boundary
- [`release-v1.0.md`](release-v1.0.md): v1.0 reviewer-contract release notes with the explicit non-SIEM boundary
- [`v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): additive artifact contract and compatibility diff from the fourth demo to the fifth
- [`evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON schema contracts for reviewer-facing evidence artifacts
- [`evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON/JSONL schema contracts for reviewer-facing evidence artifacts
- [`reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes
- [`vocabulary.md`](vocabulary.md): cross-demo vocabulary for evidence workflow terms and bounded correlation
- [`architecture.md`](architecture.md): local file-based workflow diagram
Expand Down
12 changes: 9 additions & 3 deletions docs/evidence-pipeline-contract.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Evidence Pipeline Contract

`telemetry-lab` v1.0 treats reviewer-facing JSON artifacts as evidence pipeline contracts. The schemas below define the current machine-readable artifact shapes across the five-demo matrix without turning the repo into a SIEM, dashboard, or monitoring platform.
`telemetry-lab` v1.0 treats reviewer-facing JSON and JSONL artifacts as evidence pipeline contracts. The schemas below define the current machine-readable artifact shapes across the five-demo matrix without turning the repo into a SIEM, dashboard, or monitoring platform.

Use [`docs/vocabulary.md`](vocabulary.md) for the cross-demo meaning of `event`, `signal`, `hit`, `finding`, `case_bundle`, `summary`, `report`, and `audit_trace`.

The contract is intentionally local and file-based:

- schemas live under `schemas/`
- committed artifacts live under `demos/*/artifacts/`
- tests validate the schemas against the committed artifacts
- tests validate the schemas against the committed JSON artifacts and JSONL records
- raw evidence payloads may preserve source-specific fields, but deterministic wrapper fields stay explicit

## Schema Matrix
Expand All @@ -18,8 +18,14 @@ The contract is intentionally local and file-based:
| `schemas/telemetry_summary.schema.json` | `data/processed/summary.json`, `data/processed/richer_sample/summary.json` | telemetry-window run counts, rule counts, cooldown, and generated artifact references |
| `schemas/rule_hits.schema.json` | `demos/ai-assisted-detection-demo/artifacts/rule_hits.json` | deterministic rule-hit fields before case grouping |
| `schemas/case_bundles.schema.json` | `demos/ai-assisted-detection-demo/artifacts/case_bundles.json` | bounded case bundles passed to JSON-only drafting |
| `schemas/case_summaries.schema.json` | `demos/ai-assisted-detection-demo/artifacts/case_summaries.json` | accepted case summaries after schema and semantic validation |
| `schemas/ai_audit_traces.schema.json` | `demos/ai-assisted-detection-demo/artifacts/audit_traces.jsonl` | JSONL audit records for accepted and rejected AI-assisted drafting paths |
| `schemas/dedup_rule_hits.schema.json` | `demos/rule-evaluation-and-dedup-demo/artifacts/rule_hits_before_dedup.json`, `demos/rule-evaluation-and-dedup-demo/artifacts/rule_hits_after_dedup.json` | raw rule hits before deduplication and retained alert records after cooldown handling |
| `schemas/dedup_explanations.schema.json` | `demos/rule-evaluation-and-dedup-demo/artifacts/dedup_explanations.json` | retained/suppressed cooldown explanations |
| `schemas/config_change_events.schema.json` | `demos/config-change-investigation-demo/artifacts/change_events_normalized.json` | normalized risky-change input context before rule matching |
| `schemas/config_investigation_hits.schema.json` | `demos/config-change-investigation-demo/artifacts/investigation_hits.json` | full config-change investigation records with triggering changes and attached bounded evidence |
| `schemas/investigation_summary.schema.json` | `demos/config-change-investigation-demo/artifacts/investigation_summary.json` | config-change investigation summaries and bounded evidence counts |
| `schemas/cloudtrail_normalized_events.schema.json` | `demos/cloud-iam-change-investigation-demo/artifacts/normalized_cloudtrail_events.json` | normalized synthetic CloudTrail-like event records |
| `schemas/cloud_iam_findings.schema.json` | `demos/cloud-iam-change-investigation-demo/artifacts/investigation_signals.json` | bounded CloudTrail-like IAM findings |
| `schemas/cloud_iam_summary.schema.json` | `demos/cloud-iam-change-investigation-demo/artifacts/investigation_summary.json` | Cloud IAM investigation summary and time-model metadata |

Expand All @@ -43,7 +49,7 @@ python -m pytest tests/test_evidence_pipeline_schemas.py

The regeneration check compares byte-stable CSV, JSON, JSONL, and Markdown artifacts with fresh pipeline output. It also regenerates PNG visual snapshots to verify that the plotting path still runs, but it does not byte-compare those images because Matplotlib rendering can vary across platforms.

The schema test validates each schema file and checks that the committed artifact listed in the schema matrix conforms to it.
The schema test validates each schema file and checks that every committed JSON artifact and JSONL record listed in the schema matrix conforms to it.

## Release Artifact Diff

Expand Down
10 changes: 4 additions & 6 deletions docs/release-v1.0.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
# v1.0 Reviewer Contract Release Notes (Draft)
# v1.0 Reviewer Contract Release Notes

**This is a reviewer-contract release, not a production SIEM.**

Release status: Draft. This document does not create a `v1.0` tag or GitHub
release. Final publication remains gated by
Release status: v1.0 reviewer-contract release. Publication is gated by
[`docs/v1-readiness-gate.md`](v1-readiness-gate.md).

## Release Scope
Expand All @@ -26,7 +25,7 @@ The v1.0 release contract requires:

- fixed synthetic inputs for the five demos
- fixed reviewer-visible output paths
- JSON Schema validation for contracted evidence artifacts
- JSON Schema validation for reviewer-facing JSON and JSONL evidence artifacts
- reproducible committed artifacts
- a passing full test suite

Expand All @@ -51,8 +50,7 @@ field-level and semantic diff. Future release diffs follow

## Validation Snapshot

Current draft validation snapshot; refresh these results from the release
candidate commit before publication:
Validation snapshot from the release candidate commit:

```bash
python scripts/regenerate_artifacts.py --check
Expand Down
14 changes: 10 additions & 4 deletions docs/reviewer-pack.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ The current artifact names are reviewer-facing contracts for the v1 reviewer con

## Evidence Pipeline Contract

See [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md) for the v1 JSON schema contract covering reviewer-facing JSON evidence artifacts across the five-demo matrix.
See [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md) for the v1 JSON schema contract covering reviewer-facing JSON and JSONL evidence artifacts across the five-demo matrix.

See [`docs/vocabulary.md`](vocabulary.md) for the cross-demo meaning of `event`, `signal`, `hit`, `finding`, `case_bundle`, `summary`, `report`, and `audit_trace`.

Expand All @@ -51,8 +51,14 @@ The current schema contract covers:

- `schemas/rule_hits.schema.json`
- `schemas/case_bundles.schema.json`
- `schemas/case_summaries.schema.json`
- `schemas/ai_audit_traces.schema.json`
- `schemas/dedup_rule_hits.schema.json`
- `schemas/dedup_explanations.schema.json`
- `schemas/config_change_events.schema.json`
- `schemas/config_investigation_hits.schema.json`
- `schemas/investigation_summary.schema.json`
- `schemas/cloudtrail_normalized_events.schema.json`
- `schemas/cloud_iam_findings.schema.json`
- `schemas/cloud_iam_summary.schema.json`
- `schemas/telemetry_summary.schema.json`
Expand Down Expand Up @@ -106,11 +112,11 @@ Use the same Python interpreter for install, tests, and demo commands.
- [`docs/README.md`](README.md): docs index for the current route, supporting docs, and historical release evidence
- [`docs/reviewer-brief.md`](reviewer-brief.md): short problem / value summary
- [`docs/reviewer-path.md`](reviewer-path.md): demo choice by review question
- [`docs/v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze and release drift note
- [`docs/v1-contract-freeze.md`](v1-contract-freeze.md): v1.0 five-demo contract freeze, release status, and contract scope
- [`docs/v1-readiness-gate.md`](v1-readiness-gate.md): v1.0 readiness gate for fixed inputs, fixed outputs, schema validation, artifact regeneration, and test pass
- [`docs/release-v1.0.md`](release-v1.0.md): draft v1.0 reviewer-contract release notes and explicit non-SIEM boundary
- [`docs/release-v1.0.md`](release-v1.0.md): v1.0 reviewer-contract release notes and explicit non-SIEM boundary
- [`docs/v0.6-to-v1-artifact-diff.md`](v0.6-to-v1-artifact-diff.md): fourth-to-fifth-demo artifact contract and compatibility diff
- [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON schema contracts for five-demo evidence artifacts
- [`docs/evidence-pipeline-contract.md`](evidence-pipeline-contract.md): JSON/JSONL schema contracts for five-demo evidence artifacts
- [`docs/reviewer-artifact-diff.md`](reviewer-artifact-diff.md): release diff contract for reviewer-facing artifact changes
- [`docs/vocabulary.md`](vocabulary.md): cross-demo evidence workflow vocabulary
- [`docs/architecture.md`](architecture.md): local file-based workflow diagram
Expand Down
4 changes: 2 additions & 2 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Next phase: v1 reviewer contract stabilization.

The concrete milestone is [`v1.0 Five-Demo Contract Freeze`](v1-contract-freeze.md).

The repo now has five reviewer-verifiable demos and a clear [`docs/reviewer-path.md`](reviewer-path.md). The priority is to keep the demo matrix stable, preserve artifact names, make release drift explicit, and make review faster without implying a SIEM, dashboard, or production monitoring platform.
The repo now has five reviewer-verifiable demos and a clear [`docs/reviewer-path.md`](reviewer-path.md). The priority is to keep the demo matrix stable, preserve artifact names, keep release evidence explicit, and make review faster without implying a SIEM, dashboard, or production monitoring platform.

Recently added:

Expand All @@ -26,7 +26,7 @@ Recently added:
1. Stabilize the demo matrix.
2. Treat v1.0 as a five-demo contract freeze, not a feature expansion.
3. Freeze reviewer-visible artifact names unless a rename is intentional and documented across README, reviewer docs, demo docs, tests, and sample outputs.
4. Keep JSON schema contracts aligned with reviewer-facing JSON evidence artifacts across the five-demo matrix.
4. Keep JSON schema contracts aligned with reviewer-facing JSON and JSONL evidence artifacts across the five-demo matrix.
5. Keep committed evidence artifacts aligned with regenerated pipeline output through `python scripts/regenerate_artifacts.py --check`.
6. Keep cross-demo vocabulary stable for evidence workflow terms.
7. Include reviewer-facing artifact diffs in every release, including explicit `no-artifact-change` notes when committed reviewer artifacts are unchanged.
Expand Down
Loading
Loading