Skip to content

Milestone 0.6 deep metadata inventory test runbook

Use this runbook to validate DataMuru 0.6.0a1 feature by feature. The release extends 0.6.0a0 inventory to tables, views, and columns without provider mutation or table-data reads.

Evidence and safety rules

  • Use only approved sandbox, trial, or read-only identities.
  • Retain command, exit code, redacted output, commit, tester, timestamp, and pass/fail/skip decision for each checkpoint.
  • Never retain tokens, passwords, hosts, account IDs, query IDs, customer identifiers, or provider response bodies.
  • Live tests may inspect only whether DATAMURU_LIVE_DATABRICKS_CONFIG and DATAMURU_LIVE_SNOWFLAKE_CONFIG exist. Do not print their values.
  • Missing configuration is an explicit prerequisite skip, not a pass.
  • Do not use --replace until the existing artifact diff is reviewed.

For a bug, record the smallest redacted fixture, provider, asset type, qualified-name shape, expected/actual result, error code, execution mode, and artifact fingerprint.

1. RED/GREEN TDD evidence

Run the focused history checks and tests:

git log --oneline v0.6.0a0..HEAD
python -m pytest -q tests/unit/test_metadata_inventory_providers.py
python -m pytest -q tests/unit/test_metadata_inventory.py

Expected: history contains separate contract, Databricks, Snowflake, and normalization commits. Recorded RED failures are missing nested contracts, missing provider list/detail calls, missing deep assets, missing budget errors, and missing table/view conflict rejection. Current runs pass.

2. Databricks read-only provider contract

python -m pytest -q tests/unit/test_metadata_inventory_providers.py -k databricks

Expected: budget-sentinel pagination of GET /api/2.1/unity-catalog/tables in pages of at most 50 with columns, properties, and usernames omitted from list responses; encoded table-detail paths; native table/view mapping; deterministic column order; and fail-closed 1,000-relation/10,000-column budgets. No POST, PATCH, DELETE, SQL DDL, apply, destroy, state write, comments, locations, definitions, or credentials.

3. Snowflake read-only provider contract

python -m pytest -q tests/unit/test_metadata_inventory_providers.py -k snowflake

Expected: quoted schema-scoped SHOW TABLES, SHOW VIEWS, and SHOW COLUMNS; preserved case, dots, spaces, Unicode, and quotes; semantic facts/dimensions/metrics excluded; the same global budgets fail closed. A raw schema response of exactly 10,000 rows also fails closed before semantic rows are filtered because Snowflake can truncate SHOW COLUMNS at that provider ceiling.

4. Hierarchy, stable identity, and serialization

python -m pytest -q tests/unit/test_metadata_inventory.py -k "deep or identity or serial"

Expected hierarchy: catalog|database -> schema -> table|view -> column. Every child has the immediate parent ID and address. Qualified names encode each native component independently; IDs and addresses reproduce from published fields. Output ordering is deterministic and schema remains datamuru.metadata_inventory.v1.

5. Duplicate and publication conflicts

python -m pytest -q tests/unit/test_metadata_inventory.py -k "duplicate or conflicting or publication"

Expected: duplicate addresses and table/view classification conflicts fail with DMR-META-1001; first publication is created, semantically identical content is unchanged even when observation timestamps differ, different content does not mutate without --replace, explicit replacement is replaced, concurrent creation is not clobbered, and failed replacement leaves no temporary artifact.

6. CLI, Python API, JSON, and banner compatibility

python -m pytest -q tests/unit/test_metadata_inventory_cli.py
python -m datamuru.cli.main --no-banner inventory export --help

Expected: existing Python signatures and CLI options remain valid. Every JSON success, usage error, and provider error is exactly one parseable object with no ANSI, banner, progress, Rich prose, stderr spill, or secret-bearing provider values. --no-banner always suppresses text banners.

7. ASCII wordmark approval matrix (design only)

Do not implement the wordmark in this release. Retain a static dependency-free proposal below 80 columns. Future compatibility tests must cover interactive TTY text, redirected text, JSON success, JSON usage error, JSON provider error, color-enabled terminals, nested commands, and --no-banner. Only interactive human text may display it; JSON never may; --no-banner is authoritative.

8. Databricks live-readonly validation

python -m pytest -q -m live tests/live/test_metadata_inventory_live.py::test_databricks_metadata_inventory_live_readonly -rs

Expected: pass with only catalog/schema/table/view/column assets, at least one relation and column, immediate parent links, and available column type metadata when DATAMURU_LIVE_DATABRICKS_CONFIG is configured; otherwise skip exactly DATAMURU_LIVE_DATABRICKS_CONFIG is not configured. Inspect provider audit evidence to confirm GET-only metadata calls and no state change.

9. Snowflake live-readonly validation

python -m pytest -q -m live tests/live/test_metadata_inventory_live.py::test_snowflake_metadata_inventory_live_readonly -rs

Expected: pass with only database/schema/table/view/column assets, at least one relation and column, immediate parent links, and available column type metadata when DATAMURU_LIVE_SNOWFLAKE_CONFIG is configured; otherwise skip exactly DATAMURU_LIVE_SNOWFLAKE_CONFIG is not configured. Confirm SHOW-only SQL, no DDL/DML, and no state change.

10. Secret-exposure inspection

python -m pytest -q tests/unit/test_metadata_inventory_cli.py tests/live/test_metadata_inventory_live.py -rs

Expected: credential keys, URLs, provider response bodies, and configured secret values do not appear. Never print a secret to prove its absence.

11. Full local release gate

python -m ruff check datamuru tests
python -m pytest -q
$env:NO_MKDOCS_2_WARNING="1"
python -m mkdocs build --strict
python -m build
python -m twine check dist/*
git diff --check
git status --short

Expected: zero lint/test/docs/build/Twine failures and no unintended files.

12. Independent review, Project, and CI

Review v0.6.0a0..HEAD read-only against the design and plan. Resolve every Critical and Important finding with a new RED/GREEN cycle. Push a PR, set issue

91's Project item to Review, and require every Python matrix, quality,

documentation, and distribution check to succeed before merge.

13. Public release verification

After merge, publish tag v0.6.0a1 through the GitHub Release workflow. Verify:

  • exact GitHub tag/release commit and workflow URL;
  • https://pypi.org/pypi/datamuru/0.6.0a1/json;
  • https://pypi.org/project/datamuru/0.6.0a1/;
  • exact wheel and source filenames;
  • clean-environment installation of datamuru==0.6.0a1;
  • python -m datamuru.cli.main --no-banner --help;
  • a one-object JSON inventory smoke fixture.

Only after all checks pass: attach evidence to issue #91, close it, update its Project Evidence link to the release, and move the item to Done. Record live provider pass/skip results and any remaining grants/lineage/policy/mutation scope.