v0.6.0a2 readiness report¶

This report records every quality-gate failure found while preparing DataMuru 0.6.0a2, its root cause, the corrective change, and the verification evidence. It contains no credential values. Live provider tests remain read-only.

Baseline¶

The milestone branch was created from origin/main commit 5e400635700b425d3fa7246fb6ee3c57162bc291. Baseline commands used clean Python 3.12 and 3.13 virtual environments with the declared development, documentation, and Databricks extras.

Gate	Baseline result	Finding
Non-live pytest	Pass	301 passed and 2 live tests deselected on Python 3.13.
Branch coverage	Pass, measured	72% overall; Databricks auth 87%, client 34%, execution 89%, and provider 58%.
Ruff	Pass	Ruff 0.15.22 was selected by the broad `>=0.2,<0.16` range, so runs were not deterministic.
Strict mypy	Fail	263 errors in 37 files with mypy 1.20.2; the source backlog was 256 errors in 36 files under the pre-existing global environment.
Strict MkDocs	Pass	The site built successfully with MkDocs 1.6.1.
Example validation	Pass after correcting command	`examples/minimal/datamuru.yml` is valid.
Documented validation command	Fail	`datamuru.yml` is intentionally ignored and absent from a clean checkout.
Wheel and sdist build	Pass	`datamuru-0.6.0a1` wheel and source distribution built successfully.
Twine metadata	Pass	Both baseline artifacts passed Twine 6.2.0.
Dependency vulnerability audit	Not configured	The development extra and CI had no dependency-audit tool or command.

Failure ledger¶

Local pytest temporary-directory access¶

Failure: pytest produced setup errors before project behavior because it could not scan %TEMP%\pytest-of-ajays; the same restriction affected a new local base temp when Python ran inside the desktop sandbox.
Root cause: the sandbox denied the Python child process directory enumeration, while the parent shell could create the directory. This was a local execution-layer restriction, not a repository defect.
Fix: no repository workaround. Test commands run outside the sandbox with an explicit ignored worktree-local --basetemp and cache provider disabled.
Verification: python -m pytest -m "not live" --basetemp .pytest-tmp\baseline313 -p no:cacheprovider completed with 301 passed and 2 deselected.

Strict mypy backlog and incomplete typing environment¶

Failure: python -m mypy datamuru reported 263 errors in 37 files.
Root cause: public, core, importer, enterprise, provider, and CLI functions lack strict annotations; JSON dictionaries use unparameterized types; several decoded values are not narrowed; ValidationIssue is constructed positionally; and the development environment lacks PyYAML stubs and a typed Snowflake import boundary.
Fix: add explicit external typing dependencies and repair source annotations, JSON boundaries, return contracts, decorators, and narrowing without changing strict mode or adding blanket ignores.
Verification: strict mypy reports zero errors across all 86 datamuru source files in clean Python 3.12 and 3.13 environments. Python 3.11 is verified by the required pull-request CI matrix because no local 3.11 interpreter is installed.

Typing remediation is being verified in dependency order. Foundational contracts reduced the backlog from 256 to 216 errors. Concrete JSON payload, importer, fulfillment, metadata, engine, and public-API annotations reduced it to 63 errors. The service/API cluster reports zero errors, its focused suite passes all 139 tests, and malformed required fulfillment mappings are covered by four path-specific cases. Typed Snowflake provider models and CLI callback/decorator contracts then reduced the remaining 63 errors to zero across all 86 package source files.

Invalid legacy schema validation raised TypeError¶

Failure: the legacy datamuru.core.schema.SchemaValidator raised TypeError: BaseModel.__init__() takes 1 positional argument but 4 were given while reporting missing or invalid configuration fields.
Root cause: its ValidationIssue Pydantic models were constructed positionally, although Pydantic model initialization requires keyword arguments.
Fix: construct every issue with explicit level, path, and message keywords.
Verification: a new regression test exercises invalid root configuration and asserts structured issues; the focused configuration, governance, and provider suite passes all 24 tests.

Foundational typing import cycle¶

Failure: the first focused run after adding provider/apply annotations failed at import time through apply -> provider base -> importer -> provider factory.
Root cause: imports used only to evaluate annotations were executed at runtime, exposing the pre-existing package initialization dependency cycle.
Fix: retain postponed annotations and place dependency-only imports behind TYPE_CHECKING, leaving runtime imports acyclic.
Verification: the focused 24-test suite passes and strict mypy reports zero errors across schema, configuration, apply, state manager, provider-base/factory, and governance modules; the repository total fell from 256 to 216 errors.

Non-object import resume checkpoint¶

Failure: a syntactically valid JSON checkpoint containing a list or scalar passed through the CLI loader and would later fail when import code treated it as a mapping.
Root cause: the JSON decoding boundary declared a dictionary contract without validating the decoded root type.
Fix: require a JSON object immediately and return a clear Click validation error before loading configuration or contacting a provider.
Verification: a CLI regression supplies [], asserts the path-specific message, and confirms no AttributeError; the complete provider/CLI focused suite passes all 64 tests.

Databricks transport accepted invalid response shapes¶

Failure: GET, POST, and account-SCIM transports accepted JSON lists or scalars despite declaring object results; POST also allowed Requests' raw JSON decode error to escape for a non-JSON success response.
Root cause: decoded responses were cast or returned as dictionaries without runtime root-type validation, and the POST path lacked the error mapping already used by GET and SCIM.
Fix: require a JSON object at every transport boundary, preserve empty successful responses where the API allows them, and map invalid JSON and invalid root shapes to structured ProviderError instances.
Verification: red/green regression cases cover lists, scalars, non-JSON bodies, empty bodies, HTTP failures, generic request failures, accepted statuses, and timeout propagation for GET, POST, DELETE, and SCIM.

Databricks failure context could echo bearer tokens¶

Failure: SDK and Requests exception text was copied verbatim into provider error context and could echo a resolved bearer token if the underlying exception included it.
Root cause: transport diagnostics were bounded but not credential-aware.
Fix: redact the resolved bearer token from all dynamic Databricks exception and response text before storing it in result details or error context.
Verification: failing-then-passing SDK and timeout regressions use sentinel tokens, assert the sentinel is absent, and assert [REDACTED] is present.

Databricks branch-coverage gaps¶

Failure: the focused baseline covered only 40% of Databricks implementation branches: auth 52%, client 25%, execution 78%, and provider 46%.
Root cause: existing tests concentrated on high-level discovery and successful metadata mapping, leaving profile precedence, SDK construction, request errors, response shapes, SCIM capabilities, pagination, SQL polling, and timeout behavior lightly exercised.
Fix: add isolated auth and mocked transport/probe/SCIM/SQL/pagination tests. Mocked mutation builders verify payloads without contacting a workspace.
Verification: the focused 89-test suite reaches 67% branch coverage overall: auth 96%, client 86%, execution 89%, and provider 54%. All I/O in these tests is mocked; no live mutation occurs.

Live-readonly SQL lacked a client-level guard¶

Failure: direct Databricks SQL client calls could submit a mutating statement even when authentication declared execution_mode: live-readonly; provider apply paths were guarded, but the lower-level client boundary was not.
Root cause: the execution policy protected provider resources but SQL submission trusted every caller to enforce the mode.
Fix: reject SQL before transport in live-readonly mode unless its leading statement type is SELECT, SHOW, DESCRIBE, or EXPLAIN. Error context records only the statement type, not SQL text or object names.
Verification: a failing-then-passing regression proves DROP is blocked before the mocked POST transport. Existing live-apply mutation-builder tests explicitly switch to live-apply.

Configured Databricks live-readonly verification¶

Call review: workspace connectivity uses GET against SCIM Me with a Unity Catalog GET fallback; identity capability and optional group listing use SCIM GET; bounded inventory uses Unity Catalog catalog/schema/table/table-detail GET; SQL submits only SELECT current_user(), current_catalog() and polls statement status with GET.
Boundaries: the ignored config contains environment-variable names only, declares live-readonly, selects one readable non-system catalog, and caps discovery at 50 relations and 5,000 columns. No create, update, patch, delete, grant, revoke, apply, or destroy path is called.
Verification: three configured Databricks live tests passed in 62.9 seconds: connectivity/SCIM, bounded metadata hierarchy, and SQL metadata. The unconfigured Snowflake live test skipped. Results were checked in memory for the configured token.

Non-deterministic Ruff selection¶

Failure: the development dependency permits any Ruff from 0.2 through 0.15, selecting 0.15.22 in the clean environment.
Root cause: the wide range makes lint behavior dependent on install date and resolver state.
Fix: pin Ruff 0.15.22, the final verified 0.15 patch. Ruff 0.16.1 changes the effective default selection from the repository's established E/F policy to a much broader set and produced 112 unrelated migration findings. Expanding lint policy is intentionally separate from readiness defect repair.
Verification: the exact configured python -m ruff check datamuru tests command exits zero with Ruff 0.15.22 locally; CI repeats the same deterministic gate.

Missing dependency security gate¶

Failure: neither project metadata nor CI defines a dependency vulnerability audit.
Root cause: the original quality workflow covers lint, docs, build, and Twine, but not known-vulnerability scanning.
Fix: pin pip-audit 2.10.1 and add an enforcing CI command.
Verification: python -m pip_audit exits zero with no known vulnerabilities in the resolved Python 3.12 and 3.13 release environments; CI repeats the same gate.

Vulnerable resolved development and Snowflake dependencies¶

Failure: the first pip-audit run found nine advisory records across cryptography 46.0.0, pyOpenSSL 25.3.0, and pytest 8.4.2.
Root cause: Snowflake connector 3.18 constrained pyOpenSSL below the fixed 26.0 line and allowed a vulnerable cryptography release; the test extra constrained pytest below the fixed 9.0.3 release. The unused pytest-asyncio dependency also constrained the older pytest generation.
Fix: require Snowflake connector 4.6 or newer, which upstream documents as having removed the pyOpenSSL upper bound and raised the cryptography security floor; require pytest 9.0.3 or newer; remove unused pytest-asyncio.
Verification: reinstall all quality/provider extras, rerun every test, and require python -m pip_audit to report no known vulnerabilities. A clean Python 3.13 environment resolved Snowflake connector 4.7.1, cryptography 49.0.0, pyOpenSSL 26.3.0, and pytest 9.1.1; pip check passed and pip-audit reported no known vulnerabilities.

Invalid documented validation command¶

Failure: python -m datamuru.cli.main validate --config datamuru.yml fails in a clean checkout because the referenced file is absent.
Root cause: repository-root generated project files are intentionally ignored; the documentation used a local generated path instead of the tracked runnable example.
Fix: update the documented gate to use examples/minimal/datamuru.yml.
Verification: the corrected command already exits zero and reports Configuration is valid.

GitHub Actions runtime maintenance¶

Failure: the workflows use action revisions released for the superseded Node.js 20 runtime, including checkout/setup and Pages artifact/deployment actions.
Root cause: action references were current for the previous milestone but were not refreshed after upstream Node.js 24 major releases.
Fix: upgrade checkout and setup-python to 7.0.1 and 7.0.0; the Pages actions to configure 6.0.0, upload 5.0.0, and deploy 5.0.0; upload/download-artifact to 7.0.1 and 8.0.1; lychee-action to 2.9.0; and PyPI publish to 1.14.1.
Verification: pull-request and post-merge workflows must complete without deprecated-runtime annotations.

Readiness ledger version-policy false positive¶

Failure: the first pytest 9 run failed the stale-public-version contract because this readiness report intentionally names both the baseline 0.6.0a1 artifacts and target 0.6.0a2 release.
Root cause: the documentation contract excluded historical runbooks and planning records, but not the operational audit report that necessarily records before/after release versions.
Fix: add this exact audit-report path to the version-record allowlist; active release documentation remains subject to the canonical-version assertion.
Verification: rerun the documentation contract and full non-live suite under pytest 9.

Non-configured Ruff formatter probe¶

Failure: an additional ruff format --check . probe reported that 76 files would be reformatted.
Root cause: the repository does not configure or enforce Ruff formatting; its established lint contract is ruff check datamuru tests, and the formatter would rewrite broad pre-existing style outside this milestone's behavioral scope.
Fix: no source rewrite. Keep the deterministic configured lint policy and record formatter adoption as a separate repository-wide migration decision.
Verification: the configured Ruff 0.15.22 lint command exits zero.

Wheel smoke output truncation¶

Failure: the first packaged-CLI smoke command returned a nonzero shell status after piping --help through PowerShell output truncation, although wheel installation and the 0.6.0a2 import assertion had already passed.
Root cause: the consumer pipeline closed before the CLI finished writing help; this was a shell-pipeline artifact, not a package or CLI failure.
Fix: rerun the installed CLI without an output-truncating pipeline.
Verification: the wheel-installed datamuru --no-banner --help command exits zero.

Final verification evidence¶

Gate	Verified result
Python 3.12 non-live suite	358 passed, 4 live tests deselected.
Python 3.13 non-live suite and branch coverage	358 passed, 4 live tests deselected; 77% overall branch coverage.
Python 3.11 compatibility	Required Python 3.11 CI matrix job; no local 3.11 interpreter is available.
Databricks focused suite	89 passed; auth 96%, client 86%, execution 89%, provider 54%, 67% combined branch coverage.
Databricks configured live-readonly suite	3 passed; the unconfigured Snowflake live test skipped.
Ruff	Ruff 0.15.22, all configured checks passed.
Strict mypy	Zero errors across 86 source files on Python 3.12 and 3.13.
Strict documentation	MkDocs 1.6.1 built the complete site successfully.
Dependency consistency and audit	`pip check` found no broken requirements; pip-audit 2.10.1 found no known vulnerabilities.
Build metadata	Wheel and source distribution are both versioned `0.6.0a2`; strict Twine checks passed.
Installed-wheel smoke test	Runtime version is `0.6.0a2`; packaged CLI help exits zero.
Archive hygiene	97 wheel and 121 sdist members inspected; no environment, pytest temporary, cache, coverage, site, or nested distribution content found.

The generated artifacts remain ignored and uncommitted. SHA-256 digests are calculated from the final post-verification build for handoff rather than embedded here, because archive timestamps make a digest recorded before the final build stale.

Live-readonly boundary¶

The available environment provides Databricks host, token, and SQL warehouse values. Their values must never be printed. Live verification will run only reviewed GET, SCIM-list, and allowlisted read-only SQL metadata paths. Mutation-capable methods will be covered with mocked transport tests and will not be called against the configured workspace.

Release boundary¶

This milestone prepares and validates 0.6.0a2. It does not publish to PyPI, create a GitHub release, or create a version tag. Those actions require separate explicit approval confirming the version.