Reporting & connections

Conformare captures the authored pipeline once and then lets you report on it at two levels:

  • Per run — a self-contained, interactive HTML report (plus machine-readable exports) that documents a single pipeline execution: lineage diagram, profiles, risks, sensitivity, and the column catalog.
  • Across runs — a record_run call that appends an immutable, metadata-only record to a central fleet store, so a whole organisation can answer governance questions over every run: global risks, process health, active sources and sinks, the last healthy run, owner succession, and implementation-vs-risk drift.

The shape of it

The design deliberately separates three concerns, so where results land is a swappable detail:

  1. CapturetrackSpark() / trackPandas() / trackNarwhals() build the in-memory lineage store as your code runs.
  2. Report / recordto_html(), to_json(), … render one run; to_run_record() serializes one run into a stable, versioned RunRecord (pure, no I/O).
  3. Connect — a writer sends the RunRecord to a destination: a Spark table (no blob access needed), JSON or Parquet files, or any callable.

Nothing is sent anywhere automatically — per-run reporting and cross-run recording are both explicit calls.

The pages here

  • The HTML report — what’s in the interactive per-run report and the other per-run exports (to_json, to_mermaid, to_risk_checklist).
  • Fleet recordingrecord_run, the RunRecord schema, run identity and immutability.
  • Connections — the available writers/stores (spark_table, json, parquet, callable), configure_store, and reading runs back.
  • Cross-run governance — the review report (implementation-vs-risk drift), last healthy run, owner succession, and sinks “as of” a time.

Table of contents


This site uses Just the Docs, a documentation theme for Jekyll.