Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)

Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Implementation Specs (157)

Detailed engineering and feature implementation specifications.

implementation-specs/specs/BrinkPosOrdersFirstEtlJob/TASK6.3.md 2,809 bytes · Updated Oct 23, 2026 10:44 AM EDT

Raw File

TASK 6.3: Implement Curated SQL Catalog, Execution, And Diagnostics

Summary

Implement the execution boundary for the first staging_to_curated stage: run target-specific SQL for the resolved curated target set, apply the chosen rerun-safe curated replacement strategy, and publish safe per-target diagnostics that later publish-stage work can consume.

Dependencies

TASK 6.2 - curated target dependency matrix and v1 field behavior

Files to Create

1. Curated SQL catalog and diagnostics models

Create or finalize the components that select curated SQL templates, execute them, and return target-level diagnostics.

Recommended logical services:

BrinkPosCuratedSqlTemplateCatalog
BrinkPosCuratedSqlExecutor

Required diagnostics details:

target name
required staging tables
SQL template identifier
affected-row count or equivalent execution summary
timing
safe status summary

Files to Modify

2. Warehouse curated execution path

Modify the tenant warehouse curated execution path so staging_to_curated can run the chosen replacement strategy and then execute target SQL for every required curated table.

3. Staging-to-curated batch details

Modify or finalize the staging_to_curated details_json shape so it preserves upstream load-to-staging references and per-target curated execution results.

Required Behavior

the chosen v1 curated replacement strategy is applied consistently before or alongside target SQL as required
curated SQL executes for every required target in the resolved upstream inventory
if any required target fails, the endpoint fails and the full staging_to_curated batch fails
upstream staging state remains for replay and diagnostics
diagnostics must not store secrets
batch metadata must preserve enough detail for TASK7.md to resolve publish_curated inputs confidently

Testing

Unit tests

Add unit tests for:

SQL template selection per target
execution ordering of replacement strategy and target SQL
per-target success and failure diagnostics recording
failure behavior when one required target fails
rerun-safe stage behavior and lifecycle integration
diagnostics sanitization

Acceptance Criteria

[x] Curated SQL executes for every required target in the resolved upstream inventory
[x] The chosen replacement strategy is applied consistently for reruns
[x] Target failures fail the endpoint and stage correctly
[x] Diagnostics stay secret-free and downstream-readable
[x] Unit tests cover execution, rerun semantics, and diagnostics behavior

Notes

This subtask creates the first business-facing Redshift curated outputs for the slice.
Keep curated SQL execution ownership in shared Warehouse code rather than in the Brink adapter.