Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)

Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Implementation Specs (157)

Detailed engineering and feature implementation specifications.

implementation-specs/specs/BrinkPosOrdersFirstEtlJob/TASK6.1.md 2,554 bytes · Updated Oct 23, 2026 10:44 AM EDT

Raw File

TASK 6.1: Implement Staging-To-Curated Orchestration And Upstream Load-To-Staging Batch Resolution

Summary

Implement the tenant-day staging_to_curated entry point and the ETL/Warehouse orchestration flow that starts one staging -> curated run for Brink order-centric targets.

This subtask turns the replay boundary from TASK5.md and the lifecycle contract from TASK3.md into a concrete execution path that resolves the upstream load-to-staging batch, reads the available staging-load inventory, and prepares curated target execution.

Dependencies

TASK 3 - batch lifecycle, checkpoints, and failure packaging
TASK 5 - load to Redshift staging

Files to Create

1. Staging-to-curated orchestration services

Create or finalize the services responsible for one tenant-day staging_to_curated execution.

Recommended logical services:

BrinkPosStagingToCuratedJobs
BrinkPosLoadToStagingBatchResolver
BrinkPosCuratedTransformOrchestrator

Required Behavior

child staging-to-curated accepts a BrinkPosStagingToCuratedJobInput-style contract
explicit LoadToStagingBatchId wins when provided
otherwise the resolver loads the current successful load_to_staging batch for the same tenant-day key
the upstream load-to-staging batch must be successful and replayable
orchestration resolves required staging-table availability from persisted upstream batch metadata rather than from assumptions about the warehouse state alone
the orchestration layer passes only upstream batch context and target execution inputs into the curated boundary

Testing

Unit tests

Add unit tests for:

explicit upstream load-to-staging batch selection
implicit upstream load-to-staging batch selection through current-state lookup
rejection of failed or invalid upstream load-to-staging batches
staging-inventory resolution from persisted batch metadata
orchestration handoff from child job to curated execution

Acceptance Criteria

[x] Child staging-to-curated job entry exists
[x] Upstream load-to-staging batch resolution exists for explicit and implicit selection paths
[x] Failed or invalid upstream load-to-staging batches are rejected
[x] Required staging inventory is resolved from persisted batch metadata
[x] Unit tests cover upstream resolution and orchestration behavior

Notes

Keep this subtask focused on stage orchestration readiness.
Do not implement dependency matrix or SQL execution here; those belong to later subtasks in TASK 6.