Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)

Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Implementation Specs (157)

Detailed engineering and feature implementation specifications.

implementation-specs/specs/BrinkPosOrdersFirstEtlJob/TASK3.1.md 2,664 bytes · Updated Oct 23, 2026 10:44 AM EDT

Raw File

TASK 3.1: Extend Stage Batch And Checkpoint Contracts

Summary

Extend the shared ingestion metadata contract so stage-level tenant-day batches and checkpoints are explicit first-class runtime concepts for the Brink orders ETL flow.

This subtask turns the lifecycle decisions into a concrete batch and checkpoint surface that later stages can reuse without inventing their own metadata shape.

Dependencies

TASK 1 - source binding and dispatch
TASK 2 - extract raw XML

Files to Create

1. Shared stage lifecycle contracts

Create or finalize the minimum shared models and service contracts for:

stage batch creation
stage current-batch lookup
stage checkpoint read/write
explicit upstream batch references

Recommended logical services:

IngestionBatchLifecycleService
CurrentIngestionBatchResolver
IngestionCheckpointService

Files to Modify

2. Ingestion batch model and persistence

Modify the batch entity, mapping, and repository surface so the following data is queryable without parsing ad-hoc JSON only:

company_id
tenant_id
business_date
pipeline_stage
details_json
error_details
reset_of_batch_id
is_current
explicit upstream batch reference

3. Checkpoint model and persistence

Modify the checkpoint path so stage checkpoints remain tenant-scoped, deterministic, and independent per pipeline stage.

Required Behavior

every stage uses its own tenant-day logical batch key
the logical key is tenant_id + business_date + pipeline_stage
company_id remains traceability metadata, not the primary current-state selector
every downstream stage can store an explicit upstream batch reference
checkpoint keys remain generic and must not encode Brink-only watermark semantics
repository/query APIs must support current successful batch lookup and historical batch inspection

Testing

Unit tests

Add unit tests for:

batch persistence of the required lifecycle fields
current-batch query behavior
checkpoint persistence and lookup for stage-specific tenant-day keys
explicit upstream batch reference persistence

Acceptance Criteria

[x] Stage lifecycle fields exist in the shared batch contract
[x] Stage-specific checkpoint support exists
[x] Explicit upstream batch reference support exists
[x] Current successful batch lookup is queryable
[x] Unit tests cover batch and checkpoint contract persistence

Notes

Keep the contract generic enough for non-Brink adapters.
Do not move endpoint diagnostics out of details_json; only lifecycle identity and lineage must become first-class.