Built Word Documents (.docx):
Spensara Overview (.docx)
Capability Assessment (.docx)
Brink POS Ingestion Analysis (.docx)
The Borg Ingestion Architecture (.docx)
Categories
Implementation Specs (157)
Detailed engineering and feature implementation specifications.
- specs/AnomalyDetection/TASK1.1.md
- specs/AnomalyDetection/TASK1.2.md
- specs/AnomalyDetection/TASK1.3.md
- specs/AnomalyDetection/TASK1.4.md
- specs/AnomalyDetection/TASK2.1.md
- specs/AnomalyDetection/TASK2.2.md
- specs/AnomalyDetection/TASK3.1.md
- specs/AnomalyDetection/TASK3.2.md
- specs/AnomalyDetection/TASK3.3.md
- specs/AnomalyDetection/TASK4.1.md
- specs/AnomalyDetection/TASK4.2.md
- specs/AnomalyDetection/TASK4.3.md
- specs/AnomalyDetection/TASK4.4.md
- specs/AnomalyDetection/TASK5.1.md
- specs/AnomalyDetection/TASK5.2.md
- specs/AnomalyDetection/TASK5.3.md
- specs/AnomalyDetection/TASK6.1.md
- specs/AnomalyDetection/TASK6.2.md
- specs/AnomalyDetection/TASK6.3.md
- specs/AnomalyDetection/TASKS.md
- specs/ApiServicePostgresTestStability/TASK.md
- specs/BackendDatabaseConsolidation/TASK.md
- specs/BackendDatabaseConsolidation/TASK1.1.md
- specs/BackendDatabaseConsolidation/TASK1.2.md
- specs/BackendDatabaseConsolidation/TASK1.3.md
- specs/BackendDatabaseConsolidation/TASK2.1.md
- specs/BackendDatabaseConsolidation/TASK2.2.md
- specs/BackendDatabaseConsolidation/TASK2.3.md
- specs/BackendDatabaseConsolidation/TASK3.1.md
- specs/BackendDatabaseConsolidation/TASK3.2.md
- specs/BackendDatabaseConsolidation/TASK4.1.md
- specs/BackendDatabaseConsolidation/TASK4.2.md
- specs/BackendDatabaseConsolidation/TASK4.3.md
- specs/BackendDatabaseConsolidation/TASK5.1.md
- specs/BackendDatabaseConsolidation/TASK5.2.md
- specs/BrinkLaborGetShiftsStaging/TASK.md
- specs/BrinkPosOrdersFirstEtlJob/TASK.md
- specs/BrinkPosOrdersFirstEtlJob/TASK0.md
- specs/BrinkPosOrdersFirstEtlJob/TASK1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.md
- specs/BrinkPosOrdersFirstEtlJob/TASK7.md
- specs/BrinkPosOrdersFirstEtlJob/TASK8.md
- specs/BrinkPosOrdersFirstEtlJob/TASK9.md
- specs/BrinkSettings2ToStaging/TASK.MD
- specs/CuratedToS3Stage/TASK.md
- specs/CuratedToS3Stage/TASK0.md
- specs/CuratedToS3Stage/TASK1.md
- specs/CuratedToS3Stage/TASK2.md
- specs/CuratedToS3Stage/TASK3.md
- specs/CuratedToS3Stage/TASK4.md
- specs/CustomerCredentialCustomHeaders/TASK.md
- specs/CustomerCredentialCustomHeaders/TASK1.1.md
- specs/CustomerCredentialCustomHeaders/TASK1.2.md
- specs/CustomerCredentialCustomHeaders/TASK2.1.md
- specs/CustomerCredentialCustomHeaders/TASK2.2.md
- specs/CustomerCredentialCustomHeaders/TASK3.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK.md
- specs/CustomerCredentialTriggeredProvisioning/TASK1.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK1.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK2.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK2.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK3.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK3.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK4.1.md
- specs/EtlSourceAdapterFramework/TASK.md
- specs/EtlSourceAdapterFramework/TASK0.md
- specs/EtlSourceAdapterFramework/TASK1.md
- specs/EtlSourceAdapterFramework/TASK2.md
- specs/EtlSourceAdapterFramework/TASK3.md
- specs/EtlSourceAdapterFramework/TASK4.md
- specs/EtlSourceAdapterFramework/TASK5.md
- specs/EtlSourceAdapterFramework/TASK6.md
- specs/KdsCdcDebeziumIntegration/TASK.md
- specs/KdsCdcDebeziumIntegration/TASK1.md
- specs/KdsCdcDebeziumIntegration/TASK2.md
- specs/KdsCdcDebeziumIntegration/TASK3.md
- specs/KdsCdcDebeziumIntegration/TASK4.md
- specs/KdsCdcDebeziumIntegration/TASK5.md
- specs/KdsCdcDurableRelationRouting/TASK.md
- specs/KdsCdcDurableRelationRouting/TASK1.md
- specs/KdsCdcDurableRelationRouting/TASK2.md
- specs/KdsCdcDurableRelationRouting/TASK3.md
- specs/KdsCdcDurableRelationRouting/TASK4.md
- specs/KdsCdcDurableRelationRouting/TASK5.md
- specs/KdsCdcIngestion/TASK.md
- specs/KdsCdcIngestion/TASK1.md
- specs/KdsCdcIngestion/TASK2.md
- specs/KdsCdcIngestion/TASK3.md
- specs/KdsCdcIngestion/TASK4.md
- specs/KdsCdcIngestion/TASK5.md
- specs/KdsCdcIngestion/TASK6.md
- specs/KdsCdcIngestion/TASK7.md
- specs/KdsCdcIngestion/TASK8.md
- specs/KdsCdcIngestion/TASK9.md
- specs/KdsCdcReplayExecution/TASK.md
- specs/KdsCdcReplayExecution/TASK1.md
- specs/KdsCdcReplayExecution/TASK2.md
- specs/KdsCdcReplayExecution/TASK3.md
- specs/KdsCdcReplayExecution/TASK4.md
- specs/KdsMonitorServingContractMigration/TASK.md
- specs/KdsMonitorServingContractMigration/TASK1.1.md
- specs/KdsMonitorServingContractMigration/TASK1.2.md
- specs/KdsMonitorServingContractMigration/TASK2.1.md
- specs/KdsMonitorServingContractMigration/TASK2.2.md
- specs/KdsMonitorServingContractMigration/TASK3.1.md
- specs/KdsMonitorServingContractMigration/TASK3.2.md
- specs/RedshiftFoundation/TASK.md
- specs/RedshiftFoundation/TASK1.1.md
- specs/RedshiftFoundation/TASK1.2.md
- specs/RedshiftFoundation/TASK1.3.md
- specs/RedshiftFoundation/TASK1.4.md
- specs/RedshiftFoundation/TASK2.1.md
- specs/RedshiftFoundation/TASK2.2.md
- specs/RedshiftFoundation/TASK3.1.md
- specs/RedshiftFoundation/TASK3.2.md
- specs/RedshiftFoundation/TASK3.3.md
- specs/RedshiftSyntheticDataFeed/TASK.md
- specs/RedshiftSyntheticDataFeed/TASK1.1.md
- specs/RedshiftSyntheticDataFeed/TASK1.2.md
- specs/RedshiftSyntheticDataFeed/TASK1.3.md
- specs/RedshiftSyntheticDataFeed/TASK2.1.md
- specs/RedshiftSyntheticDataFeed/TASK2.2.md
- specs/RedshiftSyntheticDataFeed/TASK3.1.md
- specs/RedshiftSyntheticDataFeed/TASK3.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK1.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK1.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.3.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK3.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK3.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK4.1.md
TASK 3.2: Implement Current-State, Supersede, And Upstream-Batch Semantics
Summary
Implement the runtime behavior that decides which batch is current, how reruns supersede prior success, and how downstream stages resolve the exact upstream batch they consume.
This subtask makes tenant-day stage reruns safe and replayable without relying on newest-row inference.
Dependencies
- TASK 3.1 - stage batch and checkpoint contracts
Files to Create
1. Lifecycle transition services
Create or finalize the services that:
- start a stage batch
- complete a stage batch successfully
- fail a stage batch
- supersede the prior current batch on successful rerun
- resolve upstream batches by explicit id or by current-state lookup
Recommended logical services:
IngestionBatchLifecycleServiceStageUpstreamBatchResolver
Required Behavior
- only successful batches may become current
- failed or partial batches never become current
- rerun creates a new batch and links to the prior current successful batch
- a successful rerun clears
is_currenton the previously current successful batch for the same stage key - supersede behavior is stage-local and must not mutate unrelated stage state
- downstream stages must accept either an explicitly requested upstream batch id or resolve the current successful upstream batch automatically
- downstream stages must reject nonexistent, failed, or non-current upstream references unless an explicit replay contract later allows them
Testing
Unit tests
Add unit tests for:
- first successful batch becoming current
- successful rerun superseding the previous current batch
- failed rerun preserving the previous current batch
- upstream batch resolution by explicit id
- upstream batch resolution by current successful batch lookup
- rejection of invalid upstream batch references
Acceptance Criteria
- [x] Current-state transitions exist for successful stage completion
- [x] Successful rerun supersedes the previous current batch
- [x] Failed rerun does not clear the previous current batch
- [x] Upstream-batch resolution supports explicit and implicit selection paths
- [x] Unit tests cover current-state, supersede, and upstream selection behavior
Notes
- Keep current-state resolution explicit and deterministic.
- Do not let downstream stages infer upstream input from the newest created batch record.