Built Word Documents (.docx):
Spensara Overview (.docx)
Capability Assessment (.docx)
Brink POS Ingestion Analysis (.docx)
The Borg Ingestion Architecture (.docx)
Categories
Implementation Specs (157)
Detailed engineering and feature implementation specifications.
- specs/AnomalyDetection/TASK1.1.md
- specs/AnomalyDetection/TASK1.2.md
- specs/AnomalyDetection/TASK1.3.md
- specs/AnomalyDetection/TASK1.4.md
- specs/AnomalyDetection/TASK2.1.md
- specs/AnomalyDetection/TASK2.2.md
- specs/AnomalyDetection/TASK3.1.md
- specs/AnomalyDetection/TASK3.2.md
- specs/AnomalyDetection/TASK3.3.md
- specs/AnomalyDetection/TASK4.1.md
- specs/AnomalyDetection/TASK4.2.md
- specs/AnomalyDetection/TASK4.3.md
- specs/AnomalyDetection/TASK4.4.md
- specs/AnomalyDetection/TASK5.1.md
- specs/AnomalyDetection/TASK5.2.md
- specs/AnomalyDetection/TASK5.3.md
- specs/AnomalyDetection/TASK6.1.md
- specs/AnomalyDetection/TASK6.2.md
- specs/AnomalyDetection/TASK6.3.md
- specs/AnomalyDetection/TASKS.md
- specs/ApiServicePostgresTestStability/TASK.md
- specs/BackendDatabaseConsolidation/TASK.md
- specs/BackendDatabaseConsolidation/TASK1.1.md
- specs/BackendDatabaseConsolidation/TASK1.2.md
- specs/BackendDatabaseConsolidation/TASK1.3.md
- specs/BackendDatabaseConsolidation/TASK2.1.md
- specs/BackendDatabaseConsolidation/TASK2.2.md
- specs/BackendDatabaseConsolidation/TASK2.3.md
- specs/BackendDatabaseConsolidation/TASK3.1.md
- specs/BackendDatabaseConsolidation/TASK3.2.md
- specs/BackendDatabaseConsolidation/TASK4.1.md
- specs/BackendDatabaseConsolidation/TASK4.2.md
- specs/BackendDatabaseConsolidation/TASK4.3.md
- specs/BackendDatabaseConsolidation/TASK5.1.md
- specs/BackendDatabaseConsolidation/TASK5.2.md
- specs/BrinkLaborGetShiftsStaging/TASK.md
- specs/BrinkPosOrdersFirstEtlJob/TASK.md
- specs/BrinkPosOrdersFirstEtlJob/TASK0.md
- specs/BrinkPosOrdersFirstEtlJob/TASK1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.md
- specs/BrinkPosOrdersFirstEtlJob/TASK7.md
- specs/BrinkPosOrdersFirstEtlJob/TASK8.md
- specs/BrinkPosOrdersFirstEtlJob/TASK9.md
- specs/BrinkSettings2ToStaging/TASK.MD
- specs/CuratedToS3Stage/TASK.md
- specs/CuratedToS3Stage/TASK0.md
- specs/CuratedToS3Stage/TASK1.md
- specs/CuratedToS3Stage/TASK2.md
- specs/CuratedToS3Stage/TASK3.md
- specs/CuratedToS3Stage/TASK4.md
- specs/CustomerCredentialCustomHeaders/TASK.md
- specs/CustomerCredentialCustomHeaders/TASK1.1.md
- specs/CustomerCredentialCustomHeaders/TASK1.2.md
- specs/CustomerCredentialCustomHeaders/TASK2.1.md
- specs/CustomerCredentialCustomHeaders/TASK2.2.md
- specs/CustomerCredentialCustomHeaders/TASK3.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK.md
- specs/CustomerCredentialTriggeredProvisioning/TASK1.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK1.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK2.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK2.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK3.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK3.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK4.1.md
- specs/EtlSourceAdapterFramework/TASK.md
- specs/EtlSourceAdapterFramework/TASK0.md
- specs/EtlSourceAdapterFramework/TASK1.md
- specs/EtlSourceAdapterFramework/TASK2.md
- specs/EtlSourceAdapterFramework/TASK3.md
- specs/EtlSourceAdapterFramework/TASK4.md
- specs/EtlSourceAdapterFramework/TASK5.md
- specs/EtlSourceAdapterFramework/TASK6.md
- specs/KdsCdcDebeziumIntegration/TASK.md
- specs/KdsCdcDebeziumIntegration/TASK1.md
- specs/KdsCdcDebeziumIntegration/TASK2.md
- specs/KdsCdcDebeziumIntegration/TASK3.md
- specs/KdsCdcDebeziumIntegration/TASK4.md
- specs/KdsCdcDebeziumIntegration/TASK5.md
- specs/KdsCdcDurableRelationRouting/TASK.md
- specs/KdsCdcDurableRelationRouting/TASK1.md
- specs/KdsCdcDurableRelationRouting/TASK2.md
- specs/KdsCdcDurableRelationRouting/TASK3.md
- specs/KdsCdcDurableRelationRouting/TASK4.md
- specs/KdsCdcDurableRelationRouting/TASK5.md
- specs/KdsCdcIngestion/TASK.md
- specs/KdsCdcIngestion/TASK1.md
- specs/KdsCdcIngestion/TASK2.md
- specs/KdsCdcIngestion/TASK3.md
- specs/KdsCdcIngestion/TASK4.md
- specs/KdsCdcIngestion/TASK5.md
- specs/KdsCdcIngestion/TASK6.md
- specs/KdsCdcIngestion/TASK7.md
- specs/KdsCdcIngestion/TASK8.md
- specs/KdsCdcIngestion/TASK9.md
- specs/KdsCdcReplayExecution/TASK.md
- specs/KdsCdcReplayExecution/TASK1.md
- specs/KdsCdcReplayExecution/TASK2.md
- specs/KdsCdcReplayExecution/TASK3.md
- specs/KdsCdcReplayExecution/TASK4.md
- specs/KdsMonitorServingContractMigration/TASK.md
- specs/KdsMonitorServingContractMigration/TASK1.1.md
- specs/KdsMonitorServingContractMigration/TASK1.2.md
- specs/KdsMonitorServingContractMigration/TASK2.1.md
- specs/KdsMonitorServingContractMigration/TASK2.2.md
- specs/KdsMonitorServingContractMigration/TASK3.1.md
- specs/KdsMonitorServingContractMigration/TASK3.2.md
- specs/RedshiftFoundation/TASK.md
- specs/RedshiftFoundation/TASK1.1.md
- specs/RedshiftFoundation/TASK1.2.md
- specs/RedshiftFoundation/TASK1.3.md
- specs/RedshiftFoundation/TASK1.4.md
- specs/RedshiftFoundation/TASK2.1.md
- specs/RedshiftFoundation/TASK2.2.md
- specs/RedshiftFoundation/TASK3.1.md
- specs/RedshiftFoundation/TASK3.2.md
- specs/RedshiftFoundation/TASK3.3.md
- specs/RedshiftSyntheticDataFeed/TASK.md
- specs/RedshiftSyntheticDataFeed/TASK1.1.md
- specs/RedshiftSyntheticDataFeed/TASK1.2.md
- specs/RedshiftSyntheticDataFeed/TASK1.3.md
- specs/RedshiftSyntheticDataFeed/TASK2.1.md
- specs/RedshiftSyntheticDataFeed/TASK2.2.md
- specs/RedshiftSyntheticDataFeed/TASK3.1.md
- specs/RedshiftSyntheticDataFeed/TASK3.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK1.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK1.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.3.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK3.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK3.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK4.1.md
TASK 1: Define Bounded Replay Contracts And Durable Resolution Rules
Summary
Define the executable replay model for KDS CDC. This task freezes supported replay levels, request contracts, replay-target resolution rules, and the durable metadata each execution path requires before any operator-visible replay launcher is built.
Current Codebase Findings
IKdsCdcReplayServicealready supports replay lookup by:- batch id,
- tenant/entity/stage,
- coordinate range,
- artifact URI.
KdsCdcReplayServicecurrently only forwards those queries toIKdsCdcReplayIndexService.KdsCdcCheckpointSupport.CreateReplayIndexRequest(...)already persists enough facts to locate batches by stage, coordinate range, and primary artifact URI.KdsCdcStageJobInputalready gives downstream stages a clean replay-ready execution seam throughUpstreamBatchId.- Branch audit confirmed no current service consumes replay lookup results to launch replay.
Current repo constraints this task must respect:
- Replay selection must be deterministic even when multiple candidate batches exist.
- Replay after acknowledgement must depend on durable raw/load/curated artifacts, not transport state.
- Current replay-index rows do not themselves execute work; this task must define the mapping from lookup result to execution target.
Dependencies
../../docs/features/ETL/kds-cdc/index.md../../docs/features/ETL/kds-cdc/branch-audit.md../KdsCdcIngestion/TASK8.md
Source References
../../docs/features/ETL/kds-cdc/index.md#checkpointing-and-replay../../docs/features/ETL/kds-cdc/index.md#batch-lifecycle-and-lineage../../docs/features/ETL/kds-cdc/index.md#supported-corner-cases../../docs/features/ETL/kds-cdc/branch-audit.md
Files to Create
- Replay contracts under
backend/Kds.Ai.Integrations/KdsCdc/Replay/orbackend/Kds.Ai.Integrations/KdsCdc/Models/: KdsCdcReplayRequestKdsCdcReplayModeKdsCdcReplayTargetKdsCdcReplayExecutionPlanKdsCdcReplayResolutionFailure
Responsibilities:
- describe supported replay levels,
- capture explicit bounded request inputs,
- describe resolved durable starting point and downstream stages to execute.
- Replay resolution seam:
IKdsCdcReplayPlannerKdsCdcReplayPlanner
Responsibilities:
- resolve user/internal replay requests into durable execution plans,
- validate ambiguity and unsupported requests before any stage work starts.
Files to Modify
backend/Kds.Ai.Integrations/KdsCdc/IKdsCdcReplayService.csbackend/Kds.Ai.Integrations/KdsCdc/KdsCdcReplayService.cs
Guidance:
- keep the current lookup API if still useful,
- add planning/execution seams instead of overloading plain lookup methods with side effects.
Required Behavior
Supported Replay Levels
The replay model must explicitly support:
- raw batch replay
- tenant/entity replay
- coordinate range replay
- load artifact retry
- curated retry
Each mode must define:
- required inputs,
- durable starting artifact or upstream batch,
- first stage to execute,
- downstream stop point when bounded.
Deterministic Resolution Rules
- Raw batch replay resolves from an explicit stage batch id or raw artifact pointer.
- Load retry resolves from an explicit normalized/load batch or load artifact pointer.
- Curated retry resolves from an explicit staging-to-curated or curated batch.
- Tenant/entity replay resolves one bounded set of raw batches for a named tenant/entity and requested replay window or explicit bound.
- Coordinate replay resolves all relevant raw-stage candidates whose coordinate ranges overlap the requested range.
Ambiguity Handling
- Replay planning must fail clearly when a request resolves to:
- zero durable candidates,
- multiple candidates where one exact target was required,
- a candidate whose required upstream durable artifact is missing,
- a candidate whose stage lineage is broken or inconsistent.
Replay Execution Plan Shape
The resolved plan should state at minimum:
- replay mode,
- company id,
- tenant id,
- entity if applicable,
- starting stage,
- terminal stage,
- upstream durable batch ids or artifact URIs,
- whether execution is single-batch or multi-batch,
- human-readable reason/context for diagnostics.
Testing
- Unit tests for replay request validation.
- Planner tests for:
- raw batch exact resolution,
- load retry exact resolution,
- curated retry exact resolution,
- tenant/entity resolution,
- coordinate-range overlap resolution,
- ambiguity failure,
- missing durable artifact failure,
- lineage break failure.
Acceptance Criteria
- Supported replay modes are explicitly modeled in code.
- Replay requests resolve to deterministic execution plans before any stage job runs.
- Ambiguous or impossible requests fail with clear diagnostics.
- The codebase has a first-class replay planning seam rather than only replay-index lookup helpers.
Notes
- Prefer explicit request contracts over loosely shaped dictionaries or untyped context objects.
- Preserve
IKdsCdcReplayServicelookup behavior if other tests or tooling already depend on it.