Spensara Spensara Knowledge Base & Documentation Repository
System Overview Brink & Borg Master Index (722 Docs) Spensara Platform Project Tracker

Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)
Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Categories

Implementation Specs (157)

Detailed engineering and feature implementation specifications.
implementation-specs/specs/KdsCdcReplayExecution/TASK.md 8,301 bytes · Updated Oct 23, 2026 10:44 AM EDT
Raw File

KDS CDC Replay Execution - Implementation Tasks

Overview

Implementation of the KDS CDC Replay Execution slice.

This task plan is derived from:

  • ../../docs/features/ETL/kds-cdc/index.md
  • ../../docs/features/ETL/kds-cdc/implementation-breakdown.md
  • ../../docs/features/ETL/kds-cdc/branch-audit.md
  • ../KdsCdcIngestion/TASK8.md
  • ../KdsCdcIngestion/TASK9.md

Repository analysis for this spec also reviewed the current KDS CDC replay-related implementation in:

  • backend/Kds.Ai.Integrations/KdsCdc/IKdsCdcReplayService.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcReplayService.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcPipelineLauncher.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcStageJobs.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcExtractJobs.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcNormalizeLoadJobs.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcLoadToStagingJobs.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcStagingToCuratedJobs.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcCuratedToS3Jobs.cs
  • backend/Kds.Ai.Integrations/KdsCdc/KdsCdcCheckpointSupport.cs
  • backend/Kds.Ai.Data/Services/KdsCdcReplayIndexService.cs
  • backend/Kds.Ai.Data/Repositories/EfIngestionBatchRepository.cs
  • backend/Kds.Ai.ApiService/Jobs/BrinkPosEtlJobs.cs

The current branch already implements replay metadata persistence and replay-index lookup, but branch audit confirmed a concrete remaining gap: there is no verified replay execution flow that consumes durable replay lookups and re-runs downstream stages from raw, load, or curated state.

The target slice adds that missing execution layer.

Source Documents

  • ../../docs/features/ETL/kds-cdc/index.md
  • ../../docs/features/ETL/kds-cdc/implementation-breakdown.md
  • ../../docs/features/ETL/kds-cdc/branch-audit.md
  • ../KdsCdcIngestion/TASK8.md
  • ../KdsCdcIngestion/TASK9.md

Spec Name

This implementation spec should be referred to as KDS CDC Replay Execution.

Fixed Decisions

  • This slice closes the replay-execution gap identified after KDS CDC ingestion delivery.
  • Replay must run from durable state, not from SQS retention.
  • Existing replay-index persistence and lookup seams should be reused instead of replaced.
  • Existing KDS CDC stage job interfaces already provide the main execution seam for replay:
  • IKdsCdcNormalizeLoadJobs
  • IKdsCdcLoadToStagingJobs
  • IKdsCdcStagingToCuratedJobs
  • IKdsCdcCuratedToS3Jobs
  • Raw-batch replay may reuse the existing pipeline launch path because raw replay still starts at extract_raw-adjacent durable state.
  • Load retry and curated retry should prefer direct downstream stage execution from persisted UpstreamBatchId rather than redesigning the general ETL runner.
  • The generic EtlPipelineRunner should not be redesigned unless a concrete replay scenario cannot be expressed through existing KDS CDC stage-job seams.
  • Replay requests must be explicit and bounded. No implicit replay of arbitrary tenant history.
  • Replay execution must preserve current IngestionBatch lifecycle semantics, including IsCurrent, ResetOfBatchId, UpstreamBatchId, checkpoint advancement, and failed rerun behavior.
  • Replay must remain additive. Existing KDS CDC intake behavior, Brink POS behavior, and synthetic KDS behavior must remain unchanged.

Scope

This task plan includes:

  • defining the KDS CDC replay execution contract,
  • resolving replay entry requests into durable raw/load/curated starting points,
  • executing bounded replay from:
  • raw batch,
  • tenant/entity slice,
  • coordinate range,
  • load artifact retry,
  • curated retry,
  • exposing an operator-callable replay entry seam,
  • persisting replay-safe lineage and diagnostics through existing stage jobs,
  • adding deterministic tests for replay resolution and execution paths.

In short, this slice ends when KDS CDC replay is no longer only queryable metadata and becomes an executable bounded recovery workflow.

Out of Scope

The following items are explicitly excluded from this task plan:

  • redesigning KDS CDC intake or batching behavior,
  • changing KDS CDC source contracts,
  • replacing replay-index persistence with a different storage model,
  • building a full admin UI for replay control,
  • adding new non-KDS replay behavior to other ETL sources,
  • broad live-readiness coverage for KDS CDC replay beyond what is needed to prove the execution seam,
  • speculative framework-wide support for arbitrary mid-pipeline starts if KDS CDC-specific seams are sufficient.

Task Checklist

Phase 1: Replay Contract And Resolution

  • [ ] TASK 1 - Define bounded replay contracts and durable resolution rules

Sources:

  • ../../docs/features/ETL/kds-cdc/index.md#checkpointing-and-replay
  • ../../docs/features/ETL/kds-cdc/index.md#batch-lifecycle-and-lineage
  • ../../docs/features/ETL/kds-cdc/branch-audit.md
  • ../KdsCdcIngestion/TASK8.md

Phase 2: Replay Execution Engine

  • [ ] TASK 2 - Implement replay coordinator and downstream stage execution paths

Sources:

  • ../../docs/features/ETL/kds-cdc/index.md#checkpointing-and-replay
  • ../../docs/features/ETL/kds-cdc/index.md#supported-corner-cases
  • ../KdsCdcIngestion/TASK8.md

Phase 3: Replay Entry Points And Diagnostics

  • [ ] TASK 3 - Expose replay launch seams for operators and internal jobs

Sources:

  • ../../docs/features/ETL/kds-cdc/index.md#checkpointing-and-replay
  • ../../docs/features/ETL/kds-cdc/index.md#quarantine-and-failed-packages
  • ../../docs/features/ETL/kds-cdc/branch-audit.md

Phase 4: Verification And Guardrails

  • [ ] TASK 4 - Add replay execution coverage and regression guardrails

Sources:

  • ../../docs/features/ETL/kds-cdc/index.md#supported-corner-cases
  • ../../docs/features/ETL/kds-cdc/index.md#non-goals
  • ../KdsCdcIngestion/TASK9.md

Dependency Graph

text
Phase 1
    +-- TASK 1 freezes replay request/target model and durable selection rules

Phase 2
    +-- TASK 2 replay coordinator and execution engine <-- depends on TASK 1

Phase 3
    +-- TASK 3 replay entrypoint exposure and diagnostics <-- depends on TASK 1, TASK 2

Phase 4
    +-- TASK 4 replay validation and regressions <-- depends on TASK 1, TASK 2, TASK 3
SessionTasksDescription
11Freeze replay request model, supported replay levels, and durable lookup rules
22Implement replay coordinator and connect raw/load/curated execution paths
33Add internal/operator entry seam and diagnostics surface
44Add deterministic coverage, regression guards, and remaining readiness checks

Key Design Decisions

Replay Should Use Existing Stage Job Seams

The KDS CDC code already has direct stage-job interfaces that accept UpstreamBatchId. That is the most natural replay seam for load retry and curated retry. It avoids unnecessary redesign of the generic pipeline runner.

Raw Replay And Downstream Retry Are Different Problems

Raw replay begins from durable raw state and may still resemble pipeline-style execution. Load retry and curated retry begin from already completed upstream durable stages and should re-enter lower stages directly.

Bounded Replay Only

The feature docs define replay as a bounded recovery operation. This spec does not introduce open-ended tenant backfills or catch-up jobs.

Resolution Must Be Durable And Operator-Facing

The system already stores enough replay-index facts to locate candidate batches. This slice must convert those facts into a deterministic execution target, including clear failure messages when the request is ambiguous or impossible.

Preserve Existing Lifecycle Semantics

Replay should not invent a second lineage model. It should create ordinary IngestionBatch reruns and rely on current lifecycle rules for Succeeded, Failed, IsCurrent, ResetOfBatchId, and upstream chaining.

Start With KDS CDC-Specific Entry Seams

If replay can be expressed through a KDS CDC-specific coordinator and API/job entry seam, prefer that over broad ETL-framework generalization. Generalization should happen only if concrete duplication appears.