Spensara Spensara Knowledge Base & Documentation Repository
System Overview Brink & Borg Master Index (722 Docs) Spensara Platform Project Tracker

Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)
Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Categories

Implementation Specs (157)

Detailed engineering and feature implementation specifications.
implementation-specs/specs/EtlSourceAdapterFramework/TASK6.md 4,877 bytes · Updated Oct 23, 2026 10:44 AM EDT
Raw File

TASK 6: Add Regression Coverage And Reserve The Future Publish Stage Boundary

Summary

Finalize the slice with regression coverage and an explicit reserved boundary for the future curated export stage.

Implementation Status

Status: implemented, with TASK 4 gap still open.

Verified in code:

  • regression coverage exists for stage support, curated execution, pipeline runner behavior, Brink catalog consistency, and manual Brink orchestration wiring,
  • reserved future-stage boundary exists in backend/Kds.Ai.Integrations/Etl/EtlStageNames.cs with curated_to_s3,
  • the current runner and adapter model can accept another ordered stage without redesign.

Primary evidence:

  • backend/tests/Kds.Ai.Integrations.Tests/Etl/StageExecutionSupportTests.cs
  • backend/tests/Kds.Ai.Integrations.Tests/Etl/CuratedTargetExecutorTests.cs
  • backend/tests/Kds.Ai.Integrations.Tests/Etl/EtlPipelineRunnerTests.cs
  • backend/tests/Kds.Ai.Integrations.Tests/BrinkPos/BrinkPosEtlCatalogTests.cs
  • backend/tests/Kds.Ai.ApiService.Tests/Kds.Ai.ApiService.Tests/Endpoints/BrinkPosManualEtlEndpointTests.cs

The ETL architecture in docs/ETL.md already expects curated export to S3 for Athena and cross-tenant use. That stage does not need to be fully implemented in this slice, but the framework should leave a clear place for it.

Dependencies

Files to Modify

1. Framework runtime and contracts

  • framework-oriented files introduced by TASK 1 through TASK 5

2. Brink adapter and orchestration tests

  • test files under:
  • backend/tests/Kds.Ai.Integrations.Tests/BrinkPos/
  • backend/tests/Kds.Ai.ApiService.Tests/
  • and other backend test projects where the framework seams are exercised

3. Documentation or contract files if needed

  • specs/EtlSourceAdapterFramework/TASK.md
  • optionally docs/features/ETL/etl-source-adapter-framework.md if implementation tightens the reserved stage naming

Required Behavior

Regression coverage

Add or extend tests so the extracted framework seams are covered for at least:

  • stage failure and checkpoint behavior,
  • curated transaction execution behavior,
  • pipeline stage ordering,
  • Brink adapter registration and orchestration.

Reserved publish-stage boundary

The framework should explicitly support the addition of a future stage such as:

  • curated_to_s3,
  • or publish_curated.

This task does not need to implement export behavior. It only needs to ensure the runtime model has a clean place for such a stage.

No placeholder-only abstraction

Do not add empty fake runtime layers.

If a reserved stage boundary is represented in code, it must be a real extension seam, not dead scaffolding.

Proposed Contracts

Suggested minimal forward-compatible shape:

csharp
public static class EtlStageNames
{
    public const string ExtractRaw = "extract_raw";
    public const string NormalizeLoad = "normalize_load";
    public const string LoadToStaging = "load_to_staging";
    public const string StagingToCurated = "staging_to_curated";
    public const string CuratedToS3 = "curated_to_s3";
}

If a dedicated future publish-stage contract is needed, keep it minimal, for example:

csharp
public interface IPublishCuratedStage : IEtlStage
{
}

This should be introduced only if code already benefits from the distinction.

Proposed Regression Matrix

At minimum, this task should leave tests for:

  1. stage support helper:
  2. checkpoint advances with supplied source descriptor,
  3. failure path writes a failed package and fails batch,
  4. curated target executor:
  5. success path commits all declared targets,
  6. failure path rolls back execution unit,
  7. pipeline runner:
  8. stages execute in adapter-declared order,
  9. previous stage result is visible to later stage,
  10. Brink migration:
  11. Brink adapter registers the active four stages only,
  12. scheduler/manual orchestration resolves Brink through the framework seam.

Proposed Forward-Compatible Shape

The runtime model should be considered sufficient if adding a future publish stage only requires:

  1. creating one new stage implementation,
  2. appending it to the adapter stage list,
  3. adding stage-specific tests.

It should not require redesigning the runner or replacing prior framework contracts.

Acceptance Criteria

  • regression tests cover the main extracted framework seams,
  • the framework has a clear and minimal extension point for a future publish stage,
  • no speculative non-functional framework layers are added,
  • the migrated Brink path remains verified through the new tests.

Notes

  • Keep this task focused on validation and forward-compatible runtime shape.
  • Do not implement Athena, Glue, or S3 curated export here.
  • Prefer extension by ordered stage registration over adding special-case branching inside the runner.