# TASK 3: Introduce Pipeline Runner And ETL Source Adapter Contracts

## Summary

Introduce the core runtime contracts that separate pipeline mechanics from source-specific stage composition.

## Implementation Status

Status: implemented.

Verified in code:

- shared runtime contracts exist in `backend/Kds.Ai.Integrations/Etl/IEtlPipelineRunner.cs`, `IEtlSourceAdapter.cs`, and `IEtlStage.cs`,
- shared runner exists in `backend/Kds.Ai.Integrations/Etl/EtlPipelineRunner.cs`,
- stage context and result passing exist in `EtlStageContext.cs` and `EtlStageResult.cs`,
- `backend/Kds.Ai.ApiService/Jobs/BrinkPosEtlJobs.cs` now resolves the Brink adapter and invokes `IEtlPipelineRunner`,
- stage ordering and upstream-result flow are covered in `backend/tests/Kds.Ai.Integrations.Tests/Etl/EtlPipelineRunnerTests.cs`.

This task should add the minimal framework-facing contracts for:

- pipeline runner,
- ETL stage,
- ETL source adapter.

The goal is to replace hardcoded source orchestration with a generic pipeline runtime seam without fully rewriting all stage internals in the same step.

## Dependencies

- [TASK 0](./TASK0.md)
- [TASK 1](./TASK1.md)

## Files to Modify

### 1. Shared runtime contracts

- create or modify framework-oriented files under a backend location such as:
  - `backend/Kds.Ai.Etl/`
  - or `backend/Kds.Ai.Integrations.Abstractions/`

### 2. Orchestration entrypoints

- `backend/Kds.Ai.ApiService/Jobs/BrinkPosEtlJobs.cs`
- optionally related endpoint or dispatch files if orchestration wiring changes

### 3. Dependency injection registration

- DI modules or service registration files touched by ETL runtime composition

### 4. Tests

- add or extend tests under:
  - `backend/tests/Kds.Ai.ApiService.Tests/`
  - `backend/tests/Kds.Ai.Integrations.Tests/`

## Required Behavior

### Pipeline runner contract

Introduce a shared runtime contract capable of executing stages in order and passing upstream state between stages.

### ETL stage contract

Introduce a stage contract that makes stage identity explicit and allows stages to run under a shared pipeline runtime.

### ETL source adapter contract

Introduce a source adapter contract that owns source stage composition.

The adapter should expose the stages that make up its pipeline.

### Minimal migration rule

This task should create the framework seam first.

It does not need to fully migrate Brink stage internals yet, but it must make the new orchestration boundary real and usable.

## Proposed Contracts

Suggested minimal shared runtime contracts:

```csharp
public interface IEtlPipelineRunner
{
    Task<EtlPipelineResult> RunAsync(EtlPipelineRequest request, CancellationToken ct = default);
}

public interface IEtlSourceAdapter
{
    EtlSourceDescriptor Descriptor { get; }
    IReadOnlyList<IEtlStage> GetStages();
}

public interface IEtlStage
{
    string StageName { get; }
    Task<EtlStageResult> ExecuteAsync(EtlStageContext context, CancellationToken ct = default);
}

public sealed record EtlPipelineRequest(
    string CompanyId,
    string TenantId,
    DateOnly? BusinessDate,
    string Trigger,
    bool ForceReset,
    string? RequestedBy,
    IEtlSourceAdapter Adapter);

public sealed record EtlStageContext(
    EtlPipelineRequest Pipeline,
    IReadOnlyDictionary<string, EtlStageResult> CompletedStages);
```

The exact `EtlStageContext` shape may differ.
What matters is that a stage can read shared pipeline inputs and prior stage results without hardcoded method signatures between every pair of stages.

## Proposed Execution Shape

Expected pipeline runner flow:

1. receive a `PipelineRequest` with source adapter and trigger context,
2. iterate stages from `adapter.GetStages()` in declared order,
3. build a stage context that contains pipeline input and prior results,
4. execute stage,
5. store result by stage name,
6. return a final pipeline result with per-stage outputs.

## Brink Migration Shape For This Task

At the end of this task, `BrinkPosEtlJobs` should be able to do something conceptually equivalent to:

```csharp
await pipelineRunner.RunAsync(
    new EtlPipelineRequest(..., brinkPosSourceAdapter),
    ct);
```

The internals of the Brink stages may still delegate to current source-specific code until TASK 5 completes the migration.

## Acceptance Criteria

- shared contracts exist for pipeline runner, stage, and source adapter,
- the orchestration boundary no longer requires a permanently hardcoded four-call chain as the only design path,
- DI registration supports runtime composition through the new contracts,
- tests cover the basic pipeline contract and stage ordering behavior.

## Notes

- Keep this task focused on runtime shape and orchestration boundaries.
- Detailed source metadata and catalog contracts belong in TASK 4.
- Do not over-design a stage graph or DAG runtime. Ordered linear execution is enough for this slice.