# TASK 0 - Module 0 Decision Record

## Purpose

Freeze the true v1 scope for the first Brink POS ETL job before implementation work starts in dispatch, extraction, normalization, Redshift load, and curated mapping.

This record overrides the broader and partly outdated Brink POS groundwork documents for this specific implementation slice.

## Decision Summary

The first Brink POS ETL job is strictly limited to:

- `orders`
- `order_items`

The job must run daily and therefore always supports business-date resolution through:

- `Sales2.GetCurrentBusinessDate`
- `Sales2.GetOrders`

The first implemented end-to-end path is:

1. resolve business date
2. extract raw XML
3. persist raw XML to S3 `raw/`
4. normalize to Parquet in S3 `load/`
5. load Parquet to Redshift `staging`
6. map `staging -> curated`
7. expose correct data in:
   - `curated.pos_orders`
   - `curated.pos_order_items`

Testing is a mandatory part of every module:

- every module must include unit tests
- every completed ETL stage should include live E2E validation whenever the stage can be verified against real integrations
- Aspire secrets are the standard way to access AWS services and Brink POS API during live verification

## In Scope

- one daily tenant-day ETL job for Brink POS
- `Sales2.GetCurrentBusinessDate` as required business-date helper
- `Sales2.GetOrders` as the main transactional source
- full raw XML preservation in S3
- Parquet `load/` artifacts derived from `GetOrders`
- Redshift staging load from Parquet
- Redshift curated mapping for:
  - `curated.pos_orders`
  - `curated.pos_order_items`
- stage-level metadata, checkpoints, rerun semantics, and traceability
- unit tests for every module
- live E2E verification for each completed ETL stage whenever practical through Aspire-managed secrets and integrations

## Out Of Scope

- every Brink endpoint other than `GetCurrentBusinessDate` and `GetOrders`
- kitchen contracts and kitchen-oriented curated outputs
- item, destination, tender, tax, employee, modifier, or table enrichment endpoints
- widening the first slice to any additional business contract beyond `orders` and `order_items`
- treating old Brink groundwork docs as binding scope for this implementation

## Final Curated Targets

The curated target names for v1 are already fixed and must not be renamed in this slice:

- `curated.pos_orders`
- `curated.pos_order_items`

Source of truth:

- `backend/Kds.Ai.Warehouse/Sql/Bootstrap/Tenant/Curated/0612_create_curated_pos_orders.sql`
- `backend/Kds.Ai.Warehouse/Sql/Bootstrap/Tenant/Curated/0611_create_curated_pos_order_items.sql`

## Source Of Truth For V1 Contracts

For this slice, the source of truth is:

- the existing curated bootstrap SQL for `pos_orders` and `pos_order_items`
- the existing staging models under `backend/integrations/Kds.Ai.Integrations.BrinkPos/Models/Staging/Sales2/GetOrders`
- the curated model docs and Brink mapping docs used as semantic reference

The broader documents under:

- `docs/features/ETL/epics/06-source-adapter-brink-pos-api/01-first-job-foundation/`

are treated only as supporting material. They may remain broader than this actual implementation scope.

## Staging And Load Interpretation

For this slice, the pipeline layers are interpreted as follows:

- S3 `raw/` stores the original XML payload from Brink
- S3 `load/` stores processed Parquet contracts
- Redshift `staging` stores data loaded from those Parquet contracts
- Redshift `curated` stores business-facing mappings built from Redshift `staging`

Important rule:

- the existing `GetOrders` staging model is reused
- Module 0 does not redesign staging
- later implementation work must only verify completeness and correctness of the existing staging model against:
  - Brink portal documentation
  - `temp/GetOrders_WithKnownBusinessDate_CallsRealBrinkEndpoint_2024-10-06.xml`

## Field Coverage Decision

For Module 0, field-scope decisions are frozen as follows:

- the job targets the full existing curated contracts for `curated.pos_orders` and `curated.pos_order_items`
- the job uses the full existing `GetOrders` staging contract already modeled in code
- fields that cannot be populated correctly from `GetOrders` plus `GetCurrentBusinessDate` alone are allowed to be `NULL` in v1
- missing enrichment from other Brink endpoints is not a blocker for the first slice

This means:

- no extra endpoint is added just to backfill one field
- nullable output is preferred over hidden scope expansion
- any later enrichment must be added as a separate, atomic scope extension

## Extensibility Rule

The first slice must be easy to extend.

Future addition of a new Brink contract should be:

- fast
- isolated
- atomic
- non-breaking for the existing `orders` and `order_items` flow

This implies:

- orchestration is stage-based and contract-based
- adding a new endpoint later must not require redesign of the current `GetOrders` path
- adding enrichment later must be possible without invalidating the current v1 delivery

## Testing Rule

Testing is part of the delivery contract, not a later hardening task.

### Unit tests

- every module must include unit tests
- unit tests should verify module-local rules, mappings, validation, orchestration branching, and failure semantics

### Live E2E tests

- every module should include live E2E coverage when the module represents a completed ETL step that can be validated against real services
- live E2E tests should use Aspire-managed secrets and environment wiring to reach:
  - Brink POS API
  - AWS services such as S3
  - later Redshift when the stage requires it

### Examples

- `extract_raw` should have a live E2E test that calls real Brink POS endpoints and verifies that raw XML is persisted to S3
- `normalize_load` should have a live E2E test that starts from a real raw artifact and verifies written Parquet output
- `load_to_staging` should have a live E2E test that verifies Redshift staging load from produced Parquet
- `staging_to_curated` should have a live E2E test that verifies curated rows after SQL mapping

### Practical rule

- if a module ends with a clearly verifiable ETL boundary, that module should not be treated as complete without E2E validation unless there is a concrete technical blocker
- if a live E2E test is temporarily not possible, the blocker must be documented explicitly in the module task

## Success Criteria For Module 0

Module 0 is complete when the team treats the following as fixed:

- the first Brink ETL slice is only `orders` and `order_items`
- `GetCurrentBusinessDate` is required for daily execution support
- `GetOrders` is the main transactional source
- curated targets are fixed as:
  - `curated.pos_orders`
  - `curated.pos_order_items`
- the existing `GetOrders` staging model is reused and later only verified, not redesigned
- fields missing non-`GetOrders` enrichment are allowed to remain `NULL` in v1
- older groundwork docs are non-authoritative when they widen the scope beyond this decision record
- every module requires unit tests, and completed ETL stages require live E2E verification whenever feasible