# TASK 3: Add Landing RAW And Durable Promotion State

## Summary

Move valid CDC durability ahead of tenant resolution and add durable landing-to-tenant promotion state so unresolved children no longer depend on SQS receipt visibility for eventual success.

## Current Codebase Findings

Repository analysis shows:

- current branch still mostly writes raw data after tenant resolution succeeds,
- unresolved child rows remain in SQS retry/visibility loop,
- worker acknowledgment is still coupled too tightly to resolved handoff behavior,
- branch docs already identify landing RAW plus tenant RAW promotion as target architecture.

## Dependencies

- TASK 1
- TASK 2

## Source References

- `../KdsCdcDebeziumIntegration/TASK3.md`
- `../../docs/features/ETL/kds-cdc/branch-direction-and-flow.md#7-raw-capture-and-handoff`
- `../../docs/features/ETL/kds-cdc/debezium-payload-compatibility-notes.md`

## Scope From Breakdown

This task covers:

- landing RAW write before tenant promotion,
- durable promotion-state persistence,
- ack behavior tied to landing/quarantine durability,
- idempotent promotion from landing RAW to tenant RAW.

## Files to Create

- landing/promotion contracts under `backend/Kds.Ai.Integrations/KdsCdc/` or `.../Landing/`
- promotion-state entities/configuration/repositories under `backend/Kds.Ai.Data/`
- landing RAW metadata contracts if new raw record shape or object family is needed

## Files to Modify

- `backend/Kds.Ai.ApiService/Jobs/KdsCdc/KdsCdcWorkerService.cs`
- `backend/Kds.Ai.Integrations/KdsCdc/KdsCdcExtractJobs.cs`
- `backend/Kds.Ai.Integrations/KdsCdc/Raw/KdsCdcRawArtifactWriter.cs`
- `backend/Kds.Ai.Integrations/KdsCdc/KdsCdcIntakeSupport.cs`
- any checkpoint/details models affected by landing/promotion state
- tests under `backend/tests/Kds.Ai.Integrations.Tests/KdsCdc/` and `backend/tests/Kds.Ai.Data.Tests/`

## Required Behavior

### Landing RAW first

For valid supported Debezium payloads:

1. parse and validate message
2. write landing RAW durably
3. write promotion-state row durably
4. acknowledge SQS
5. resolve tenant and promote from durable state

### Promotion-state durability

Promotion state must record enough information to continue without SQS.

Minimum fields:

- stable Debezium identity
- landing RAW object key
- source database/table
- operation
- primary key
- source coordinates
- comparable ordering token when available
- current promotion status
- retry reason
- retry count
- last attempt timestamp
- resolved tenant/store/company when known
- tenant RAW object key when promoted

### Promotion status model

Need explicit statuses at minimum:

- `pending`
- `tenant_unresolved`
- `promoted`
- `quarantined`
- `failed`

### Ack behavior

Required ack behavior:

- landing RAW write fails: do not acknowledge SQS
- quarantine durability fails: do not acknowledge SQS
- landing RAW and promotion-state write succeed: acknowledge SQS even if tenant still unresolved
- tenant promotion transient failure after durable landing: do not rely on SQS for retry

### Promotion idempotency

If same Debezium message is received again after landing was already persisted:

- do not create duplicate logical promotion state,
- do not create duplicate tenant RAW logical event,
- converge on existing landing/promotion record by stable Debezium identity.

## Testing

Add tests for:

- landing RAW written before tenant promotion,
- acknowledged after landing success for unresolved child,
- no acknowledge when landing write fails,
- no acknowledge when durable quarantine write fails,
- duplicate redelivery converges on same promotion state,
- promotion creates tenant RAW only once,
- unsupported or malformed payload still uses durable quarantine path.

## Acceptance Criteria

- [ ] Landing RAW becomes first durable boundary for valid messages
- [ ] SQS ack depends on landing/quarantine durability, not immediate tenant resolution success
- [ ] Promotion-state persistence exists and is durable
- [ ] Unresolved child rows leave SQS hot path after landing success
- [ ] Promotion is idempotent across SQS redelivery
- [ ] Tenant RAW remains explicit downstream boundary
- [ ] Tests cover landing-first, ack behavior, and idempotent promotion semantics

## Notes

- Do not redesign downstream ETL stages after tenant RAW here.
- Keep unsupported/malformed payload quarantine behavior explicit and durable.