# TASK 0: Freeze Fleet Import Contracts, Constants, Options, And Phase Details

## Implementation Status

**Completed: 2026-07-22**

- Added frozen Fleet constants, canonical source identity normalization, server-bound options, and startup validation under `backend/Kds.Ai.Integrations/FleetS3/`.
- Added immutable snapshot, Redshift replacement, Athena publication, orchestration, cleanup, durable details, and bounded diagnostic contracts without AWS SDK or Npgsql types.
- Added web-JSON details serialization with exact frozen snake_case phase values.
- Added focused TUnit coverage for constants, identity, option safety, all phase round-trips, bounded diagnostics, and forbidden sensitive/payload detail fields.
- Verification: `Kds.Ai.Integrations.Tests` passed 159 runnable tests; 11 opt-in live tests were skipped. The targeted warnings-as-errors build passed with zero warnings.
- Repository-wide warnings-as-errors build is currently blocked outside TASK 0 by existing NU1903 findings for `Microsoft.OpenApi` 2.0.0 in `Kds.Ai.ApiService` and its test project.

## Summary

Create the small, Fleet-specific contract layer used by storage, Redshift, publication, orchestration, and API work. Freeze names and invariants before parallel implementation begins.

## Dependencies

- [proposed-solution.md](./proposed-solution.md)
- [transaction_schema.json](./transaction_schema.json)
- [ORD-015D3A48C0.json](./ORD-015D3A48C0.json)

## Files To Create

Recommended logical locations:

- `backend/Kds.Ai.Integrations/FleetS3/FleetS3ImportConstants.cs`
- `backend/Kds.Ai.Integrations/FleetS3/Configuration/FleetS3ImportOptions.cs`
- `backend/Kds.Ai.Integrations/FleetS3/Configuration/FleetS3ImportOptionsValidator.cs`
- `backend/Kds.Ai.Integrations/FleetS3/Models/FleetS3ImportInput.cs`
- `backend/Kds.Ai.Integrations/FleetS3/Models/FleetS3ImportPhase.cs`
- `backend/Kds.Ai.Integrations/FleetS3/Models/FleetS3ImportDetails.cs`
- source snapshot request/result and manifest-index summary records used by Task 2;
- Redshift request, durable replacement-intent, result, and reconciliation records used by Task 3;
- publication request/result/journal records used by Task 4;
- focused bounded diagnostic records used by Task 5.

If S3 implementation options must live with the Warehouse S3 implementation, place the storage-only options under `Kds.Ai.Warehouse/Configuration` and expose them through a Fleet storage registration extension. Do not introduce circular project references.

## Fixed Constants

| Name | Value |
|---|---|
| source system | `fleet_s3` |
| source discriminator prefix | `fleet_s3:` |
| pipeline stage | `fleet_s3_direct_to_curated` |
| schema version | `1.0` |
| currency | `USD` |
| order target | `curated.pos_orders` |
| item target | `curated.pos_order_items` |
| default source bucket | `kds-warehouse-ai-test` |
| default source root | `fleet/` |
| ingestion mode | `FileUpload` |

Validate `sourceFolder` with `^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$` and preserve its exact case. The canonical value is the validated submitted value; do not lowercase it because S3 keys are case-sensitive. Use it consistently in S3 boundaries, `store_id`, terminal validation, operational source scope, and `_etl_source_type`.

## Required Options

Server-controlled options must cover:

- source bucket and fixed `fleet/` root;
- batch workspace root outside query-visible `curated/` data;
- bounded conditional-copy concurrency;
- maximum source object size, strictly below Redshift's 4 MiB JSON object limit;
- maximum validation sample count and maximum persisted error length;
- Redshift/import command timeout when the shared timeout is not sufficient;
- retained workspace duration after success/failure;
- publication retry lifetime, after which retained artifacts may be removed and a new full import is required;
- staged Athena export root outside query-visible `curated/` data.

Validation must reject:

- blank bucket/root/workspace values;
- a workspace under the customer source root;
- a staging workspace under the stable query-visible Fleet prefix;
- non-positive concurrency, timeout, size, sample, or retention values;
- source object limit greater than or equal to `4,194,304` bytes.

The caller must never override these options.

## Durable Details Contract

`FleetS3ImportDetails` is a versioned JSON document stored in `IngestionBatch.DetailsJson`. Use web JSON conventions and include a details version. It must remain small and contain summaries/URIs rather than one record per source object.

Minimum fields:

- details version and importer version;
- batch, customer, tenant, tenant database, requested-by identity;
- canonical source folder, source scope, override flag;
- current phase and phase timestamps;
- source prefix, snapshot/index URI, source object count/bytes;
- min/max business date and populated date count;
- candidate order/item counts and bounded validation summary;
- old/new affected date sets or an S3 URI when the set is externalized;
- durable replacement-intent ID/status, recorded before permanent Redshift mutation;
- deleted/inserted Redshift counts;
- `redshiftCommitted` and commit timestamp;
- per-date/entity publication journal or journal URI;
- publication completed flag;
- cleanup eligibility timestamp;
- bounded failure category/message and retryability.

Freeze these phases:

```text
reserved
snapshotting
snapshot_ready
loading_redshift
validating
replacing_redshift
redshift_committed
publishing_athena
completed
failed_pre_commit
failed_publication
publication_expired
superseded
```

## Fixed Athena Publication Contract

Task 0 must verify the deployed Glue/Athena table location for each entity before Tasks 2 through 4 begin. The fixed active object key is:

```text
curated/company_id={customerId}/tenant_id={tenantId}/entity={entity}/
dt={yyyy-MM-dd}/source_type=fleet_s3/source_id={canonicalSourceFolder}/data.parquet
```

The table's partition location must resolve to or above the corresponding `dt={yyyy-MM-dd}/` directory so Athena recursively scans the source descendants. If the deployed table does not do this, update the environment's Glue table location/projection as a release prerequisite. Do not make runtime key ordering configurable and do not defer the decision to Task 7.

## Public Service Contracts

Freeze narrowly scoped interfaces before implementations:

- source snapshot service: request -> immutable snapshot/index result;
- Redshift import executor: snapshot + warehouse + lineage -> replacement result;
- replacement-intent checkpoint: persist old/new/affected dates and expected counts before Redshift deletes or commit;
- Athena publisher: replacement result + lineage -> publication result;
- import orchestrator: execute/resume one reserved batch;
- workspace cleanup service.

Do not expose AWS SDK or Npgsql types through these contracts.

## Testing

Add unit tests for:

- source folder validation and normalization;
- exact-case folder behavior, including distinct `Front-Counter` and `front-counter` prefixes;
- exact source scope/discriminator generation and 100-character limit;
- options validation;
- details JSON round-trip for every phase;
- fixed Athena key and deployed table-location compatibility;
- bounded diagnostics behavior;
- no credentials, SQL, or full payload fields in details models.

## Acceptance Criteria

- all later tasks share one constants/options/details contract;
- source identity cannot drift between Redshift, S3, and operational metadata;
- options are server-controlled and fail at startup when unsafe;
- details are versioned and resumable without containing the complete source inventory;
- no generic ETL contract is changed only to accommodate Fleet.