TASK 2: Extract Raw XML

Summary

Implement the first real ETL data boundary for the Brink POS orders flow: live extraction of raw SOAP XML and immutable persistence to S3 raw/.

This task starts from the tenant-day child job contract defined in TASK1.md and turns it into a real extract_raw execution step for:

• Sales2.GetCurrentBusinessDate
• Sales2.GetOrders

This task owns:

• child extract orchestration for one tenant-day run
• business-date resolution
• Warehouse-side binding resolution and Brink extract boundary usage
• streaming raw capture to S3
• persisted-raw validation
• extract-stage diagnostics

This task does not implement raw -> load, Redshift COPY, curated SQL, or curated export yet.

This task must preserve the architectural boundary introduced by the source-binding refactor:

• Warehouse/app-layer resolves tenant context, loads the active source binding, validates it, and builds BrinkConnectionDefinition
• Kds.Ai.Integrations.BrinkPos receives that connection context from the outside and only performs Brink protocol work
• ETL batch semantics, S3 ownership, and replay-boundary semantics stay outside the Brink integration module

Dependencies

• TASK 0 - Module 0 decision record
• TASK 1 - source binding and dispatch

Subtasks

• [x] TASK 2.1 - Implement child extract orchestration and business-date resolution
• [x] TASK 2.2 - Implement Brink raw capture boundary for GetOrders
• [x] TASK 2.3 - Implement S3 raw persistence and persisted-raw validation
• [x] TASK 2.4 - Add unit tests and live E2E validation for extract_raw

Files to Create

1. Extract job orchestration contracts and services

Create or finalize the Warehouse/app-layer services that execute one tenant-day extract_raw run.

Required logical responsibilities:

• child extract job entry
• resolve active Brink binding for the tenant-day run through Warehouse/app-layer source-binding services
• business-date resolver
• extract orchestrator
• endpoint catalog for the narrowed v1 scope
• endpoint executor

Recommended logical services:

• BrinkPosExtractJobs
• BrinkPosBusinessDateResolver
• BrinkPosExtractOrchestrator
• BrinkPosEndpointCatalog
• BrinkPosEndpointExecutor

2. Raw capture and validation seam

Create or finalize the seam between Brink raw extraction and generic ETL storage.

Required logical responsibilities:

• capture response stream through IBrinkRawClient using externally provided BrinkConnectionDefinition
• persist original XML through shared ETL S3 storage
• reopen persisted object from storage
• validate XML, SOAP fault state, and Brink business success using Brink-specific validation components without moving S3 or batch semantics into Kds.Ai.Integrations.BrinkPos

3. Extract-stage result and diagnostics models

Create or finalize the minimum result/diagnostics shape needed for one extract_raw endpoint attempt and one full tenant-day extract batch.

These models belong to ETL/Warehouse-facing code, not to Kds.Ai.Integrations.BrinkPos.

Required minimum details:

• company_id
• tenant_id
• resolved business_date
• pipeline_stage = extract_raw
• stream_name
• service_name
• operation_name
• raw_s3_path
• started_at
• completed_at
• endpoint status and validation outcome

Files to Modify

4. Brink integration boundary

Modify the Brink integration so the ETL path uses:

• IBrinkPosFacade only for GetCurrentBusinessDate
• IBrinkRawClient for GetOrders raw capture
• externally provided BrinkConnectionDefinition or a factory-created bound Brink client for every ETL call path

This task should not keep a parallel typed extract path for GetOrders in the ETL flow.

This task must not add or keep the following inside Kds.Ai.Integrations.BrinkPos for the ETL path:

• tenant resolution
• source-binding lookup from operational DB
• ISourceConnectionRepository
• IRuntimeTenantResolver
• S3 raw persistence ownership
• ETL batch diagnostics models

5. S3 raw storage integration

Modify the shared ETL storage path so raw capture writes immutable XML into the standard S3 raw/ layout from docs/ETL.md.

6. Batch metadata touch points

Use the existing ingestion-batch foundation where possible, but this task may need to extend or adapt the extract-stage metadata written during execution.

Any remaining gaps in batch/checkpoint richness should be carried forward into TASK3.md rather than blocking raw extraction design here.

Required Behavior

Scope rule

This task is strict v1 extract_raw only for:

• GetCurrentBusinessDate as required helper
• GetOrders as the only replayable extract endpoint

No other Brink endpoint belongs to this task.

Connection ownership rule

• Warehouse/app-layer resolves the active binding and builds BrinkConnectionDefinition
• ETL orchestration passes that connection context into the Brink integration boundary
• Kds.Ai.Integrations.BrinkPos must not resolve tenant context or load source bindings from the operational database for this flow
• the exact API shape may be either explicit BrinkConnectionDefinition parameters or a factory-created bound Brink client, but the connection context must come from outside the integration module

Business-date rule

For one child extract request:

1. if BusinessDate is provided, use it exactly
2. otherwise call GetCurrentBusinessDate
3. use the resolved value for the full tenant-day extract run

Extract boundary rule

• IBrinkPosFacade is control-only in this task
• IBrinkRawClient is the only ETL extraction API for GetOrders
• both consume externally provided Brink connection context
• job-layer code must not build SOAP envelopes, routes, headers, or auth rules
• integration-layer code must not own ETL batch state, S3 persistence, or tenant-binding lookup

Raw capture rule

• GetOrders must be captured as streaming raw SOAP XML
• original XML must be persisted to S3 raw/ without mutation
• the persisted raw artifact is the replay boundary
• transport success alone is not enough for endpoint success

Validation rule

Endpoint success requires persisted-raw validation after upload.

Validation must confirm at least:

• XML is well formed
• no SOAP fault is present
• Brink business success semantics are satisfied

Responsibility split:

• Brink-specific payload inspection and validation logic may live in Kds.Ai.Integrations.BrinkPos
• reopening the persisted S3 object, recording raw_s3_path, and deciding extract-stage success or failure belong to ETL/Warehouse orchestration

Orchestration rule

• one child extract job executes one tenant-day extract boundary
• fail-fast behavior is acceptable because v1 includes one replayable endpoint only
• endpoint diagnostics must be written in a form that later stages can reference

Testing

Unit tests

Unit tests are mandatory for this task.

Add unit tests for:

• Warehouse-side connection resolution handoff into the Brink integration boundary
• business-date resolution with explicit date override
• business-date resolution through GetCurrentBusinessDate
• endpoint catalog for the narrowed v1 scope
• raw capture orchestration using IBrinkRawClient
• persisted-raw validation success and failure
• SOAP fault failure handling
• Brink business-failure handling
• raw capture diagnostics and S3 path recording

Live E2E tests

This task ends at a real ETL boundary, so live E2E is required whenever Aspire-managed secrets and integrations are available.

Add live E2E coverage that verifies at least:

• the child extract flow can resolve real Brink connection data
• GetCurrentBusinessDate works against real Brink when BusinessDate is omitted
• GetOrders calls the real Brink API
• the returned XML is written to S3 raw/
• the persisted raw object can be reopened and validated successfully

Recommended live verification shape:

• run one tenant-day extract against real Brink
• assert that one expected raw artifact exists in S3
• assert that metadata and diagnostics point to the persisted object
• assert that no secret values are written into S3 metadata or diagnostics
• assert that the ETL path does not depend on DB-coupled connection resolution inside Kds.Ai.Integrations.BrinkPos

Acceptance Criteria

• [x] Child extract job entry exists for the tenant-day extract_raw stage
• [x] Warehouse/app-layer resolves the active Brink binding and supplies connection context to the integration boundary
• [x] GetCurrentBusinessDate is used when BusinessDate is omitted
• [x] GetOrders is captured through IBrinkRawClient
• [x] Kds.Ai.Integrations.BrinkPos no longer owns tenant resolution or source-binding lookup for this flow
• [x] ETL orchestration does not use typed GetOrders DTO extraction
• [x] Original raw SOAP XML is written to S3 raw/
• [x] Persisted raw validation is required before endpoint success
• [x] Extract diagnostics capture stream, service, operation, timing, and raw path
• [x] Unit tests cover business-date resolution, raw capture, and validation behavior
• [x] Live E2E verifies real Brink extraction and raw S3 persistence
• [x] Solution builds without errors

Notes

• Keep this task strict: extract_raw only.
• Do not widen scope to normalization or batch-lifecycle redesign inside this task.
• This is the first task where live E2E against real services is mandatory unless a concrete blocker is documented.
• Prefer the ongoing source-binding refactor direction: Kds.Ai.Integrations.BrinkPos should consume externally provided connection context rather than resolve DB-backed bindings internally.