Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)

Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Implementation Specs (157)

Detailed engineering and feature implementation specifications.

implementation-specs/specs/BrinkPosOrdersFirstEtlJob/TASK2.3.md 2,888 bytes · Updated Oct 23, 2026 10:44 AM EDT

Raw File

TASK 2.3: Implement S3 Raw Persistence And Persisted-Raw Validation

Summary

Implement the actual extract_raw persistence boundary in ETL/Warehouse-facing code: write the streamed GetOrders XML to S3 raw/, reopen the persisted object, and validate it before endpoint success is accepted. This subtask is the core replay boundary for the first Brink ETL slice.

The Brink integration may contribute Brink-specific payload validation components, but it must not take ownership of S3 persistence, replay-boundary semantics, or ETL diagnostics models.

Dependencies

TASK 2.2 - Brink raw capture boundary

Files to Create

1. Raw capture result and diagnostics models

Create or finalize the minimum models needed to preserve:

raw S3 path
service and operation identity
timing
validation outcome
safe endpoint diagnostics

Files to Modify

2. Shared ETL storage integration

Modify the shared S3 storage path so raw XML is written into the standard raw/ layout and can be reopened for validation.

Do not add S3 storage dependencies or raw/ path ownership into Kds.Ai.Integrations.BrinkPos.

3. Brink persisted-raw validation flow

Modify or finalize the ETL validation flow so endpoint success depends on validation of the persisted raw object, using Brink-specific validation components only for payload inspection semantics.

Required Behavior

original GetOrders XML is written to S3 raw/ without mutation
the persisted raw object is reopened after upload
validation confirms well-formed XML, no SOAP fault, and Brink business success
transport success alone does not mark the endpoint as successful
diagnostics must not store secrets
S3 object ownership, raw_s3_path, and stage-success decisions remain outside Kds.Ai.Integrations.BrinkPos

Testing

Unit tests

Add unit tests for:

raw S3 path generation and recording
persisted-raw reopen and validation flow
XML invalidity failure
SOAP fault failure
Brink business-failure handling
diagnostics sanitization
integration payload-validation behavior in isolation from S3 storage concerns

Acceptance Criteria

[x] Raw XML is written to S3 raw/
[x] Persisted raw object is reopened after upload
[x] Endpoint success requires persisted-raw validation
[x] Validation failures are surfaced as endpoint failures
[x] Diagnostics record raw artifact location without persisting secrets
[x] Kds.Ai.Integrations.BrinkPos is not given S3 persistence ownership or ETL diagnostics ownership
[x] Unit tests cover persistence and validation behavior

Notes

This subtask creates the first immutable replay boundary.
Use the Brink integration only as the source-system adapter and payload validator.
Keep failure-package redesign out of scope here; the broader lifecycle and failure model belongs to TASK 3.