# TASK 1: Source Binding And Dispatch ## Summary Implement the first job-start boundary for the Brink POS orders ETL flow. This task establishes the control-plane and scheduling layer that starts one tenant-day ETL run for the first Brink slice defined in `TASK0.md`. Analysis of `specs/BackendDatabaseConsolidation` shows that this stream provides important reusable groundwork for: - backend-owned customer and tenant persistence - identity-backed tenant membership - authenticated tenant resolution in backend APIs However, it does not yet define the ETL source-binding model needed for Brink API pull execution. This means Module 1 should: - reuse the future consolidated tenant/customer foundation where possible - avoid creating a conflicting tenant model - add the missing ETL source-binding and dispatch model explicitly This task owns: - verification and reuse analysis of source-binding groundwork from `specs/BackendDatabaseConsolidation` - source binding persistence and lookup - company-level dispatch - tenant-level child job contract - scheduled versus manual trigger convergence - required business-date resolution rule at child-job level - dispatch and child-entry observability This task does not implement raw extraction, S3 persistence, normalization, Redshift load, or curated mapping yet. ## Dependencies - [TASK 0](./TASK0.md) - Module 0 decision record ## Files to Create ### 1. Source binding model and repository contracts First verify whether the source-binding groundwork already being created in: - `specs/BackendDatabaseConsolidation` can be reused directly, or reused with narrow extensions. Do not create a parallel competing source-binding model for Brink ETL if the consolidation stream already provides the right persistence and repository direction. Analysis result for this task: - `BackendDatabaseConsolidation` is relevant for tenant/customer ownership and tenant membership resolution - it is not yet sufficient as the ETL source-binding model for Brink pull jobs - no reusable source connection contract is currently defined there for external integrations, schedules, or ETL dispatch Create or finalize the generic source binding model used to resolve one active Brink connection for one tenant. Minimum logical fields: - `id` - `company_id` - `tenant_id` - `source_system_type` - `display_name` - `service_url` - `access_token` - `location_token` - `location_key` - optional `configuration_json` - `cron_expression` - `time_zone_id` - `is_active` - `created_at` - `updated_at` This model must stay reusable, but for this implementation the Brink connection data is persisted in the database. Current implementation decision for this stream: - implement this entity now in backend persistence rather than waiting for `BackendDatabaseConsolidation` - later streams may reuse this entity and extend it if needed - add one seeded mock Brink binding record so live test credentials can be filled directly in the database Required implementation rule: - prefer reuse of the consolidation-stream tenant/customer foundations where compatible - add a dedicated ETL source-binding model for external source connections because the consolidation stream does not currently define one - document the relationship between the ETL source-binding model and the consolidated tenant/customer model explicitly ### 2. Dispatch and child job contracts Create the first-job contracts used by scheduled and manual execution. Required logical contracts: - `BrinkPosDispatchJobInput` - `BrinkPosExtractJobInput` - `BrinkPosExtractTrigger` Required fields: - `CompanyId` - `TenantId` on the child contract - optional `BusinessDate` - `Trigger` - `ForceReset` - optional `RequestedBy` ### 3. Cron sync and dispatch services Create or finalize the services responsible for: - syncing TickerQ cron entries from active source bindings - company-level dispatch - child extract job entry Recommended logical services: - `BrinkPosExtractCronSyncService` - `BrinkPosDispatchJobs` - `BrinkPosExtractJobs` Exact names may differ, but these responsibilities must be represented clearly. ### 4. Source binding and secret resolution abstractions Create or finalize abstractions needed so dispatch and child execution can resolve runtime Brink connectivity without leaking secret logic into scheduler code. Minimum logical responsibilities: - enumerate active bindings by `company_id` - resolve one active binding by `tenant_id` - read Brink connection data from the persisted source-binding record - build the runtime `BrinkConnectionDefinition` in Warehouse/app-layer and pass it into the Brink integration boundary explicitly ## Files to Modify ### 5. Dependency injection and host wiring Register the source binding repository, secret-provider path, dispatch services, child job entry, and cron sync service in the appropriate backend modules. Likely modules: - `backend/Kds.Ai.Data` - `backend/Kds.Ai.Warehouse` - `backend/Kds.Ai.ApiService` - `backend/Kds.Ai.Cli` - `backend/Kds.Ai.AppHost` - `backend/Kds.Ai.Integrations.BrinkPos` only for adapting its public API to consume externally provided connection context ### 5a. Cross-stream spec touch points If reuse or extension is needed, update the relevant Brink ETL spec and note the dependency on: - `specs/BackendDatabaseConsolidation/TASK.md` - related `BackendDatabaseConsolidation` subtasks that define or implement reusable persistence and tenant-binding foundations ### 6. Manual trigger entry point Add or finalize one API entry point for manual execution. The manual path must enqueue the same contract family used by scheduled execution. It must not bypass the job orchestration model. ## Required Behavior ### Source binding rules - reuse the source-binding foundation from `BackendDatabaseConsolidation` where possible - one active Brink binding maps to one tenant execution target - `company_id` is the dispatch grouping boundary - `tenant_id` is the atomic execution boundary - active uniqueness must be enforced for Brink bindings per tenant - `service_url`, `access_token`, `location_token`, and `location_key` are stored in the database as part of the source-binding data owned by backend - runtime Brink connection details are resolved from the persisted binding record, not from global app settings - runtime tenant resolution, source-connection lookup, and binding validation are Warehouse/app-layer responsibilities, not Brink integration responsibilities - scheduler code still must not interpret or manipulate Brink transport details beyond passing the tenant/job contract forward - if the reused model is insufficient, Brink ETL may extend it only through explicit documented additions ### Reuse analysis result For this task, reuse from `BackendDatabaseConsolidation` should be interpreted as follows: - reuse the consolidated tenant/customer identity and membership direction - reuse tenant ownership concepts and IDs that become the backend source of truth - do not create a second incompatible tenant model for ETL - do create a new ETL source-binding persistence model because the consolidation stream does not yet model: - external source connections - integration credentials - service URLs - source-system-specific schedule metadata - company-level ETL dispatch enumeration ### Scheduling and dispatch rules - scheduling uses two levels only: 1. company dispatch 2. tenant extract - scheduled and manual execution must converge into the same child contract - dispatch enumerates active tenant bindings for one company - dispatch enqueues one tenant child job per active binding - dispatch passes through `BusinessDate`, `Trigger`, `ForceReset`, and `RequestedBy` unchanged - dispatch does not resolve secrets - dispatch does not determine endpoint ordering - dispatch does not persist ETL batch state - one tenant enqueue failure must not fail the whole dispatch cycle - no-active-binding dispatch should log a warning and finish cleanly ### Business-date rule `GetCurrentBusinessDate` is required for this slice because the job runs daily. The child extract contract must follow this rule: 1. if `BusinessDate` is provided, use it exactly 2. otherwise resolve the effective date through `Sales2.GetCurrentBusinessDate` 3. use the resolved business date as the tenant-day execution boundary ### Child entry rule - the child extract job owns one tenant-day execution boundary only - this task stops at child-job readiness and contract definition - endpoint execution, raw capture, validation, and downstream ETL stages belong to later tasks ## Testing ### Unit tests Unit tests are mandatory for this task. Add unit tests for: - reuse/extension rules for the source-binding model when integrating with `BackendDatabaseConsolidation` - source binding uniqueness rules - active binding lookup by `company_id` and `tenant_id` - persisted Brink connection-data resolution from the database - mapping from dispatch input to child input - pass-through of `BusinessDate`, `Trigger`, `ForceReset`, and `RequestedBy` - no-active-binding dispatch behavior - partial enqueue-failure dispatch behavior - manual API path enqueueing the same contract family as scheduled execution - child business-date resolution routing ### Integration or host-level tests Add integration or host-level tests for: - repository-backed source-binding resolution using the consolidated persistence model when available - cron sync creation/update from active source bindings - scheduled dispatch enqueueing tenant child jobs - manual API enqueueing tenant child jobs ### Live E2E expectation This module is not yet a finished ETL data boundary, so a full live E2E against AWS or raw S3 output is not required here. However, if practical through Aspire wiring, add a live host-level verification that: - the app can resolve one active Brink binding from persisted metadata - the manual or scheduled path can enqueue the child extract job with the expected payload - the child path can resolve the effective business date using real Brink credentials when `BusinessDate` is omitted If this live verification is not implemented in this task, document the blocker explicitly and keep the requirement attached to the first extract task. ## Acceptance Criteria - [ ] Source-binding groundwork from `BackendDatabaseConsolidation` is reviewed before introducing Brink-specific persistence changes - [ ] Reusable tenant/customer foundations from `BackendDatabaseConsolidation` are adopted where compatible - [ ] The gap between consolidated tenant/customer persistence and ETL source-binding needs is documented explicitly - [ ] A dedicated ETL source-binding model is defined because `BackendDatabaseConsolidation` does not currently provide one - [ ] Source binding model exists or is finalized with generic metadata fields - [ ] Active uniqueness is enforced for Brink tenant bindings - [ ] Repository access exists for active binding enumeration by `company_id` - [ ] Repository access exists for active binding resolution by `tenant_id` - [ ] `service_url`, `access_token`, `location_token`, and `location_key` are persisted in the database for active Brink bindings - [ ] Runtime Brink connection resolution no longer depends on one global temporary connection or `BrinkPosConnectionOptions` - [ ] Runtime Brink connection resolution no longer lives inside `Kds.Ai.Integrations.BrinkPos` - [ ] `BrinkPosDispatchJobInput` exists - [ ] `BrinkPosExtractJobInput` exists - [ ] `BrinkPosExtractTrigger` exists with `Scheduled` and `Manual` - [ ] Scheduled and manual execution converge into the same child contract family - [ ] Cron sync is driven by active source bindings - [ ] Dispatch logs no-binding warnings and enqueue failures - [ ] Child contract supports explicit or resolved business date - [ ] Unit tests cover binding rules, dispatch behavior, and business-date routing - [ ] Integration or host-level tests cover cron sync and enqueue paths - [ ] Solution builds without errors ## Notes - Follow `TASK0.md` as the authoritative scope document. - Keep this task strict: it starts the `orders` and `order_items` ETL path, but does not widen business scope. - The source binding contract should remain reusable for future API pull integrations. - `specs/BackendDatabaseConsolidation` is relevant mainly for tenant/customer ownership and membership, not yet for ETL source bindings. - The current `TemporaryBrinkConnectionResolver` and `BrinkPosConnectionOptions` path should be treated as transitional and replaced by repository-backed database resolution. - `Kds.Ai.Integrations.BrinkPos` should consume an externally provided `BrinkConnectionDefinition` or a factory-created bound client, not resolve tenant binding from DB on its own. - For this implementation, Brink connection data is stored in backend persistence rather than in an external secret store. - The live data-boundary E2E requirement starts fully in the next ETL task where raw extraction and S3 persistence are implemented.