Technologies

This document captures the technology choices for the agent-based monitoring system and recommends options for knowledgebase storage + retrieval.

Model hosting

Primary choice: Amazon Bedrock

• Use Bedrock for model inference (and optionally embeddings, if you want a managed embedding model).
• Agent orchestration remains in the application layer (Microsoft Agent Framework), so the model host is a replaceable backend.

Agent runtime / orchestration

Primary choice: Microsoft Agent Framework

• GitHub: microsoft/agent-framework
• Docs: Microsoft Learn: Agent Framework overview

Rationale:

• Multi-agent orchestration and workflow primitives with .NET support.
• Pluggable context providers and tool integration patterns.
• Built-in support for Model Context Protocol (MCP) clients (for tool connectivity).

Hosting (vendor-neutral):

• Run agents as containerized services (Docker) on ECS/EKS/Kubernetes (or any VM/container platform).
• Keep agent state/specs in your KB stores (below) so compute is stateless and portable.

Database connectivity

Primary choice: Model Context Protocol (MCP) servers

Pattern:

• Each source system (POS, KDS, inventory, etc.) is exposed as an MCP server (or a small set of servers).
• The Data Access Agent is an MCP client and is the only component allowed to touch data sources directly.

Why:

• Keeps schema/tool noise out of other agents’ contexts.
• Allows consistent, permissioned access across heterogeneous sources.

Minimum guardrails to bake into MCP tools:

• Restaurant/tenant scoping (every query requires restaurant_id and time window).
• Read-only enforcement and query auditing.
• Prefer returning feature panels (aggregates/cohorts/distributions) over raw rows.

Warehouse & ETL (future)

Current (MVP):

• One MySQL instance with multiple databases; a single service account can query all tenants.

Recommended direction for a managed warehouse you operate:

• Raw zone: S3 (or S3-compatible) with immutable extracts in open formats (prefer Parquet), partitioned by restaurant_id, source_system, and date.
• Serving warehouse (OLAP): ClickHouse for fast analytical SQL over large, mostly-immutable event data (orders, line items, transactions, etc.).
• Operational metadata DB: keep MySQL for tenant config, KB structured facts, MonitorSpec, run history, and alert dedupe.

Why ClickHouse fits this system:

• Optimized for append-heavy, analytical workloads (lots of GROUP BY, time-window scans, cohorts).
• Strong compression and performance for time-series/event-style tables.

Tenant isolation options (in increasing “strictness”):

• App-layer isolation only: simplest, but weakest; relies on the Data Access Agent to always enforce restaurant_id scoping.
• Shared ClickHouse cluster, separate DB + user per tenant: good defense-in-depth while still allowing a unified platform.
• Deployment per restaurant (separate warehouse stack per tenant): strongest isolation, but more ops cost; also makes future cross-restaurant comparisons harder unless you add an aggregation layer.

Portability / escape hatch:

• Keeping the raw zone in Parquet on S3 lets you move to a lakehouse query engine later (e.g., Iceberg + Trino/Spark) without re-extracting from restaurants.

Knowledgebase storage + retrieval (recommended)

The system needs two kinds of knowledge:

1) Structured “facts” (authoritative): schema catalog, join graph, entity mappings, KPI definitions, versions. 2) Unstructured text (supporting): manuals/SOPs, vendor docs, “what does this mean?” passages.

Recommended architecture (AWS-friendly, still portable)

• Structured KB store: keep your current MySQL (or use PostgreSQL) for:
• Capabilities Manifest (tables/columns, units, freshness, join edges)
• KPI Catalog (definitions/targets/exceptions)
• MonitorSpec versions and approvals
• Run history + alert dedupe memory
• Document store: S3 (or any S3-compatible object store like MinIO) for raw manuals and vendor docs (plus extracted text artifacts).
• Retrieval / vector search (choose one):
• OpenSearch (AWS OpenSearch Service or self-managed): hybrid retrieval (BM25 + vector) with metadata filters.
• ChromaDB (self-hosted): simple vector DB for chunks + embeddings + metadata (keep MySQL as the system-of-record for structured facts).
• Redis Stack (RediSearch vector search): fast “serving index” for vectors + metadata filters; keep MySQL/S3 as the source of truth and rebuild/sync the Redis index as needed.
• PostgreSQL + pgvector: simplest “single DB” option if you’re willing to use Postgres for both structured KB + vectors.

Why this works well for your system:

• Structured KB stays queryable and versioned (so specs don’t rot when schemas change).
• A dedicated search index (OpenSearch) or pgvector provides filterable retrieval for manuals/docs without overloading the LLM context.
• The KB/Semantic Agent can return a compact “KB packet” that merges structured facts with a few relevant doc passages.

Alternative: single-store (simpler ops, fewer moving parts)

If you strongly prefer one primary store:

• PostgreSQL + pgvector: store structured KB + chunks + embeddings in one DB.
• Pros: simplest deployment model, strong transactional semantics, easy versioning.
• Cons: retrieval quality/ranking/filters are good but typically less specialized than a dedicated search service.
• OpenSearch-only: store doc chunks + embeddings + metadata in OpenSearch; keep structured facts in a relational DB anyway.
• Pros: strong retrieval UX; flexible filtering.
• Cons: not a great system-of-record for versioned structured facts.

MVP option: vectors in-memory (fastest to ship)

If the KB is small (manuals + a bit of schema docs), you can start with an in-memory vector index inside the KB/Semantic Agent:

• Store doc chunks + metadata in your relational DB (MySQL).
• Store embeddings either:
• in MySQL as a BLOB (so you can reload quickly on startup), or
• regenerated on startup if the embedding volume is tiny and cold-start time is acceptable.
• Build an in-memory ANN index (or brute-force similarity for small N) per restaurant/collection at runtime.

This is fine for early prototyping, but you’ll likely want a persistent vector store (Chroma/OpenSearch/Postgres) once you have multiple instances, frequent KB updates, or you need consistent results across restarts.

Notes on Redis as a “middle option”

Redis can be a good fit when you want very low-latency retrieval and you’re comfortable treating the vector index as derived state:

• Use Redis primarily as a serving index / cache for document chunk vectors and metadata filters (restaurant_id, doc_type, etc.).
• Keep the source of truth in MySQL/S3; make the Redis index rebuildable from those stores.
• Beware memory sizing (vector indexes can be large) and confirm your chosen Redis deployment supports the required vector search features (e.g., RediSearch).

Optional (AWS-managed RAG for documents)

If you want a managed “docs RAG” layer that integrates directly with Bedrock:

• Knowledge Bases for Amazon Bedrock can handle ingestion + chunking + embeddings + retrieval for unstructured documents.
• You will still want the structured KB store (MySQL/PostgreSQL) for schema facts, join graph, KPI catalog, specs, and run history.

Retrieval strategy (how agents should use the KB)

To avoid hallucinating schema/joins/KPIs:

• Prefer structured lookups first (manifest + KPI catalog).
• Use doc retrieval only to:
• confirm definitions (“prep time” meaning, units, exceptions)
• reference the manual in explanations
• discover additional candidate KPIs the structured extractor missed

Practical “KB packet” returned by the KB/Semantic Agent should include:

• Canonical entity mapping (e.g., Order, LineItem, Station) → source tables/columns
• Verified join path(s) with confidence
• KPI definitions relevant to the monitor prompt (targets + exceptions)
• 3–8 short doc excerpts (manual/vendor) with doc IDs + section headings

Observability (recommended)

• OpenTelemetry tracing across orchestrator, MCP calls, and retrieval.
• Persist per-run “evidence panels” (or references) so alerts can be audited later.