Spensara Spensara Knowledge Base & Documentation Repository

Repository Docs & Knowledge Base

Consolidated Spensara documentation repository (722 Markdown & architecture documents in ../repository)
Built Word Documents (.docx): Spensara Overview (.docx) Capability Assessment (.docx) Brink POS Ingestion Analysis (.docx) The Borg Ingestion Architecture (.docx)

Implementation Specs (157)

Detailed engineering and feature implementation specifications.
implementation-specs/specs/AnomalyDetection/TASK6.1.md 16,459 bytes · Updated Oct 23, 2026 10:44 AM EDT
Raw File

TASK 6.1: Create Mock Testing Infrastructure

Summary

Create mock and fake implementations that allow the anomaly detection system to run without a real MCP server, MySQL database, or LLM, enabling local development and automated testing. Since the system is generic (no typed data access services), mocking happens at two levels:

  1. Mock IChatClient - returns pre-scripted agent responses (no LLM calls)
  2. Fake IMcpToolProvider - provides fake MCP tools that return canned SQL results

Prerequisite Refactor: Extract IMcpToolProvider Interface

The AnomalyDetectorAgent currently depends on the concrete McpToolProvider class, which makes it impossible to substitute a fake in tests. Before creating fakes, extract an IMcpToolProvider interface from McpToolProvider so that both the real provider and the fake implement the same contract. This follows the codebase pattern: public interface, internal sealed implementation.

Dependencies

  • TASK 1.4 (McpToolProvider)
  • TASK 2.1 (IAnomalyDetectorAgent, AnomalyDetectionResult)
  • TASK 2.2 (AnomalyDetectorAgent implementation)

Design Principles

  • No deleted interfaces: No IKdsDataAccessService, no IMcpClient wrapper, no domain DTOs
  • Mock at boundaries: Mock the IChatClient (LLM) and IMcpToolProvider (MCP server), not internal code
  • Generic: Mock infrastructure knows nothing about prep times or KDSDev - it returns generic SQL results
  • Deterministic: Tests produce repeatable results

Files to Modify (Prerequisite Refactor)

0a. backend/Kds.Ai.AnomalyDetector/Mcp/IMcpToolProvider.cs (New)

Extract a public interface from McpToolProvider:

csharp
using Microsoft.Extensions.AI;

namespace Kds.Ai.AnomalyDetector.Mcp;

/// <summary>
/// Provides MCP tools (e.g., mysql_query) as AITool objects for agent use.
/// </summary>
public interface IMcpToolProvider
{
    /// <summary>
    /// Retrieves available tools as AITool objects
    /// that can be passed directly to an agent.
    /// </summary>
    Task<IList<AITool>> GetToolsAsync(CancellationToken ct = default);
}

0b. backend/Kds.Ai.AnomalyDetector/Mcp/McpToolProvider.cs (Modify)

Add IMcpToolProvider to the implementation:

csharp
using Kds.Ai.AnomalyDetector.Settings;
using Microsoft.Extensions.AI;
using ModelContextProtocol.Client;

namespace Kds.Ai.AnomalyDetector.Mcp;

/// <summary>
/// Manages the MCP client lifecycle and provides tools from the MySQL MCP server.
/// </summary>
internal sealed class McpToolProvider(McpMySqlSettings settings) : IMcpToolProvider, IAsyncDisposable
{
    private McpClient? _client;

    /// <summary>
    /// Creates the MCP client and connects to the MySQL MCP server.
    /// </summary>
    public async Task<McpClient> GetClientAsync(CancellationToken ct = default)
    {
        if (_client is not null)
        {
            return _client;
        }

        var transport = new StdioClientTransport(new StdioClientTransportOptions
        {
            Name = "mysql-mcp-server",
            Command = settings.Command,
            Arguments = settings.Arguments,
            EnvironmentVariables = settings.EnvironmentVariables
        });

        _client = await McpClient.CreateAsync(transport, cancellationToken: ct);
        return _client;
    }

    /// <inheritdoc />
    public async Task<IList<AITool>> GetToolsAsync(CancellationToken ct = default)
    {
        var client = await GetClientAsync(ct);
        var tools = await client.ListToolsAsync(cancellationToken: ct);
        return [.. tools.Cast<AITool>()];
    }

    public async ValueTask DisposeAsync()
    {
        if (_client is not null)
        {
            await _client.DisposeAsync();
            _client = null;
        }
    }
}

Key changes:

  • Now implements IMcpToolProvider
  • Changed from public sealed to internal sealed (follows codebase pattern)
  • Uses primary constructor

0c. backend/Kds.Ai.AnomalyDetector/Agents/AnomalyDetectorAgent.cs (Modify)

Change from concrete McpToolProvider to IMcpToolProvider:

csharp
// Change the constructor parameter type:
public sealed class AnomalyDetectorAgent(
    IChatClient chatClient,
    IMcpToolProvider mcpToolProvider,  // was: McpToolProvider
    ILogger<AnomalyDetectorAgent> logger) : IAnomalyDetectorAgent

0d. backend/Kds.Ai.AnomalyDetector/DependencyInjectionExtensions.cs (Modify)

Register McpToolProvider as IMcpToolProvider:

csharp
builder.Services.AddSingleton<IMcpToolProvider>(provider =>
{
    var settings = provider.GetRequiredService<IOptions<McpMySqlSettings>>();
    return new McpToolProvider(settings.Value);
});

Files to Create

1. backend/Kds.Ai.AnomalyDetector.Tests/Fakes/FakeMcpToolProvider.cs

A fake that replaces the real McpToolProvider with in-process tool handlers:

csharp
using Kds.Ai.AnomalyDetector.Mcp;
using Microsoft.Extensions.AI;

namespace Kds.Ai.AnomalyDetector.Tests.Fakes;

/// <summary>
/// Fake MCP tool provider that returns canned SQL query results
/// without needing a real MCP server process.
/// </summary>
public sealed class FakeMcpToolProvider : IMcpToolProvider
{
    private readonly Dictionary<string, Func<string, string>> _queryHandlers = new();
    
    /// <summary>
    /// Register a handler for SQL queries matching a pattern.
    /// The handler receives the full SQL query and returns a JSON result.
    /// </summary>
    public FakeMcpToolProvider OnQuery(string pattern, Func<string, string> handler)
    {
        _queryHandlers[pattern] = handler;
        return this;
    }
    
    /// <summary>
    /// Register a handler that returns a fixed result for queries matching a pattern.
    /// </summary>
    public FakeMcpToolProvider OnQuery(string pattern, string fixedResult)
    {
        _queryHandlers[pattern] = _ => fixedResult;
        return this;
    }
    
    /// <inheritdoc />
    public Task<IList<AITool>> GetToolsAsync(CancellationToken ct = default)
    {
        var tool = AIFunctionFactory.Create(
            (string query) =>
            {
                foreach (var (pattern, handler) in _queryHandlers)
                {
                    if (query.Contains(pattern, StringComparison.OrdinalIgnoreCase))
                    {
                        return handler(query);
                    }
                }
                
                // Default: return empty result set
                return "[]";
            },
            name: "mysql_query",
            description: "Execute a SQL query against the MySQL database");
        
        return Task.FromResult<IList<AITool>>([tool]);
    }
}

2. backend/Kds.Ai.AnomalyDetector.Tests/Fakes/FakeChatClient.cs

A fake IChatClient that returns scripted responses, simulating the LLM:

csharp
using Microsoft.Extensions.AI;

namespace Kds.Ai.AnomalyDetector.Tests.Fakes;

/// <summary>
/// Fake IChatClient that returns scripted responses without calling an LLM.
/// Supports function calling by detecting tool call requests and routing them.
/// </summary>
public sealed class FakeChatClient : IChatClient
{
    private readonly Queue<ChatResponse> _responses = new();
    
    /// <summary>
    /// Enqueue a text response (the agent will receive this as assistant message)
    /// </summary>
    public FakeChatClient EnqueueTextResponse(string text)
    {
        _responses.Enqueue(new ChatResponse(new ChatMessage(ChatRole.Assistant, text)));
        return this;
    }
    
    /// <summary>
    /// Enqueue a function call response (simulates agent requesting a tool call)
    /// </summary>
    public FakeChatClient EnqueueFunctionCall(string functionName, string argumentsJson)
    {
        var functionCall = new FunctionCallContent(
            callId: Guid.NewGuid().ToString(),
            name: functionName,
            arguments: System.Text.Json.JsonSerializer.Deserialize<Dictionary<string, object?>>(argumentsJson));
        
        var message = new ChatMessage(ChatRole.Assistant, [functionCall]);
        _responses.Enqueue(new ChatResponse(message));
        return this;
    }
    
    public Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        if (_responses.Count == 0)
        {
            throw new InvalidOperationException(
                "FakeChatClient has no more scripted responses. " +
                "Enqueue more responses or check the test setup.");
        }
        
        return Task.FromResult(_responses.Dequeue());
    }
    
    public IAsyncEnumerable<ChatResponseUpdate> GetStreamingResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken cancellationToken = default)
    {
        throw new NotSupportedException("Streaming not supported in fake");
    }
    
    public ChatClientMetadata Metadata => new("FakeChatClient");
    
    public void Dispose() { }
}

3. backend/Kds.Ai.AnomalyDetector.Tests/Fakes/ScriptedAgentScenarios.cs

Pre-built test scenarios that combine fake tools + fake LLM responses:

csharp
using Kds.Ai.Data.Enums;

namespace Kds.Ai.AnomalyDetector.Tests.Fakes;

/// <summary>
/// Pre-built scenarios for testing.
/// Each scenario configures a FakeMcpToolProvider and FakeChatClient
/// to simulate a specific detection outcome.
/// </summary>
public static class ScriptedAgentScenarios
{
    /// <summary>
    /// Scenario: Agent queries data, finds no anomalies, calls emit_conclusion with anomalyDetected=false
    /// </summary>
    public static (FakeMcpToolProvider Tools, FakeChatClient Chat) NoAnomaly()
    {
        var tools = new FakeMcpToolProvider()
            .OnQuery("SELECT", "[]"); // All queries return empty
        
        var chat = new FakeChatClient()
            // First: agent requests mysql_query
            .EnqueueFunctionCall("mysql_query", """{"query": "SELECT COUNT(*) as cnt FROM orders WHERE done=1"}""")
            // Second: after getting tool result, agent calls emit_conclusion
            .EnqueueFunctionCall("emit_conclusion", """
            {
                "anomalyDetected": false,
                "summary": "No anomalies detected in the analyzed period",
                "findings": []
            }
            """);
        
        return (tools, chat);
    }
    
    /// <summary>
    /// Scenario: Agent queries data, finds anomaly, calls emit_conclusion with findings
    /// </summary>
    public static (FakeMcpToolProvider Tools, FakeChatClient Chat) SingleAnomaly(
        EventSeverity severity = EventSeverity.Medium)
    {
        var tools = new FakeMcpToolProvider()
            .OnQuery("SELECT", """[{"hour": 12, "avg_prep": 450, "baseline": 280}]""");
        
        var chat = new FakeChatClient()
            .EnqueueFunctionCall("mysql_query", """{"query": "SELECT AVG(preparation_time) FROM orders"}""")
            .EnqueueFunctionCall("emit_conclusion", $$"""
            {
                "anomalyDetected": true,
                "severity": "{{severity}}",
                "summary": "Significant deviation detected during lunch hours",
                "detailedAnalysis": "Analysis found 60% increase in average values during 12:00-13:00 compared to 28-day baseline.",
                "findings": [
                    {
                        "description": "60% increase during 12:00-13:00",
                        "severity": "{{severity}}",
                        "affectedPeriod": "2025-01-15 12:00-13:00",
                        "evidence": "Average 450s vs baseline 280s (2.8 standard deviations)"
                    }
                ]
            }
            """);
        
        return (tools, chat);
    }
    
    /// <summary>
    /// Scenario: Agent encounters an error during data access
    /// </summary>
    public static (FakeMcpToolProvider Tools, FakeChatClient Chat) DataError()
    {
        var tools = new FakeMcpToolProvider()
            .OnQuery("SELECT", _ => throw new InvalidOperationException("Connection refused"));
        
        var chat = new FakeChatClient()
            .EnqueueFunctionCall("mysql_query", """{"query": "SELECT 1"}""");
        
        return (tools, chat);
    }
}

4. backend/Kds.Ai.AnomalyDetector.Tests/Fakes/TestingExtensions.cs

DI helpers for test setup:

csharp
using Kds.Ai.AnomalyDetector.Mcp;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;

namespace Kds.Ai.AnomalyDetector.Tests.Fakes;

public static class TestingExtensions
{
    /// <summary>
    /// Replace real services with fakes for testing.
    /// Uses a scripted scenario to configure the fake chat client and MCP tools.
    /// </summary>
    public static IServiceCollection UseFakes(
        this IServiceCollection services,
        FakeChatClient fakeChatClient,
        FakeMcpToolProvider fakeMcpToolProvider)
    {
        // Replace IChatClient with fake
        services.AddSingleton<IChatClient>(fakeChatClient);
        
        // Replace IMcpToolProvider with fake
        services.AddSingleton<IMcpToolProvider>(fakeMcpToolProvider);
        
        return services;
    }
}

Usage Examples

In Unit Tests

csharp
[Fact]
public async Task DetectsAnomaly_WhenDataShowsDeviation()
{
    var (tools, chat) = ScriptedAgentScenarios.SingleAnomaly(EventSeverity.High);
    
    // Create agent with fakes
    var agent = new AnomalyDetectorAgent(chat, tools, NullLogger<AnomalyDetectorAgent>.Instance);
    
    var spec = new MonitorSpec
    {
        Name = "Test Monitor",
        SystemPrompt = "Analyze data and detect anomalies",
        State = MonitorState.Active
    };
    
    var result = await agent.RunDetectionAsync(spec, DateTimeOffset.UtcNow);
    
    Assert.True(result.AnomalyDetected);
    Assert.Single(result.Findings);
    Assert.Contains("12:00", result.Findings[0].AffectedPeriod);
}

In CLI for Local Testing

csharp
// In Program.cs, add a --mock flag
if (args.Contains("--mock"))
{
    var (tools, chat) = ScriptedAgentScenarios.SingleAnomaly();
    builder.Services.UseFakes(chat, tools);
}

Changes from Old Design

  • Deleted MockKdsDataAccessService entirely (IKdsDataAccessService deleted)
  • Deleted MockMcpClient entirely (IMcpClient wrapper deleted)
  • Deleted MockDataScenario enum with domain-specific scenarios (AnomalyAt12, AnomalyInDineIn, etc.)
  • Deleted all domain DTOs (PrepTimeHourlyStats, OrderTypeBreakdown, etc.)
  • New approach: Mock at IChatClient + IMcpToolProvider boundaries
  • FakeChatClient scripts the LLM conversation (no real model calls)
  • FakeMcpToolProvider implements IMcpToolProvider with fake AITool objects returning canned responses
  • ScriptedAgentScenarios combines both for common test patterns
  • Extracted IMcpToolProvider interface from McpToolProvider (prerequisite refactor for testability)

Acceptance Criteria

  • [ ] IMcpToolProvider interface extracted from McpToolProvider
  • [ ] McpToolProvider changed to internal sealed, implements IMcpToolProvider
  • [ ] AnomalyDetectorAgent depends on IMcpToolProvider (not concrete McpToolProvider)
  • [ ] DI registers McpToolProvider as IMcpToolProvider
  • [ ] FakeMcpToolProvider implements IMcpToolProvider with pattern-matching query handlers
  • [ ] FakeChatClient supports scripted text and function call responses
  • [ ] ScriptedAgentScenarios provides at least: NoAnomaly, SingleAnomaly, DataError
  • [ ] TestingExtensions registers fakes as IMcpToolProvider (not concrete type)
  • [ ] No references to IKdsDataAccessService, IMcpClient, or domain DTOs
  • [ ] Fakes produce deterministic results
  • [ ] Solution builds without errors

Notes

  • The fake approach tests the full agent flow including tool handling
  • FakeChatClient scripts are ordered (first call returns first response, etc.)
  • For more sophisticated testing, consider recording real MCP responses and replaying them
  • The FakeMcpToolProvider returns real AITool objects via IMcpToolProvider.GetToolsAsync(), so the agent's tool-calling logic is exercised
  • The IMcpToolProvider extraction is a prerequisite refactor — it changes McpToolProvider from public sealed to internal sealed and adds the interface. This must be done before creating the fakes.