Categories
Implementation Specs (157)
- specs/AnomalyDetection/TASK1.1.md
- specs/AnomalyDetection/TASK1.2.md
- specs/AnomalyDetection/TASK1.3.md
- specs/AnomalyDetection/TASK1.4.md
- specs/AnomalyDetection/TASK2.1.md
- specs/AnomalyDetection/TASK2.2.md
- specs/AnomalyDetection/TASK3.1.md
- specs/AnomalyDetection/TASK3.2.md
- specs/AnomalyDetection/TASK3.3.md
- specs/AnomalyDetection/TASK4.1.md
- specs/AnomalyDetection/TASK4.2.md
- specs/AnomalyDetection/TASK4.3.md
- specs/AnomalyDetection/TASK4.4.md
- specs/AnomalyDetection/TASK5.1.md
- specs/AnomalyDetection/TASK5.2.md
- specs/AnomalyDetection/TASK5.3.md
- specs/AnomalyDetection/TASK6.1.md
- specs/AnomalyDetection/TASK6.2.md
- specs/AnomalyDetection/TASK6.3.md
- specs/AnomalyDetection/TASKS.md
- specs/ApiServicePostgresTestStability/TASK.md
- specs/BackendDatabaseConsolidation/TASK.md
- specs/BackendDatabaseConsolidation/TASK1.1.md
- specs/BackendDatabaseConsolidation/TASK1.2.md
- specs/BackendDatabaseConsolidation/TASK1.3.md
- specs/BackendDatabaseConsolidation/TASK2.1.md
- specs/BackendDatabaseConsolidation/TASK2.2.md
- specs/BackendDatabaseConsolidation/TASK2.3.md
- specs/BackendDatabaseConsolidation/TASK3.1.md
- specs/BackendDatabaseConsolidation/TASK3.2.md
- specs/BackendDatabaseConsolidation/TASK4.1.md
- specs/BackendDatabaseConsolidation/TASK4.2.md
- specs/BackendDatabaseConsolidation/TASK4.3.md
- specs/BackendDatabaseConsolidation/TASK5.1.md
- specs/BackendDatabaseConsolidation/TASK5.2.md
- specs/BrinkLaborGetShiftsStaging/TASK.md
- specs/BrinkPosOrdersFirstEtlJob/TASK.md
- specs/BrinkPosOrdersFirstEtlJob/TASK0.md
- specs/BrinkPosOrdersFirstEtlJob/TASK1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK5.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.1.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.2.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.3.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.4.md
- specs/BrinkPosOrdersFirstEtlJob/TASK6.md
- specs/BrinkPosOrdersFirstEtlJob/TASK7.md
- specs/BrinkPosOrdersFirstEtlJob/TASK8.md
- specs/BrinkPosOrdersFirstEtlJob/TASK9.md
- specs/BrinkSettings2ToStaging/TASK.MD
- specs/CuratedToS3Stage/TASK.md
- specs/CuratedToS3Stage/TASK0.md
- specs/CuratedToS3Stage/TASK1.md
- specs/CuratedToS3Stage/TASK2.md
- specs/CuratedToS3Stage/TASK3.md
- specs/CuratedToS3Stage/TASK4.md
- specs/CustomerCredentialCustomHeaders/TASK.md
- specs/CustomerCredentialCustomHeaders/TASK1.1.md
- specs/CustomerCredentialCustomHeaders/TASK1.2.md
- specs/CustomerCredentialCustomHeaders/TASK2.1.md
- specs/CustomerCredentialCustomHeaders/TASK2.2.md
- specs/CustomerCredentialCustomHeaders/TASK3.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK.md
- specs/CustomerCredentialTriggeredProvisioning/TASK1.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK1.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK2.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK2.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK3.1.md
- specs/CustomerCredentialTriggeredProvisioning/TASK3.2.md
- specs/CustomerCredentialTriggeredProvisioning/TASK4.1.md
- specs/EtlSourceAdapterFramework/TASK.md
- specs/EtlSourceAdapterFramework/TASK0.md
- specs/EtlSourceAdapterFramework/TASK1.md
- specs/EtlSourceAdapterFramework/TASK2.md
- specs/EtlSourceAdapterFramework/TASK3.md
- specs/EtlSourceAdapterFramework/TASK4.md
- specs/EtlSourceAdapterFramework/TASK5.md
- specs/EtlSourceAdapterFramework/TASK6.md
- specs/KdsCdcDebeziumIntegration/TASK.md
- specs/KdsCdcDebeziumIntegration/TASK1.md
- specs/KdsCdcDebeziumIntegration/TASK2.md
- specs/KdsCdcDebeziumIntegration/TASK3.md
- specs/KdsCdcDebeziumIntegration/TASK4.md
- specs/KdsCdcDebeziumIntegration/TASK5.md
- specs/KdsCdcDurableRelationRouting/TASK.md
- specs/KdsCdcDurableRelationRouting/TASK1.md
- specs/KdsCdcDurableRelationRouting/TASK2.md
- specs/KdsCdcDurableRelationRouting/TASK3.md
- specs/KdsCdcDurableRelationRouting/TASK4.md
- specs/KdsCdcDurableRelationRouting/TASK5.md
- specs/KdsCdcIngestion/TASK.md
- specs/KdsCdcIngestion/TASK1.md
- specs/KdsCdcIngestion/TASK2.md
- specs/KdsCdcIngestion/TASK3.md
- specs/KdsCdcIngestion/TASK4.md
- specs/KdsCdcIngestion/TASK5.md
- specs/KdsCdcIngestion/TASK6.md
- specs/KdsCdcIngestion/TASK7.md
- specs/KdsCdcIngestion/TASK8.md
- specs/KdsCdcIngestion/TASK9.md
- specs/KdsCdcReplayExecution/TASK.md
- specs/KdsCdcReplayExecution/TASK1.md
- specs/KdsCdcReplayExecution/TASK2.md
- specs/KdsCdcReplayExecution/TASK3.md
- specs/KdsCdcReplayExecution/TASK4.md
- specs/KdsMonitorServingContractMigration/TASK.md
- specs/KdsMonitorServingContractMigration/TASK1.1.md
- specs/KdsMonitorServingContractMigration/TASK1.2.md
- specs/KdsMonitorServingContractMigration/TASK2.1.md
- specs/KdsMonitorServingContractMigration/TASK2.2.md
- specs/KdsMonitorServingContractMigration/TASK3.1.md
- specs/KdsMonitorServingContractMigration/TASK3.2.md
- specs/RedshiftFoundation/TASK.md
- specs/RedshiftFoundation/TASK1.1.md
- specs/RedshiftFoundation/TASK1.2.md
- specs/RedshiftFoundation/TASK1.3.md
- specs/RedshiftFoundation/TASK1.4.md
- specs/RedshiftFoundation/TASK2.1.md
- specs/RedshiftFoundation/TASK2.2.md
- specs/RedshiftFoundation/TASK3.1.md
- specs/RedshiftFoundation/TASK3.2.md
- specs/RedshiftFoundation/TASK3.3.md
- specs/RedshiftSyntheticDataFeed/TASK.md
- specs/RedshiftSyntheticDataFeed/TASK1.1.md
- specs/RedshiftSyntheticDataFeed/TASK1.2.md
- specs/RedshiftSyntheticDataFeed/TASK1.3.md
- specs/RedshiftSyntheticDataFeed/TASK2.1.md
- specs/RedshiftSyntheticDataFeed/TASK2.2.md
- specs/RedshiftSyntheticDataFeed/TASK3.1.md
- specs/RedshiftSyntheticDataFeed/TASK3.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK1.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK1.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK2.3.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK3.1.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK3.2.md
- specs/SingleStoreRedshiftRuntimeReadPath/TASK4.1.md
TASK 6.2: Add Integration Tests for End-to-End Flow
Summary
Create integration tests that verify the complete anomaly detection flow works correctly, from orchestrator execution through agent detection to result storage. Tests use the FakeChatClient and FakeMcpToolProvider from TASK 6.1 to avoid real LLM and MCP calls.
Dependencies
- All previous tasks completed
- TASK 6.1 (FakeChatClient, FakeMcpToolProvider, ScriptedAgentScenarios, TestingExtensions)
Files to Create
1. backend/Kds.Ai.AnomalyDetector.Tests/Kds.Ai.AnomalyDetector.Tests.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<IsPackable>false</IsPackable>
<IsTestProject>true</IsTestProject>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" />
<PackageReference Include="xunit" />
<PackageReference Include="xunit.runner.visualstudio">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<PackageReference Include="FluentAssertions" />
<PackageReference Include="Microsoft.EntityFrameworkCore.InMemory" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Kds.Ai.AnomalyDetector\Kds.Ai.AnomalyDetector.csproj" />
<ProjectReference Include="..\Kds.Ai.Data\Kds.Ai.Data.csproj" />
</ItemGroup>
</Project>Note: No Moq dependency. Tests use FakeChatClient and FakeMcpToolProvider from TASK 6.1 instead of mock frameworks. The fakes exercise the real agent tool-calling logic.
Important: The test project needs InternalsVisibleTo attributes in both referenced projects to access internal sealed implementation classes:
In backend/Kds.Ai.Data/Kds.Ai.Data.csproj:
<ItemGroup>
<InternalsVisibleTo Include="Kds.Ai.AnomalyDetector.Tests" />
</ItemGroup>In backend/Kds.Ai.AnomalyDetector/Kds.Ai.AnomalyDetector.csproj:
<ItemGroup>
<InternalsVisibleTo Include="Kds.Ai.AnomalyDetector.Tests" />
</ItemGroup>This allows tests to directly instantiate internal sealed classes like EfMonitorSpecRepository, EfMonitorRunRepository, MonitorRunOrchestrator, and AlertEmitter.
2. backend/Kds.Ai.AnomalyDetector.Tests/IntegrationTests/OrchestratorIntegrationTests.cs
using FluentAssertions;
using Kds.Ai.AnomalyDetector.Agents;
using Kds.Ai.AnomalyDetector.Alerts;
using Kds.Ai.AnomalyDetector.Mcp;
using Kds.Ai.AnomalyDetector.Orchestration;
using Kds.Ai.AnomalyDetector.Repositories;
using Kds.Ai.AnomalyDetector.Seeding;
using Kds.Ai.AnomalyDetector.Tests.Fakes;
using Kds.Ai.Data;
using Kds.Ai.Data.Entities;
using Kds.Ai.Data.Enums;
using Kds.Ai.Data.Repositories;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace Kds.Ai.AnomalyDetector.Tests.IntegrationTests;
public sealed class OrchestratorIntegrationTests : IAsyncLifetime
{
private ServiceProvider _serviceProvider = null!;
private KdsAiDbContext _dbContext = null!;
public async Task InitializeAsync()
{
var services = new ServiceCollection();
// In-memory database
services.AddDbContext<KdsAiDbContext>(options =>
options.UseInMemoryDatabase($"TestDb_{Guid.NewGuid()}"));
// Logging
services.AddLogging(b => b.AddProvider(NullLoggerProvider.Instance));
// Repositories
services.AddScoped<IMonitorSpecRepository, EfMonitorSpecRepository>();
services.AddScoped<IMonitorRunRepository, EfMonitorRunRepository>();
// Alert emitter
services.AddScoped<IAlertEmitter, AlertEmitter>();
// Seeder
services.AddScoped<MonitorSpecSeeder>();
_serviceProvider = services.BuildServiceProvider();
_dbContext = _serviceProvider.GetRequiredService<KdsAiDbContext>();
await _dbContext.Database.EnsureCreatedAsync();
}
public async Task DisposeAsync()
{
await _dbContext.DisposeAsync();
await _serviceProvider.DisposeAsync();
}
[Fact]
public async Task FullFlow_NoAnomaly_CreatesCompletedRun()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.NoAnomaly();
var orchestrator = CreateOrchestrator(chat, tools);
var spec = await CreateTestSpec("No Anomaly Test");
// Act
var run = await orchestrator.ExecuteAsync(spec.Id);
// Assert
run.Status.Should().Be(MonitorRunStatus.Completed);
run.AnomaliesDetected.Should().Be(0);
run.AlertEvents.Should().BeEmpty();
run.CompletedAt.Should().NotBeNull();
}
[Fact]
public async Task FullFlow_WithAnomaly_CreatesAlertAndRun()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.SingleAnomaly(EventSeverity.High);
var orchestrator = CreateOrchestrator(chat, tools);
var spec = await CreateTestSpec("Anomaly Detection Test");
// Act
var run = await orchestrator.ExecuteAsync(spec.Id);
// Assert
run.Status.Should().Be(MonitorRunStatus.Completed);
run.AnomaliesDetected.Should().BeGreaterThan(0);
run.AlertEvents.Should().NotBeEmpty();
// Verify alert was created in the database
var alert = run.AlertEvents.First();
alert.Id.Should().NotBe(Guid.Empty);
alert.Severity.Should().Be(EventSeverity.High);
alert.Type.Should().Be(EventType.AiAnomaly);
}
[Fact]
public async Task DryRun_DoesNotPersist()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.SingleAnomaly();
var spec = await CreateTestSpec("Dry Run Test");
var orchestrator = CreateOrchestrator(chat, tools);
// Act
var run = await orchestrator.ExecuteAsync(
spec,
options: new RunOptions { DryRun = true });
// Assert
run.Status.Should().Be(MonitorRunStatus.Completed);
// Verify nothing was persisted
var runCount = await _dbContext.MonitorRuns.CountAsync();
var alertCount = await _dbContext.Events.CountAsync();
runCount.Should().Be(0);
alertCount.Should().Be(0);
}
[Fact]
public async Task SuppressAlerts_NoAlertEmitted()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.SingleAnomaly();
var spec = await CreateTestSpec("Suppress Alert Test");
var orchestrator = CreateOrchestrator(chat, tools);
// Act
var run = await orchestrator.ExecuteAsync(
spec.Id,
options: new RunOptions { SuppressAlerts = true });
// Assert
run.AlertEvents.Should().BeEmpty();
var alertCount = await _dbContext.Events.CountAsync();
alertCount.Should().Be(0);
}
[Fact]
public async Task SpecNotFound_ThrowsException()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.NoAnomaly();
var orchestrator = CreateOrchestrator(chat, tools);
// Act & Assert
var act = () => orchestrator.ExecuteAsync(Guid.NewGuid());
await act.Should().ThrowAsync<InvalidOperationException>()
.WithMessage("*not found*");
}
[Fact]
public async Task ExecuteAllActive_RunsOnlyActiveMonitors()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.NoAnomaly();
var orchestrator = CreateOrchestrator(chat, tools);
var specRepo = _serviceProvider.GetRequiredService<IMonitorSpecRepository>();
// Create 3 specs: 2 active, 1 paused
await CreateTestSpec("Active Monitor 1", MonitorState.Active);
await CreateTestSpec("Active Monitor 2", MonitorState.Active);
await CreateTestSpec("Paused Monitor", MonitorState.Paused);
// Act
var runs = await orchestrator.ExecuteAllActiveAsync();
// Assert
runs.Should().HaveCount(2);
runs.Should().OnlyContain(r => r.Status == MonitorRunStatus.Completed);
}
[Fact]
public async Task ExecuteAllActive_SkipsDraftAndArchivedMonitors()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.NoAnomaly();
var orchestrator = CreateOrchestrator(chat, tools);
// Create monitors in each state
await CreateTestSpec("Draft Monitor", MonitorState.Draft);
await CreateTestSpec("Active Monitor", MonitorState.Active);
await CreateTestSpec("Paused Monitor", MonitorState.Paused);
await CreateTestSpec("Archived Monitor", MonitorState.Archived);
// Act
var runs = await orchestrator.ExecuteAllActiveAsync();
// Assert - only the Active monitor should run
runs.Should().HaveCount(1);
}
[Fact]
public async Task AgentError_RunMarkedAsFailed()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.DataError();
var spec = await CreateTestSpec("Error Handling Test");
var orchestrator = CreateOrchestrator(chat, tools);
// Act
var run = await orchestrator.ExecuteAsync(spec.Id);
// Assert
run.Status.Should().Be(MonitorRunStatus.Failed);
run.ErrorMessage.Should().NotBeNullOrEmpty();
run.CompletedAt.Should().NotBeNull();
}
[Fact]
public async Task Run_RecordsDuration()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.NoAnomaly();
var spec = await CreateTestSpec("Duration Test");
var orchestrator = CreateOrchestrator(chat, tools);
// Act
var run = await orchestrator.ExecuteAsync(spec.Id);
// Assert
run.DurationMs.Should().BeGreaterOrEqualTo(0);
}
// === Helper Methods ===
private async Task<MonitorSpec> CreateTestSpec(
string name,
MonitorState state = MonitorState.Active)
{
var seeder = _serviceProvider.GetRequiredService<MonitorSpecSeeder>();
return await seeder.EnsureAsync(
name,
"Test system prompt: Analyze data and detect anomalies.",
initialState: state);
}
private IMonitorRunOrchestrator CreateOrchestrator(
FakeChatClient chat,
FakeMcpToolProvider tools)
{
// Create agent with fakes (single generic agent, no factory)
var agent = new AnomalyDetectorAgent(
chat,
tools,
NullLogger<AnomalyDetectorAgent>.Instance);
return new MonitorRunOrchestrator(
_serviceProvider.GetRequiredService<IMonitorSpecRepository>(),
_serviceProvider.GetRequiredService<IMonitorRunRepository>(),
agent,
_serviceProvider.GetRequiredService<IAlertEmitter>(),
NullLogger<MonitorRunOrchestrator>.Instance);
}
}3. backend/Kds.Ai.AnomalyDetector.Tests/UnitTests/AlertEmitterTests.cs
using FluentAssertions;
using Kds.Ai.AnomalyDetector.Agents;
using Kds.Ai.AnomalyDetector.Alerts;
using Kds.Ai.Data;
using Kds.Ai.Data.Entities;
using Kds.Ai.Data.Enums;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace Kds.Ai.AnomalyDetector.Tests.UnitTests;
public sealed class AlertEmitterTests : IAsyncLifetime
{
private KdsAiDbContext _dbContext = null!;
private AlertEmitter _emitter = null!;
public async Task InitializeAsync()
{
var options = new DbContextOptionsBuilder<KdsAiDbContext>()
.UseInMemoryDatabase($"AlertTest_{Guid.NewGuid()}")
.Options;
_dbContext = new KdsAiDbContext(options);
_emitter = new AlertEmitter(_dbContext, NullLogger<AlertEmitter>.Instance);
await _dbContext.Database.EnsureCreatedAsync();
}
public async Task DisposeAsync()
{
await _dbContext.DisposeAsync();
}
[Fact]
public async Task EmitAsync_CreatesAlertWithCorrectFormat()
{
// Arrange
var spec = CreateTestSpec("Prep Time Monitor - Downtown");
var result = new AnomalyDetectionResult
{
AnomalyDetected = true,
Severity = EventSeverity.High,
Summary = "Prep times elevated during lunch",
DetailedAnalysis = "Detailed markdown analysis...",
Findings =
[
new AnomalyFinding
{
Description = "60% increase during 12:00-13:00",
Severity = EventSeverity.High,
AffectedPeriod = "2025-01-15 12:00-13:00",
Evidence = "Average 450s vs baseline 280s"
}
]
};
// Act
var alert = await _emitter.EmitAsync(spec, result);
// Assert
alert.Id.Should().NotBe(Guid.Empty);
alert.Type.Should().Be(EventType.AiAnomaly);
alert.Severity.Should().Be(EventSeverity.High);
alert.Name.Should().Contain("[Prep Time Monitor - Downtown]");
alert.Content.Should().Contain("## Findings");
alert.Content.Should().Contain("60% increase during 12:00-13:00");
alert.Content.Should().Contain("Evidence");
}
[Fact]
public async Task EmitAsync_ThrowsForNonAnomaly()
{
// Arrange
var spec = CreateTestSpec("Test Monitor");
var result = AnomalyDetectionResult.NoAnomaly();
// Act & Assert
var act = () => _emitter.EmitAsync(spec, result);
await act.Should().ThrowAsync<InvalidOperationException>();
}
[Fact]
public async Task EmitAsync_MultipleFindings_AllIncludedInContent()
{
// Arrange
var spec = CreateTestSpec("Multi-Finding Monitor");
var result = new AnomalyDetectionResult
{
AnomalyDetected = true,
Severity = EventSeverity.Medium,
Summary = "Multiple anomalies detected",
Findings =
[
new AnomalyFinding
{
Description = "First anomaly",
Severity = EventSeverity.Medium,
AffectedPeriod = "12:00-13:00",
Evidence = "Evidence for first"
},
new AnomalyFinding
{
Description = "Second anomaly",
Severity = EventSeverity.Low,
AffectedPeriod = "14:00-15:00",
Evidence = "Evidence for second"
}
]
};
// Act
var alert = await _emitter.EmitAsync(spec, result);
// Assert
alert.Content.Should().Contain("First anomaly");
alert.Content.Should().Contain("Second anomaly");
alert.Content.Should().Contain("Evidence for first");
alert.Content.Should().Contain("Evidence for second");
}
[Fact]
public async Task ShouldSuppressAsync_SuppressesDuplicateWithinWindow()
{
// Arrange
var spec = CreateTestSpec("Dedup Monitor");
var result = new AnomalyDetectionResult
{
AnomalyDetected = true,
Severity = EventSeverity.Medium,
Summary = "Test anomaly"
};
// Emit first alert
await _emitter.EmitAsync(spec, result);
// Act - check if second emission should be suppressed
var shouldSuppress = await _emitter.ShouldSuppressAsync(spec, result);
// Assert
shouldSuppress.Should().BeTrue();
}
[Fact]
public async Task ShouldSuppressAsync_DoesNotSuppressDifferentMonitors()
{
// Arrange
var spec1 = CreateTestSpec("Monitor A");
var spec2 = CreateTestSpec("Monitor B");
var result = new AnomalyDetectionResult
{
AnomalyDetected = true,
Severity = EventSeverity.Medium,
Summary = "Test anomaly"
};
// Emit alert for monitor A
await _emitter.EmitAsync(spec1, result);
// Act - check if monitor B should be suppressed
var shouldSuppress = await _emitter.ShouldSuppressAsync(spec2, result);
// Assert
shouldSuppress.Should().BeFalse();
}
// === Helper ===
private static MonitorSpec CreateTestSpec(string name) => new()
{
Id = Guid.NewGuid(),
Name = name,
SystemPrompt = "Test system prompt",
State = MonitorState.Active,
CreatedAt = DateTimeOffset.UtcNow
};
}4. backend/Kds.Ai.AnomalyDetector.Tests/UnitTests/AgentDetectionTests.cs
using FluentAssertions;
using Kds.Ai.AnomalyDetector.Agents;
using Kds.Ai.AnomalyDetector.Tests.Fakes;
using Kds.Ai.Data.Entities;
using Kds.Ai.Data.Enums;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace Kds.Ai.AnomalyDetector.Tests.UnitTests;
/// <summary>
/// Tests for the AnomalyDetectorAgent using FakeChatClient and FakeMcpToolProvider.
/// These exercise the agent's tool-calling loop without real LLM or MCP calls.
/// </summary>
public sealed class AgentDetectionTests
{
[Fact]
public async Task RunDetection_NoAnomaly_ReturnsNoAnomalyResult()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.NoAnomaly();
var agent = CreateAgent(chat, tools);
var spec = CreateTestSpec("No Anomaly Monitor");
// Act
var result = await agent.RunDetectionAsync(spec, DateTimeOffset.UtcNow);
// Assert
result.AnomalyDetected.Should().BeFalse();
result.Findings.Should().BeEmpty();
}
[Fact]
public async Task RunDetection_SingleAnomaly_ReturnsFindings()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.SingleAnomaly(EventSeverity.High);
var agent = CreateAgent(chat, tools);
var spec = CreateTestSpec("Anomaly Monitor");
// Act
var result = await agent.RunDetectionAsync(spec, DateTimeOffset.UtcNow);
// Assert
result.AnomalyDetected.Should().BeTrue();
result.Severity.Should().Be(EventSeverity.High);
result.Summary.Should().NotBeNullOrEmpty();
result.Findings.Should().NotBeEmpty();
result.Findings[0].AffectedPeriod.Should().Contain("12:00");
result.Findings[0].Evidence.Should().NotBeNullOrEmpty();
}
[Fact]
public async Task RunDetection_DataError_HandlesGracefully()
{
// Arrange
var (tools, chat) = ScriptedAgentScenarios.DataError();
var agent = CreateAgent(chat, tools);
var spec = CreateTestSpec("Error Monitor");
// Act & Assert
// The agent should either return a Failed result or throw,
// depending on implementation. Either is acceptable.
var act = async () => await agent.RunDetectionAsync(spec, DateTimeOffset.UtcNow);
// DataError scenario throws during MCP tool execution,
// so agent should handle it or let it propagate
await act.Should().ThrowAsync<Exception>();
}
[Fact]
public async Task RunDetection_UsesSpecSystemPrompt()
{
// Arrange - custom tools that record the query
var tools = new FakeMcpToolProvider()
.OnQuery("SELECT", "[]");
var chat = new FakeChatClient()
.EnqueueFunctionCall("emit_conclusion", """
{
"anomalyDetected": false,
"summary": "No data found",
"findings": []
}
""");
var agent = CreateAgent(chat, tools);
var spec = CreateTestSpec("Custom Prompt Monitor");
spec.SystemPrompt = "You are a special-purpose analyzer. Always check the widgets table.";
// Act
var result = await agent.RunDetectionAsync(spec, DateTimeOffset.UtcNow);
// Assert - the agent processed without error using the custom prompt
result.Should().NotBeNull();
result.AnomalyDetected.Should().BeFalse();
}
// === Helpers ===
private static IAnomalyDetectorAgent CreateAgent(
FakeChatClient chat,
FakeMcpToolProvider tools)
{
return new AnomalyDetectorAgent(
chat,
tools,
NullLogger<AnomalyDetectorAgent>.Instance);
}
private static MonitorSpec CreateTestSpec(string name) => new()
{
Id = Guid.NewGuid(),
Name = name,
SystemPrompt = "Analyze data and detect anomalies. Use mysql_query tool to access data.",
State = MonitorState.Active,
CreatedAt = DateTimeOffset.UtcNow
};
}5. backend/Kds.Ai.AnomalyDetector.Tests/UnitTests/MonitorSpecSeederTests.cs
using FluentAssertions;
using Kds.Ai.AnomalyDetector.Seeding;
using Kds.Ai.Data;
using Kds.Ai.Data.Enums;
using Kds.Ai.Data.Repositories;
using Kds.Ai.Data.Seeding;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging.Abstractions;
using Xunit;
namespace Kds.Ai.AnomalyDetector.Tests.UnitTests;
public sealed class MonitorSpecSeederTests : IAsyncLifetime
{
private KdsAiDbContext _dbContext = null!;
private MonitorSpecSeeder _seeder = null!;
public async Task InitializeAsync()
{
var options = new DbContextOptionsBuilder<KdsAiDbContext>()
.UseInMemoryDatabase($"SeederTest_{Guid.NewGuid()}")
.Options;
_dbContext = new KdsAiDbContext(options);
var repo = new EfMonitorSpecRepository(_dbContext, NullLogger<EfMonitorSpecRepository>.Instance);
_seeder = new MonitorSpecSeeder(repo, NullLogger<MonitorSpecSeeder>.Instance);
await _dbContext.Database.EnsureCreatedAsync();
}
public async Task DisposeAsync()
{
await _dbContext.DisposeAsync();
}
[Fact]
public async Task EnsureAsync_CreatesNewMonitor()
{
// Act
var spec = await _seeder.EnsureAsync("Test Monitor", "Test prompt");
// Assert
spec.Id.Should().NotBe(Guid.Empty);
spec.Name.Should().Be("Test Monitor");
spec.SystemPrompt.Should().Be("Test prompt");
spec.State.Should().Be(MonitorState.Draft); // Default state
}
[Fact]
public async Task EnsureAsync_ReturnsExistingByName()
{
// Arrange
var first = await _seeder.EnsureAsync("Duplicate Test", "First prompt");
// Act
var second = await _seeder.EnsureAsync("Duplicate Test", "Different prompt");
// Assert
second.Id.Should().Be(first.Id);
second.SystemPrompt.Should().Be("First prompt"); // Original preserved
}
[Fact]
public async Task EnsureAsync_RespectsInitialState()
{
// Act
var spec = await _seeder.EnsureAsync(
"Active Monitor",
"Test prompt",
initialState: MonitorState.Active);
// Assert
spec.State.Should().Be(MonitorState.Active);
}
[Fact]
public async Task EnsureAsync_ExamplePromptIsValid()
{
// Act
var spec = await _seeder.EnsureAsync(
"Prep Time Example",
ExamplePrompts.PrepTimeAnomaly);
// Assert
spec.SystemPrompt.Should().Contain("mysql_query");
spec.SystemPrompt.Should().Contain("preparation_time");
spec.SystemPrompt.Should().Contain("emit_conclusion");
}
}Running Tests
# Run all tests
dotnet test backend/Kds.Ai.AnomalyDetector.Tests
# Run with verbose output
dotnet test backend/Kds.Ai.AnomalyDetector.Tests --logger "console;verbosity=detailed"
# Run specific test class
dotnet test backend/Kds.Ai.AnomalyDetector.Tests --filter "FullyQualifiedName~OrchestratorIntegrationTests"
# Run unit tests only
dotnet test backend/Kds.Ai.AnomalyDetector.Tests --filter "FullyQualifiedName~UnitTests"Changes from Old Design
- Removed
Moqdependency - tests useFakeChatClientandFakeMcpToolProviderfrom TASK 6.1 - Removed
PrepTimeQueryBuilderTestsentirely -PrepTimeQueryBuilderwas deleted - Removed
Mock<IAnomalyDetectorAgentFactory>- no factory pattern, single generic agent - Removed
mock.SetupGet(a => a.MonitorType)- agent has noMonitorTypeproperty - Removed
PrepTimeMonitorSpecFactory.CreateDefault(storeGuid)- usesMonitorSpecSeeder.EnsureAsync()or inlineMonitorSpeccreation - Removed
spec.IsEnabled = false- usesspec.State = MonitorState.Paused - Removed
run.AlertEventIdassertions - usesrun.AlertEventscollection (many-to-many) - Removed
ExecuteAllByTypeAsync("PrepTimeAnomaly")- usesExecuteAllActiveAsync() - Removed
MockDataScenarioenum references - usesScriptedAgentScenariosmethods - Removed
AnomalyDetectionResultwithType,DeviationPercentfields - usesDescription,Severity,AffectedPeriod,Evidence - Added
AgentDetectionTeststhat exercise the agent directly with fakes - Added
MonitorSpecSeederTestsfor seeder idempotency - All test classes are
sealed EfMonitorSpecRepositoryuses primary constructor (dbContext + logger parameters)IMonitorSpecRepositoryimported fromKds.Ai.Data.Repositories(registered byAddDataModule())- Test project requires
InternalsVisibleTofrom bothKds.Ai.DataandKds.Ai.AnomalyDetectorto accessinternal sealedimplementations AnomalyDetectorAgenttakesIMcpToolProviderinterface (from TASK 6.1 refactor), allowingFakeMcpToolProvidersubstitution
Acceptance Criteria
- [ ] Test project created with correct references (no Moq)
- [ ]
InternalsVisibleToadded in bothKds.Ai.Data.csprojandKds.Ai.AnomalyDetector.csprojfor test project access tointernal sealedclasses - [ ] Integration tests for orchestrator complete flow using
FakeChatClientandFakeMcpToolProvider - [ ] Unit tests for agent detection using scripted scenarios
- [ ] Unit tests for alert emitter with generic findings
- [ ] Unit tests for seeder idempotency
- [ ] Tests use in-memory database
- [ ] Tests use
ScriptedAgentScenarios(not mock interfaces) - [ ]
IMonitorSpecRepositoryimported fromKds.Ai.Data.Repositories(notKds.Ai.AnomalyDetector.Repositories) - [ ]
IMonitorRunRepositoryimported fromKds.Ai.AnomalyDetector.Repositories - [ ]
EfMonitorSpecRepositoryinstantiated with primary constructor signature (dbContext + logger) - [ ]
AnomalyDetectorAgenttakesIMcpToolProvider(not concreteMcpToolProvider) —FakeMcpToolProvidersatisfies this via interface implementation - [ ] No references to:
IAnomalyDetectorAgentFactory,MonitorType,StoreGuid,IsEnabled,AlertEventId,PrepTimeQueryBuilder,PrepTimeMonitorSpecFactory,MockDataScenario,IKdsDataAccessService,IMcpClient - [ ] All uses
run.AlertEvents(collection, not single FK) - [ ] All uses
MonitorStateenum (notbool IsEnabled) - [ ] All test classes are
sealed - [ ] All tests pass
- [ ] Solution builds without errors
Notes
- Tests exercise the full agent tool-calling loop via fakes (not just mocking the agent interface)
FakeChatClientscripts are ordered: first call returns first response, etc.- In-memory database resets per test via unique DB names
- Consider adding tests for timeout handling and concurrent execution in the future
- The
AgentDetectionTestsverify the agent works with custom MonitorSpec prompts