Docs: Update ARCHITECTURE.md for tool-use, parse queue, and bulk upload

Document AI tool-use framework, receipt parse queue system, background worker, bulk upload, ParseStatus enum, and updated AIReceiptParser flow with tool-aware vs enriched-prompt paths. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 19:14:35 -05:00
parent 516546b345
commit e6512f9b7f
1 changed files with 189 additions and 24 deletions
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -41,6 +41,11 @@ MoneyMap follows a clean, service-oriented architecture:
 │    - ReceiptMatchingService (NEW)           │
 │    - ReceiptManager                         │
 │    - AIReceiptParser                        │
+│    - ReceiptParseQueue (singleton)          │
+│    - ReceiptParseWorkerService (hosted)     │
+│  AI Tool Use:                               │
+│    - AIToolExecutor (DB query tools)        │
+│    - AIToolRegistry (tool definitions)      │
 │  Reference & Dashboard:                     │
 │    - ReferenceDataService (NEW)             │
 │    - DashboardService                       │
@@ -134,6 +139,9 @@ Stores uploaded receipt files (images/PDFs) linked to transactions.
 - `FileHashSha256` (string, 64) - SHA256 hash for deduplication
 - `UploadedAtUtc` (DateTime) - Upload timestamp

+**Parse Queue Status:**
+- `ParseStatus` (ReceiptParseStatus enum) - NotRequested(0), Queued(1), Parsing(2), Completed(3), Failed(4)
+
 **Parsed Fields (populated by AI parser):**
 - `Merchant` (string, 200) - Merchant name extracted from receipt
 - `ReceiptDate` (DateTime?) - Date on receipt
@@ -629,6 +637,11 @@ Represents a spending budget for a category or total spending.
  - Same as UploadReceiptAsync but for receipts without initial transaction mapping
  - TransactionId is null until later mapped

+- `UploadManyUnmappedReceiptsAsync(IReadOnlyList<IFormFile> files)`
+  - Bulk upload multiple receipt files
+  - Calls UploadReceiptInternalAsync for each file, collecting results
+  - Returns `BulkUploadResult` with lists of uploaded items and failures
+
 - `MapReceiptToTransactionAsync(long receiptId, long transactionId)`
  - Links an unmapped receipt to a transaction
  - Returns success boolean
@@ -654,35 +667,62 @@ Represents a spending budget for a category or total spending.

 **Location:** Services/ReceiptManager.cs:23-199

-### OpenAIReceiptParser (Services/OpenAIReceiptParser.cs)
+### AIReceiptParser (Services/AIReceiptParser.cs)
 **Interface:** `IReceiptParser`

-**Responsibility:** Parse receipts using OpenAI GPT-4o-mini Vision API.
+**Responsibility:** Parse receipts using AI vision APIs with tool-use support for database-aware parsing.

 **Key Methods:**
- `ParseReceiptAsync(long receiptId)`
+- `ParseReceiptAsync(long receiptId, string? model, string? notes)`
  - Loads receipt file from disk
  - Converts PDFs to PNG images using ImageMagick (220 DPI)
-  - Calls OpenAI Vision API with structured prompt
-  - Parses JSON response (merchant, date, due date, amounts, line items)
+  - Resolves vision client based on model prefix (openai, claude-, llamacpp:, ollama:)
+  - **Tool-aware clients (OpenAI, Claude, LlamaCpp):** Uses function calling to let the AI query the database during parsing
+  - **Non-tool clients (Ollama):** Pre-fetches database context and injects it into the prompt
+  - Parses JSON response (merchant, date, due date, amounts, line items, suggestedCategory, suggestedTransactionId)
  - Updates Receipt entity with extracted data
-  - Replaces existing line items
+  - Populates `ReceiptLineItem.Category` from AI response
+  - If AI suggested a transaction ID, attempts direct mapping before falling back to ReceiptAutoMapper
  - Logs parse attempt in ReceiptParseLog
-  - Attempts auto-mapping if receipt is unmapped
  - Returns `ReceiptParseResult`

-**API Configuration:**
- Model: `gpt-4o-mini`
- Temperature: 0.1 (deterministic)
- Max tokens: 2000
- API key: Environment variable `OPENAI_API_KEY` or config `OpenAI:ApiKey`
+**Tool-Use Flow (OpenAI, Claude, LlamaCpp):**
+```
+AI sees receipt image + prompt with tool instructions
+    ↓
+AI calls search_categories → gets existing categories from DB
+AI calls search_transactions → finds matching bank transactions
+AI calls search_merchants → normalizes merchant name
+    ↓ (up to 5 tool rounds)
+AI returns final JSON with:
+  - Standard fields (merchant, date, total, lineItems)
+  - suggestedCategory (from existing categories)
+  - suggestedTransactionId (matched transaction)
+  - Per-line-item category
+```

-**Prompt Strategy:**
- Structured JSON request with schema example
- Extracts: merchant, date, dueDate (for bills), subtotal, tax, total, confidence
- Line items with: description, quantity, unitPrice, lineTotal
- Special handling: Services/fees have null quantity (not products)
- Due date extraction: For bills (utility, credit card, etc.), extracts payment due date
+**Ollama Fallback (enriched prompt):**
+```
+Pre-fetch all categories, matching merchants, candidate transactions
+    ↓
+Inject as text block in prompt
+    ↓
+AI returns JSON using the provided context
+```
+
+**Supported Providers:**
+| Provider | Model Prefix | Tool Use | Wire Format |
+|----------|-------------|----------|-------------|
+| OpenAI | (default) | Native | OpenAI /v1/chat/completions |
+| Anthropic | claude- | Native | Anthropic /v1/messages |
+| LlamaCpp | llamacpp: | Native | OpenAI-compatible /v1/chat/completions |
+| Ollama | ollama: | Enriched prompt fallback | /api/generate |
+
+**Response Fields (ParsedReceiptData):**
+- `Merchant`, `ReceiptDate`, `DueDate`, `Subtotal`, `Tax`, `Total`, `Confidence` - Standard fields
+- `SuggestedCategory` (NEW) - AI's best category for the overall receipt
+- `SuggestedTransactionId` (NEW) - Transaction ID the AI thinks matches this receipt
+- `LineItems[].Category` (NEW) - Per-line-item category

 **PDF Handling:**
 - ImageMagick converts first page to PNG at 220 DPI
@@ -690,11 +730,47 @@ Represents a spending budget for a category or total spending.
 - TrueColor 8-bit RGB output

 **Auto-Mapping Integration:**
- After successful parse of unmapped receipts, triggers ReceiptAutoMapper
- Attempts to automatically link receipt to matching transaction
+- If AI suggests a specific transaction, attempts direct mapping first
+- Falls back to ReceiptAutoMapper if AI mapping fails or no suggestion
 - Silently fails if auto-mapping unsuccessful (parsing still successful)

-**Location:** Services/OpenAIReceiptParser.cs:23-342
+**Location:** Services/AIReceiptParser.cs
+
+### AIToolExecutor (Services/AITools/AIToolExecutor.cs)
+**Interface:** `IAIToolExecutor`
+
+**Responsibility:** Execute AI tool calls against the database during receipt parsing.
+
+**Tools Available:**
+| Tool | Purpose | Parameters |
+|------|---------|------------|
+| `search_categories` | Find existing categories with patterns and merchants | `query?` (optional filter) |
+| `search_transactions` | Find unmapped bank transactions | `merchant?`, `minDate?`, `maxDate?`, `minAmount?`, `maxAmount?`, `limit?` |
+| `search_merchants` | Look up known merchants | `query` (required) |
+
+**Key Methods:**
+- `ExecuteAsync(AIToolCall)` - Dispatches to the correct handler, runs EF Core query, returns JSON result
+- `GetEnrichedContextAsync(receiptDate?, total?, merchantHint?)` - Pre-fetches all data as text block for Ollama fallback
+
+**Design Constraints:**
+- All tools are **read-only** database queries
+- Results capped at 20 items
+- Tool rounds capped at 5 per parse request
+- Transactions already mapped to receipts are excluded from search results
+
+**Location:** Services/AITools/AIToolExecutor.cs
+
+### AIToolRegistry (Services/AITools/AIToolDefinitions.cs)
+
+**Responsibility:** Define tool schemas in a provider-agnostic format.
+
+**Key Types:**
+- `AIToolDefinition` - Tool name, description, parameters
+- `AIToolParameter` - Parameter name, type, description, required flag
+- `AIToolCall` - Incoming tool call with arguments
+- `AIToolResult` - Tool execution result returned to the AI
+
+**Location:** Services/AITools/AIToolDefinitions.cs

 ### ReceiptAutoMapper (Services/ReceiptAutoMapper.cs)
 **Interface:** `IReceiptAutoMapper`
@@ -743,6 +819,30 @@ Represents a spending budget for a category or total spending.

 **Location:** Services/ReceiptAutoMapper.cs

+### ReceiptParseQueue (Services/ReceiptParseQueue.cs)
+**Interface:** `IReceiptParseQueue`
+**Lifetime:** Singleton
+
+**Responsibility:** Thread-safe queue for receipt parsing jobs using `System.Threading.Channels`.
+
+**Key Members:**
+- `EnqueueAsync(long receiptId)` - Add a receipt to the parse queue
+- `EnqueueManyAsync(IEnumerable<long> receiptIds)` - Bulk enqueue
+- `DequeueAsync(CancellationToken ct)` - Wait for and dequeue the next receipt
+- `QueueLength` - Current number of items waiting
+- `CurrentlyProcessingId` - Receipt ID currently being parsed (thread-safe via `Interlocked`)
+
+### ReceiptParseWorkerService (Services/ReceiptParseWorkerService.cs)
+**Type:** `BackgroundService` (hosted service)
+
+**Responsibility:** Continuously process receipts from the parse queue using the AI parser.
+
+**Behavior:**
+1. **Startup Recovery:** Queries DB for receipts with `ParseStatus == Queued || Parsing`, re-enqueues them in upload order
+2. **Processing Loop:** Dequeues one receipt at a time, sets status to `Parsing`, calls `IReceiptParser.ParseReceiptAsync`, updates status to `Completed` or `Failed`
+3. **Status Updates:** Uses separate DB scopes for status writes to guarantee persistence even if parsing throws
+4. **Graceful Shutdown:** Respects `CancellationToken` from host
+
 ### FinancialAuditService (Services/FinancialAuditService.cs)
 **Interface:** `IFinancialAuditService`

@@ -952,6 +1052,29 @@ EF Core DbContext managing all database entities.

 **Location:** Pages/Receipts.cshtml.cs

+### BulkReceiptUpload.cshtml / BulkReceiptUploadModel
+**Route:** `/BulkReceiptUpload`
+
+**Purpose:** Upload multiple receipt files at once with a queue dashboard showing parse progress.
+
+**Features:**
+- Multi-file upload form with file list preview and upload spinner
+- Queue dashboard with tabs: Queued (with position), Completed (merchant/total/confidence/line items), Failed (error + retry button)
+- Currently-processing indicator with spinner
+- AJAX polling every 3 seconds while items are active (auto-stops when idle)
+- Each receipt links to `/ViewReceipt/{id}`
+- Retry button for failed receipts re-queues them
+
+**Handlers:**
+- `OnGetAsync()` - Loads queue dashboard data
+- `OnPostUploadAsync(List<IFormFile> files)` - Calls `UploadManyUnmappedReceiptsAsync`
+- `OnGetQueueStatusAsync()` - Returns JSON for AJAX polling
+- `OnPostRetryAsync(long receiptId)` - Re-queues a failed receipt
+
+**Dependencies:** `IReceiptManager`, `IReceiptParseQueue`
+
+**Location:** Pages/BulkReceiptUpload.cshtml.cs
+
 ### EditTransaction.cshtml / EditTransactionModel
 **Route:** `/EditTransaction/{id}`

@@ -1170,8 +1293,16 @@ builder.Services.AddScoped<IRecentTransactionsProvider, RecentTransactionsProvid

 // Receipt Services
 builder.Services.AddScoped<IReceiptManager, ReceiptManager>();
-builder.Services.AddHttpClient<IReceiptParser, OpenAIReceiptParser>();
 builder.Services.AddScoped<IReceiptAutoMapper, ReceiptAutoMapper>();
+
+// AI Vision Clients and Tool Use
+builder.Services.AddHttpClient<OpenAIVisionClient>();
+builder.Services.AddHttpClient<ClaudeVisionClient>();
+builder.Services.AddHttpClient<OllamaVisionClient>();
+builder.Services.AddHttpClient<LlamaCppVisionClient>();
+builder.Services.AddScoped<IAIVisionClientResolver, AIVisionClientResolver>();
+builder.Services.AddScoped<IAIToolExecutor, AIToolExecutor>();
+builder.Services.AddScoped<IReceiptParser, AIReceiptParser>();
 builder.Services.AddScoped<IMerchantService, MerchantService>();
 ```

@@ -1240,7 +1371,9 @@ Subtotal (decimal(18,2))
 Tax (decimal(18,2))
 Total (decimal(18,2))
 Currency (nvarchar(8))
+ParseStatus (int, NOT NULL, DEFAULT 0) -- 0=NotRequested, 1=Queued, 2=Parsing, 3=Completed, 4=Failed
 UNIQUE INDEX: (TransactionId, FileHashSha256) WHERE TransactionId IS NOT NULL
+INDEX: ParseStatus
 ```

 ### ReceiptParseLogs Table
@@ -1683,10 +1816,31 @@ MoneyMap demonstrates a well-architected ASP.NET Core application with clear sep

 ---

-**Last Updated:** 2025-12-15
-**Version:** 1.4
+**Last Updated:** 2026-02-11
+**Version:** 1.5
 **Framework:** ASP.NET Core 8.0 / EF Core 9.0

+## Recent Changes (v1.5)
+
+### AI Tool Use for Receipt Parsing
+- **Tool-Use Support**: AI can now query the database during receipt parsing via function calling
+- **Three Tools**: `search_categories`, `search_transactions`, `search_merchants` - all read-only DB queries
+- **Multi-Provider**: OpenAI, Claude, and LlamaCpp all support native tool calling; Ollama uses enriched prompt fallback
+- **New Response Fields**: `suggestedCategory` (overall receipt), `suggestedTransactionId` (matched transaction), per-line-item `category`
+- **AI-Suggested Mapping**: If the AI identifies a matching transaction, it's mapped directly before falling back to the scoring-based ReceiptAutoMapper
+- **Category Population**: `ReceiptLineItem.Category` is now populated from AI responses (was previously unused)
+- **Shared Tool-Use Helper**: `OpenAIToolUseHelper` implements the OpenAI-compatible tool-use loop shared by OpenAI and LlamaCpp clients
+- **Anthropic Tool Use**: `ClaudeVisionClient` implements Anthropic-specific `tool_use`/`tool_result` content block format
+
+### New Files
+- `Services/AITools/AIToolDefinitions.cs` - Provider-agnostic tool schema models and registry
+- `Services/AITools/AIToolExecutor.cs` - Tool executor with database query handlers
+
+### Modified Files
+- `Services/AIVisionClient.cs` - Added `IAIToolAwareVisionClient` interface, `OpenAIToolUseHelper`, tool-use implementations
+- `Services/AIReceiptParser.cs` - Integrated tool executor, new response fields, enriched prompt fallback
+- `Program.cs` - Registered `IAIToolExecutor` and `IAIVisionClientResolver`
+
 ## Recent Changes (v1.4)

 ### Financial Audit API
@@ -1744,3 +1898,14 @@ MoneyMap demonstrates a well-architected ASP.NET Core application with clear sep
 - Implemented `ReceiptAutoMapper` service with intelligent matching algorithm
 - Updated `ReceiptManager` with unmapped receipt support and duplicate detection
 - Added `MerchantService` for merchant management
+
+## Recent Changes (v1.3)
+
+### Bulk Receipt Upload & Parse Queue
+- **ReceiptParseStatus**: New enum on `Receipt` model tracking parse lifecycle (NotRequested → Queued → Parsing → Completed/Failed)
+- **ReceiptParseQueue**: Singleton `Channel<long>`-based queue service replacing fire-and-forget `Task.Run` parsing
+- **ReceiptParseWorkerService**: `BackgroundService` that processes parse queue sequentially, with startup recovery for interrupted items
+- **Bulk Upload Page**: New `/BulkReceiptUpload` page with multi-file upload, queue dashboard (tabbed: Queued/Completed/Failed), AJAX polling, and retry for failed items
+- **ReceiptManager.UploadManyUnmappedReceiptsAsync**: New bulk upload method with per-file error handling
+- **ViewReceipt LLM Response**: Collapsible raw LLM response payload in parse log history
+- **Unified Queue**: Both single and bulk receipt uploads now go through the same parse queue