From c29c94ab62185a425c63dc5ab52dbd88213a4ab6 Mon Sep 17 00:00:00 2001 From: AJ Date: Sun, 12 Oct 2025 16:34:44 -0400 Subject: [PATCH] Update ARCHITECTURE.md with recent receipt management features MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Updated documentation to reflect v1.2 changes: - Unmapped receipts with nullable TransactionId - Duplicate detection with blocking modal - Auto-mapping service with intelligent date range logic - Due date support for bills (bill date to due date + 5 days) - Enhanced manual mapping UI with transaction selector - Transactions page improvements (search, filters, ID column) - New Receipts page with comprehensive workflow - Updated service layer, database schema, and workflows sections 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- ARCHITECTURE.md | 254 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 237 insertions(+), 17 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 1004a99..1f0f216 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -115,7 +115,7 @@ Stores uploaded receipt files (images/PDFs) linked to transactions. **Properties:** - `Id` (long) - Primary key -- `TransactionId` (long) - Foreign key to Transaction +- `TransactionId` (long?) - Foreign key to Transaction (nullable to support unmapped receipts) - `FileName` (string, 260) - Original file name (sanitized) - `ContentType` (string, 100) - MIME type - `StoragePath` (string, 1024) - Relative path in wwwroot @@ -126,6 +126,7 @@ Stores uploaded receipt files (images/PDFs) linked to transactions. **Parsed Fields (populated by AI parser):** - `Merchant` (string, 200) - Merchant name extracted from receipt - `ReceiptDate` (DateTime?) - Date on receipt +- `DueDate` (DateTime?) - Payment due date (for bills only) - `Subtotal`, `Tax`, `Total` (decimal?) - Monetary amounts - `Currency` (string, 8) - Currency code @@ -133,7 +134,7 @@ Stores uploaded receipt files (images/PDFs) linked to transactions. - `ParseLogs` - Collection of parse attempts - `LineItems` - Collection of receipt line items -**Unique Index:** TransactionId + FileHashSha256 (prevents duplicate uploads) +**Unique Index:** TransactionId + FileHashSha256 (prevents duplicate uploads, filtered WHERE TransactionId IS NOT NULL) ### ReceiptLineItem (Models/ReceiptLineItem.cs) Individual line items extracted from receipts. @@ -286,16 +287,25 @@ Pattern-based rules for auto-categorization with merchant linking. ### ReceiptManager (Services/ReceiptManager.cs) **Interface:** `IReceiptManager` -**Responsibility:** Handle receipt file uploads and storage. +**Responsibility:** Handle receipt file uploads, storage, and transaction mapping. **Key Methods:** - `UploadReceiptAsync(long transactionId, IFormFile file)` - Validates file (10MB max, allowed extensions: jpg, jpeg, png, pdf, gif, heic) - Computes SHA256 hash for deduplication + - Checks for duplicates (same hash or same filename + size) - Saves file to `wwwroot/receipts/{transactionId}_{guid}{ext}` - Sanitizes filename (removes non-ASCII like ®, ™, ©) - Creates Receipt entity in database - - Returns `ReceiptUploadResult` + - Returns `ReceiptUploadResult` with duplicate warnings + +- `UploadUnmappedReceiptAsync(IFormFile file)` + - Same as UploadReceiptAsync but for receipts without initial transaction mapping + - TransactionId is null until later mapped + +- `MapReceiptToTransactionAsync(long receiptId, long transactionId)` + - Links an unmapped receipt to a transaction + - Returns success boolean - `DeleteReceiptAsync(long receiptId)` - Deletes physical file from disk @@ -304,6 +314,12 @@ Pattern-based rules for auto-categorization with merchant linking. - `GetReceiptPhysicalPath(Receipt receipt)` - Resolves storage path - `GetReceiptAsync(long receiptId)` - Retrieves receipt with transaction +**Duplicate Detection:** +- SHA256 hash matching (identical files) +- Filename + size matching (possible duplicates) +- Returns `DuplicateWarning` list with existing receipts +- User can confirm upload anyway via modal dialog + **Security Features:** - File size validation (10MB limit) - Extension whitelist @@ -322,10 +338,11 @@ Pattern-based rules for auto-categorization with merchant linking. - Loads receipt file from disk - Converts PDFs to PNG images using ImageMagick (220 DPI) - Calls OpenAI Vision API with structured prompt - - Parses JSON response (merchant, date, amounts, line items) + - Parses JSON response (merchant, date, due date, amounts, line items) - Updates Receipt entity with extracted data - Replaces existing line items - Logs parse attempt in ReceiptParseLog + - Attempts auto-mapping if receipt is unmapped - Returns `ReceiptParseResult` **API Configuration:** @@ -336,16 +353,69 @@ Pattern-based rules for auto-categorization with merchant linking. **Prompt Strategy:** - Structured JSON request with schema example -- Extracts: merchant, date, subtotal, tax, total, confidence +- Extracts: merchant, date, dueDate (for bills), subtotal, tax, total, confidence - Line items with: description, quantity, unitPrice, lineTotal - Special handling: Services/fees have null quantity (not products) +- Due date extraction: For bills (utility, credit card, etc.), extracts payment due date **PDF Handling:** - ImageMagick converts first page to PNG at 220 DPI - Flattens alpha channel (white background) - TrueColor 8-bit RGB output -**Location:** Services/OpenAIReceiptParser.cs:23-302 +**Auto-Mapping Integration:** +- After successful parse of unmapped receipts, triggers ReceiptAutoMapper +- Attempts to automatically link receipt to matching transaction +- Silently fails if auto-mapping unsuccessful (parsing still successful) + +**Location:** Services/OpenAIReceiptParser.cs:23-342 + +### ReceiptAutoMapper (Services/ReceiptAutoMapper.cs) +**Interface:** `IReceiptAutoMapper` + +**Responsibility:** Automatically map parsed receipts to matching transactions using smart matching rules. + +**Key Methods:** +- `AutoMapReceiptAsync(long receiptId)` + - Attempts to match a single receipt to a transaction + - Uses date range, merchant name, and amount matching + - Returns `ReceiptAutoMapResult` with status (Success, NoMatch, MultipleMatches, AlreadyMapped, NotParsed, Failed) + +- `AutoMapUnmappedReceiptsAsync()` + - Bulk processes all unmapped parsed receipts + - Returns `BulkAutoMapResult` with counts (total, mapped, no match, multiple matches) + +**Matching Algorithm:** +1. **Date Range Filtering:** + - For bills with due dates: `billDate` to `dueDate + 5 days` (accounts for auto-pay delays, weekends, bank processing) + - For regular receipts: `receiptDate ± 3 days` (accounts for transaction posting delays) + - No date = no match (can't narrow down effectively) + +2. **Merchant Filtering (optional):** + - Matches if merchant name or transaction name contains receipt merchant + - Case-insensitive contains matching + +3. **Amount Matching (optional):** + - If receipt has total: matches transactions within ±$0.10 tolerance + - Accounts for rounding differences and tips + +4. **Receipt Exclusion:** + - Excludes transactions that already have receipts attached + - Uses HashSet for performance + +**Auto-Mapping Triggers:** +- Automatically triggered after successful receipt parsing (if receipt is unmapped) +- Can be manually triggered via "Auto-Map Unmapped Receipts" button on Receipts page + +**Result Status Types:** +- `Success` - Single match found, receipt mapped automatically +- `NoMatch` - No matching transactions found +- `MultipleMatches` - Multiple potential matches found (requires manual selection) +- `AlreadyMapped` - Receipt already linked to a transaction +- `NotParsed` - Receipt has no parsed data (merchant, date, or total missing) +- `Failed` - Error during mapping process + +**Location:** Services/ReceiptAutoMapper.cs ### TransactionAICategorizer (Services/TransactionAICategorizer.cs) **Interface:** `ITransactionAICategorizer` @@ -457,18 +527,69 @@ EF Core DbContext managing all database entities. ### Transactions.cshtml / TransactionsModel **Route:** `/Transactions` -**Purpose:** View and search all transactions. +**Purpose:** View and search all transactions with advanced filtering. **Features:** -- Search by name or memo -- Filter by card, category, date range +- Search by name, memo, category, notes, or merchant +- Filter by card, category, merchant, date range +- Quick date range selectors (30/60/90 days, last year, this month, last month) - Sort by date, amount, name, category -- Pagination -- Receipt count indicator +- Pagination with preserved filters +- Transaction ID display for easy reference +- Receipt count indicator with badge +- Upload CSV button in header - Edit transaction link +**Stats Display:** +- Total transaction count +- Current page showing count +- Total spent (debits sum) +- Total income (credits sum) +- Net amount with color coding + **Location:** Pages/Transactions.cshtml.cs +### Receipts.cshtml / ReceiptsModel +**Route:** `/Receipts` + +**Purpose:** Manage receipts with upload, mapping, and auto-mapping capabilities. + +**Features:** +- Upload receipts without initial transaction mapping (unmapped receipts) +- Duplicate detection with blocking modal dialog + - SHA256 hash matching (identical files) + - Filename + size matching (possible duplicates) + - User confirmation required to proceed +- View all receipts (mapped and unmapped) with status badges +- Manual receipt-to-transaction mapping via modal selector +- Auto-map unmapped receipts button (bulk operation) +- Smart transaction matching with filters +- Receipt info display: merchant, date, due date (for bills), total +- Delete receipts +- View receipt details + +**Transaction Selector Modal:** +- Displays up to 50 matching transactions in scrollable table +- Date range filtering: + - Bills with due dates: bill date to due date + 5 days + - Regular receipts: receipt date ± 3 days +- Merchant-based relevance sorting (exact match > contains > no match) +- Amount matching with green highlighting (±$0.10 tolerance) +- Auto-scroll to first amount match when modal opens +- Manual transaction ID entry option +- Link to open Transactions page for searching + +**Receipt List Columns:** +- Uploaded timestamp +- File name with icon (image/PDF) +- Receipt info (merchant, date, due date, total) +- Mapped transaction details or "Unmapped" badge +- Actions: View, Map (if unmapped), Delete + +**Dependencies:** `IReceiptManager`, `IReceiptAutoMapper` + +**Location:** Pages/Receipts.cshtml.cs + ### EditTransaction.cshtml / EditTransactionModel **Route:** `/EditTransaction/{id}` @@ -615,6 +736,8 @@ builder.Services.AddScoped(); builder.Services.AddHttpClient(); +builder.Services.AddScoped(); +builder.Services.AddScoped(); ``` **Location:** Program.cs:1-44 @@ -668,7 +791,7 @@ INDEX: Date, Amount, Category ### Receipts Table ```sql Id (bigint, PK) -TransactionId (bigint, FK → Transactions.Id, CASCADE) +TransactionId (bigint, FK → Transactions.Id, CASCADE, NULLABLE) FileName (nvarchar(260), NOT NULL) ContentType (nvarchar(100), DEFAULT 'application/octet-stream') StoragePath (nvarchar(1024), NOT NULL) @@ -677,11 +800,12 @@ FileHashSha256 (nvarchar(64), NOT NULL) UploadedAtUtc (datetime2, NOT NULL) Merchant (nvarchar(200)) ReceiptDate (datetime2) +DueDate (datetime2) -- For bills: payment due date Subtotal (decimal(18,2)) Tax (decimal(18,2)) Total (decimal(18,2)) Currency (nvarchar(8)) -UNIQUE INDEX: (TransactionId, FileHashSha256) +UNIQUE INDEX: (TransactionId, FileHashSha256) WHERE TransactionId IS NOT NULL ``` ### ReceiptParseLogs Table @@ -766,7 +890,7 @@ Iterate mappings: Update Transaction.Category and Transaction.MerchantId ``` -### 3. Upload and Parse Receipt +### 3. Upload and Parse Receipt (Traditional Flow - From Transaction) ``` User attaches receipt to transaction @@ -790,13 +914,83 @@ OpenAIReceiptParser.ParseReceiptAsync() - Encode as base64 - Call OpenAI Vision API with structured prompt - Parse JSON response - - Update Receipt (merchant, date, amounts) + - Update Receipt (merchant, date, due date, amounts) - Replace ReceiptLineItems - Create ReceiptParseLog ↓ Display parsed data ``` +### 3a. Upload, Parse, and Auto-Map Receipt (New Unmapped Flow) + +``` +User visits /Receipts page + ↓ +User uploads receipt without transaction + ↓ +ReceiptsModel.OnPostUploadAsync() + ↓ +ReceiptManager.UploadUnmappedReceiptAsync() + - Validate file (size, extension) + - Compute SHA256 hash + - Check for duplicates (hash, filename+size) + ↓ +IF duplicates found: + - Delete uploaded file temporarily + - Store duplicate warnings in TempData + - Redirect to GET with modal + ↓ + User sees blocking modal: + Option A: Cancel (don't upload) + Option B: Upload Anyway (re-upload with confirmation) + ↓ +IF no duplicates OR confirmed: + - Save to wwwroot/receipts/ + - Create Receipt entity (TransactionId = null) + ↓ +User clicks "Parse Receipt" + ↓ +OpenAIReceiptParser.ParseReceiptAsync() + - Parse receipt (merchant, date, due date, total, line items) + - Update Receipt entity + - Trigger ReceiptAutoMapper.AutoMapReceiptAsync() + ↓ +ReceiptAutoMapper.AutoMapReceiptAsync() + - Find matching transactions: + * Date filter (bill date to due date+5 OR receipt date ±3) + * Merchant filter (contains, case-insensitive) + * Amount filter (±$0.10 tolerance) + * Exclude transactions with existing receipts + ↓ + IF single match found: + - Auto-map receipt to transaction + - Return Success + ELSE IF multiple matches: + - Return MultipleMatches (requires manual selection) + ELSE: + - Return NoMatch + ↓ +IF auto-mapping failed (no match or multiple matches): + User clicks "Map" button on receipt row + ↓ + ReceiptsModel.FindMatchingTransactionsForReceipt() + - Apply same filtering logic + - Sort by merchant relevance (exact > contains > none) + - Return up to 50 matches + ↓ + Modal displays transactions in table: + - Green highlighting for amount matches + - Auto-scroll to first amount match + - Radio button selection + - Manual ID entry option + ↓ + User selects transaction + ↓ + ReceiptsModel.OnPostMapToTransactionAsync() + - Links receipt to transaction + - Redirect to receipts page +``` + ### 4. Dashboard Aggregation ``` @@ -1044,5 +1238,31 @@ MoneyMap demonstrates a well-architected ASP.NET Core application with clear sep --- **Last Updated:** 2025-10-12 -**Version:** 1.1 +**Version:** 1.2 **Framework:** ASP.NET Core 8.0 / EF Core 9.0 + +## Recent Changes (v1.2) + +### Receipt Management Enhancements +- **Unmapped Receipts**: Receipts can now be uploaded without initial transaction mapping +- **Duplicate Detection**: SHA256 hash and filename+size matching with blocking modal confirmation +- **Auto-Mapping**: Smart automatic receipt-to-transaction matching using date, merchant, and amount +- **Due Date Support**: Bills (utility, credit card) can have due dates for expanded matching windows +- **Manual Mapping UI**: Enhanced modal with transaction table, relevance sorting, amount highlighting, and auto-scroll +- **Date Range Intelligence**: + - Bills: bill date to due date + 5 days (accounts for auto-pay processing) + - Regular receipts: receipt date ± 3 days + +### UI Improvements +- **Transactions Page**: Added search, quick date filters, transaction ID column, Upload CSV button in header +- **Navigation**: Moved Upload CSV from menu to Transactions page header +- **Receipts Page**: New dedicated page for receipt management with mapping workflow +- **Transaction Selector**: Scrollable table with green amount match highlighting and auto-scroll to best match + +### Technical Improvements +- Made `Receipt.TransactionId` nullable with filtered unique index +- Added `Receipt.DueDate` column for bill payment tracking +- Enhanced OpenAI prompt to extract due dates from bills +- Implemented `ReceiptAutoMapper` service with intelligent matching algorithm +- Updated `ReceiptManager` with unmapped receipt support and duplicate detection +- Added `MerchantService` for merchant management