Update ARCHITECTURE.md with recent receipt management features

Updated documentation to reflect v1.2 changes: - Unmapped receipts with nullable TransactionId - Duplicate detection with blocking modal - Auto-mapping service with intelligent date range logic - Due date support for bills (bill date to due date + 5 days) - Enhanced manual mapping UI with transaction selector - Transactions page improvements (search, filters, ID column) - New Receipts page with comprehensive workflow - Updated service layer, database schema, and workflows sections 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-12 16:34:44 -04:00
parent c4c01ce8c6
commit c29c94ab62
1 changed files with 237 additions and 17 deletions
--- a/ARCHITECTURE.md
+++ b/ARCHITECTURE.md
@@ -115,7 +115,7 @@ Stores uploaded receipt files (images/PDFs) linked to transactions.

 **Properties:**
 - `Id` (long) - Primary key
- `TransactionId` (long) - Foreign key to Transaction
+- `TransactionId` (long?) - Foreign key to Transaction (nullable to support unmapped receipts)
 - `FileName` (string, 260) - Original file name (sanitized)
 - `ContentType` (string, 100) - MIME type
 - `StoragePath` (string, 1024) - Relative path in wwwroot
@@ -126,6 +126,7 @@ Stores uploaded receipt files (images/PDFs) linked to transactions.
 **Parsed Fields (populated by AI parser):**
 - `Merchant` (string, 200) - Merchant name extracted from receipt
 - `ReceiptDate` (DateTime?) - Date on receipt
+- `DueDate` (DateTime?) - Payment due date (for bills only)
 - `Subtotal`, `Tax`, `Total` (decimal?) - Monetary amounts
 - `Currency` (string, 8) - Currency code

@@ -133,7 +134,7 @@ Stores uploaded receipt files (images/PDFs) linked to transactions.
 - `ParseLogs` - Collection of parse attempts
 - `LineItems` - Collection of receipt line items

-**Unique Index:** TransactionId + FileHashSha256 (prevents duplicate uploads)
+**Unique Index:** TransactionId + FileHashSha256 (prevents duplicate uploads, filtered WHERE TransactionId IS NOT NULL)

 ### ReceiptLineItem (Models/ReceiptLineItem.cs)
 Individual line items extracted from receipts.
@@ -286,16 +287,25 @@ Pattern-based rules for auto-categorization with merchant linking.
 ### ReceiptManager (Services/ReceiptManager.cs)
 **Interface:** `IReceiptManager`

-**Responsibility:** Handle receipt file uploads and storage.
+**Responsibility:** Handle receipt file uploads, storage, and transaction mapping.

 **Key Methods:**
 - `UploadReceiptAsync(long transactionId, IFormFile file)`
  - Validates file (10MB max, allowed extensions: jpg, jpeg, png, pdf, gif, heic)
  - Computes SHA256 hash for deduplication
+  - Checks for duplicates (same hash or same filename + size)
  - Saves file to `wwwroot/receipts/{transactionId}_{guid}{ext}`
  - Sanitizes filename (removes non-ASCII like ®, ™, ©)
  - Creates Receipt entity in database
-  - Returns `ReceiptUploadResult`
+  - Returns `ReceiptUploadResult` with duplicate warnings
+
+- `UploadUnmappedReceiptAsync(IFormFile file)`
+  - Same as UploadReceiptAsync but for receipts without initial transaction mapping
+  - TransactionId is null until later mapped
+
+- `MapReceiptToTransactionAsync(long receiptId, long transactionId)`
+  - Links an unmapped receipt to a transaction
+  - Returns success boolean

 - `DeleteReceiptAsync(long receiptId)`
  - Deletes physical file from disk
@@ -304,6 +314,12 @@ Pattern-based rules for auto-categorization with merchant linking.
 - `GetReceiptPhysicalPath(Receipt receipt)` - Resolves storage path
 - `GetReceiptAsync(long receiptId)` - Retrieves receipt with transaction

+**Duplicate Detection:**
+- SHA256 hash matching (identical files)
+- Filename + size matching (possible duplicates)
+- Returns `DuplicateWarning` list with existing receipts
+- User can confirm upload anyway via modal dialog
+
 **Security Features:**
 - File size validation (10MB limit)
 - Extension whitelist
@@ -322,10 +338,11 @@ Pattern-based rules for auto-categorization with merchant linking.
  - Loads receipt file from disk
  - Converts PDFs to PNG images using ImageMagick (220 DPI)
  - Calls OpenAI Vision API with structured prompt
-  - Parses JSON response (merchant, date, amounts, line items)
+  - Parses JSON response (merchant, date, due date, amounts, line items)
  - Updates Receipt entity with extracted data
  - Replaces existing line items
  - Logs parse attempt in ReceiptParseLog
+  - Attempts auto-mapping if receipt is unmapped
  - Returns `ReceiptParseResult`

 **API Configuration:**
@@ -336,16 +353,69 @@ Pattern-based rules for auto-categorization with merchant linking.

 **Prompt Strategy:**
 - Structured JSON request with schema example
- Extracts: merchant, date, subtotal, tax, total, confidence
+- Extracts: merchant, date, dueDate (for bills), subtotal, tax, total, confidence
 - Line items with: description, quantity, unitPrice, lineTotal
 - Special handling: Services/fees have null quantity (not products)
+- Due date extraction: For bills (utility, credit card, etc.), extracts payment due date

 **PDF Handling:**
 - ImageMagick converts first page to PNG at 220 DPI
 - Flattens alpha channel (white background)
 - TrueColor 8-bit RGB output

-**Location:** Services/OpenAIReceiptParser.cs:23-302
+**Auto-Mapping Integration:**
+- After successful parse of unmapped receipts, triggers ReceiptAutoMapper
+- Attempts to automatically link receipt to matching transaction
+- Silently fails if auto-mapping unsuccessful (parsing still successful)
+
+**Location:** Services/OpenAIReceiptParser.cs:23-342
+
+### ReceiptAutoMapper (Services/ReceiptAutoMapper.cs)
+**Interface:** `IReceiptAutoMapper`
+
+**Responsibility:** Automatically map parsed receipts to matching transactions using smart matching rules.
+
+**Key Methods:**
+- `AutoMapReceiptAsync(long receiptId)`
+  - Attempts to match a single receipt to a transaction
+  - Uses date range, merchant name, and amount matching
+  - Returns `ReceiptAutoMapResult` with status (Success, NoMatch, MultipleMatches, AlreadyMapped, NotParsed, Failed)
+
+- `AutoMapUnmappedReceiptsAsync()`
+  - Bulk processes all unmapped parsed receipts
+  - Returns `BulkAutoMapResult` with counts (total, mapped, no match, multiple matches)
+
+**Matching Algorithm:**
+1. **Date Range Filtering:**
+   - For bills with due dates: `billDate` to `dueDate + 5 days` (accounts for auto-pay delays, weekends, bank processing)
+   - For regular receipts: `receiptDate ± 3 days` (accounts for transaction posting delays)
+   - No date = no match (can't narrow down effectively)
+
+2. **Merchant Filtering (optional):**
+   - Matches if merchant name or transaction name contains receipt merchant
+   - Case-insensitive contains matching
+
+3. **Amount Matching (optional):**
+   - If receipt has total: matches transactions within ±$0.10 tolerance
+   - Accounts for rounding differences and tips
+
+4. **Receipt Exclusion:**
+   - Excludes transactions that already have receipts attached
+   - Uses HashSet for performance
+
+**Auto-Mapping Triggers:**
+- Automatically triggered after successful receipt parsing (if receipt is unmapped)
+- Can be manually triggered via "Auto-Map Unmapped Receipts" button on Receipts page
+
+**Result Status Types:**
+- `Success` - Single match found, receipt mapped automatically
+- `NoMatch` - No matching transactions found
+- `MultipleMatches` - Multiple potential matches found (requires manual selection)
+- `AlreadyMapped` - Receipt already linked to a transaction
+- `NotParsed` - Receipt has no parsed data (merchant, date, or total missing)
+- `Failed` - Error during mapping process
+
+**Location:** Services/ReceiptAutoMapper.cs

 ### TransactionAICategorizer (Services/TransactionAICategorizer.cs)
 **Interface:** `ITransactionAICategorizer`
@@ -457,18 +527,69 @@ EF Core DbContext managing all database entities.
 ### Transactions.cshtml / TransactionsModel
 **Route:** `/Transactions`

-**Purpose:** View and search all transactions.
+**Purpose:** View and search all transactions with advanced filtering.

 **Features:**
- Search by name or memo
- Filter by card, category, date range
+- Search by name, memo, category, notes, or merchant
+- Filter by card, category, merchant, date range
+- Quick date range selectors (30/60/90 days, last year, this month, last month)
 - Sort by date, amount, name, category
- Pagination
- Receipt count indicator
+- Pagination with preserved filters
+- Transaction ID display for easy reference
+- Receipt count indicator with badge
+- Upload CSV button in header
 - Edit transaction link

+**Stats Display:**
+- Total transaction count
+- Current page showing count
+- Total spent (debits sum)
+- Total income (credits sum)
+- Net amount with color coding
+
 **Location:** Pages/Transactions.cshtml.cs

+### Receipts.cshtml / ReceiptsModel
+**Route:** `/Receipts`
+
+**Purpose:** Manage receipts with upload, mapping, and auto-mapping capabilities.
+
+**Features:**
+- Upload receipts without initial transaction mapping (unmapped receipts)
+- Duplicate detection with blocking modal dialog
+  - SHA256 hash matching (identical files)
+  - Filename + size matching (possible duplicates)
+  - User confirmation required to proceed
+- View all receipts (mapped and unmapped) with status badges
+- Manual receipt-to-transaction mapping via modal selector
+- Auto-map unmapped receipts button (bulk operation)
+- Smart transaction matching with filters
+- Receipt info display: merchant, date, due date (for bills), total
+- Delete receipts
+- View receipt details
+
+**Transaction Selector Modal:**
+- Displays up to 50 matching transactions in scrollable table
+- Date range filtering:
+  - Bills with due dates: bill date to due date + 5 days
+  - Regular receipts: receipt date ± 3 days
+- Merchant-based relevance sorting (exact match > contains > no match)
+- Amount matching with green highlighting (±$0.10 tolerance)
+- Auto-scroll to first amount match when modal opens
+- Manual transaction ID entry option
+- Link to open Transactions page for searching
+
+**Receipt List Columns:**
+- Uploaded timestamp
+- File name with icon (image/PDF)
+- Receipt info (merchant, date, due date, total)
+- Mapped transaction details or "Unmapped" badge
+- Actions: View, Map (if unmapped), Delete
+
+**Dependencies:** `IReceiptManager`, `IReceiptAutoMapper`
+
+**Location:** Pages/Receipts.cshtml.cs
+
 ### EditTransaction.cshtml / EditTransactionModel
 **Route:** `/EditTransaction/{id}`

@@ -615,6 +736,8 @@ builder.Services.AddScoped<IRecentTransactionsProvider, RecentTransactionsProvid
 // Receipt Services
 builder.Services.AddScoped<IReceiptManager, ReceiptManager>();
 builder.Services.AddHttpClient<IReceiptParser, OpenAIReceiptParser>();
+builder.Services.AddScoped<IReceiptAutoMapper, ReceiptAutoMapper>();
+builder.Services.AddScoped<IMerchantService, MerchantService>();
 ```

 **Location:** Program.cs:1-44
@@ -668,7 +791,7 @@ INDEX: Date, Amount, Category
 ### Receipts Table
 ```sql
 Id (bigint, PK)
-TransactionId (bigint, FK → Transactions.Id, CASCADE)
+TransactionId (bigint, FK → Transactions.Id, CASCADE, NULLABLE)
 FileName (nvarchar(260), NOT NULL)
 ContentType (nvarchar(100), DEFAULT 'application/octet-stream')
 StoragePath (nvarchar(1024), NOT NULL)
@@ -677,11 +800,12 @@ FileHashSha256 (nvarchar(64), NOT NULL)
 UploadedAtUtc (datetime2, NOT NULL)
 Merchant (nvarchar(200))
 ReceiptDate (datetime2)
+DueDate (datetime2) -- For bills: payment due date
 Subtotal (decimal(18,2))
 Tax (decimal(18,2))
 Total (decimal(18,2))
 Currency (nvarchar(8))
-UNIQUE INDEX: (TransactionId, FileHashSha256)
+UNIQUE INDEX: (TransactionId, FileHashSha256) WHERE TransactionId IS NOT NULL
 ```

 ### ReceiptParseLogs Table
@@ -766,7 +890,7 @@ Iterate mappings:
 Update Transaction.Category and Transaction.MerchantId
 ```

-### 3. Upload and Parse Receipt
+### 3. Upload and Parse Receipt (Traditional Flow - From Transaction)

 ```
 User attaches receipt to transaction
@@ -790,13 +914,83 @@ OpenAIReceiptParser.ParseReceiptAsync()
    - Encode as base64
    - Call OpenAI Vision API with structured prompt
    - Parse JSON response
-    - Update Receipt (merchant, date, amounts)
+    - Update Receipt (merchant, date, due date, amounts)
    - Replace ReceiptLineItems
    - Create ReceiptParseLog
    ↓
 Display parsed data
 ```

+### 3a. Upload, Parse, and Auto-Map Receipt (New Unmapped Flow)
+
+```
+User visits /Receipts page
+    ↓
+User uploads receipt without transaction
+    ↓
+ReceiptsModel.OnPostUploadAsync()
+    ↓
+ReceiptManager.UploadUnmappedReceiptAsync()
+    - Validate file (size, extension)
+    - Compute SHA256 hash
+    - Check for duplicates (hash, filename+size)
+    ↓
+IF duplicates found:
+    - Delete uploaded file temporarily
+    - Store duplicate warnings in TempData
+    - Redirect to GET with modal
+    ↓
+    User sees blocking modal:
+        Option A: Cancel (don't upload)
+        Option B: Upload Anyway (re-upload with confirmation)
+    ↓
+IF no duplicates OR confirmed:
+    - Save to wwwroot/receipts/
+    - Create Receipt entity (TransactionId = null)
+    ↓
+User clicks "Parse Receipt"
+    ↓
+OpenAIReceiptParser.ParseReceiptAsync()
+    - Parse receipt (merchant, date, due date, total, line items)
+    - Update Receipt entity
+    - Trigger ReceiptAutoMapper.AutoMapReceiptAsync()
+    ↓
+ReceiptAutoMapper.AutoMapReceiptAsync()
+    - Find matching transactions:
+        * Date filter (bill date to due date+5 OR receipt date ±3)
+        * Merchant filter (contains, case-insensitive)
+        * Amount filter (±$0.10 tolerance)
+        * Exclude transactions with existing receipts
+    ↓
+    IF single match found:
+        - Auto-map receipt to transaction
+        - Return Success
+    ELSE IF multiple matches:
+        - Return MultipleMatches (requires manual selection)
+    ELSE:
+        - Return NoMatch
+    ↓
+IF auto-mapping failed (no match or multiple matches):
+    User clicks "Map" button on receipt row
+    ↓
+    ReceiptsModel.FindMatchingTransactionsForReceipt()
+        - Apply same filtering logic
+        - Sort by merchant relevance (exact > contains > none)
+        - Return up to 50 matches
+    ↓
+    Modal displays transactions in table:
+        - Green highlighting for amount matches
+        - Auto-scroll to first amount match
+        - Radio button selection
+        - Manual ID entry option
+    ↓
+    User selects transaction
+    ↓
+    ReceiptsModel.OnPostMapToTransactionAsync()
+        - Links receipt to transaction
+        - Redirect to receipts page
+```
+
 ### 4. Dashboard Aggregation

 ```
@@ -1044,5 +1238,31 @@ MoneyMap demonstrates a well-architected ASP.NET Core application with clear sep
 ---

 **Last Updated:** 2025-10-12
-**Version:** 1.1
+**Version:** 1.2
 **Framework:** ASP.NET Core 8.0 / EF Core 9.0
+
+## Recent Changes (v1.2)
+
+### Receipt Management Enhancements
+- **Unmapped Receipts**: Receipts can now be uploaded without initial transaction mapping
+- **Duplicate Detection**: SHA256 hash and filename+size matching with blocking modal confirmation
+- **Auto-Mapping**: Smart automatic receipt-to-transaction matching using date, merchant, and amount
+- **Due Date Support**: Bills (utility, credit card) can have due dates for expanded matching windows
+- **Manual Mapping UI**: Enhanced modal with transaction table, relevance sorting, amount highlighting, and auto-scroll
+- **Date Range Intelligence**:
+  - Bills: bill date to due date + 5 days (accounts for auto-pay processing)
+  - Regular receipts: receipt date ± 3 days
+
+### UI Improvements
+- **Transactions Page**: Added search, quick date filters, transaction ID column, Upload CSV button in header
+- **Navigation**: Moved Upload CSV from menu to Transactions page header
+- **Receipts Page**: New dedicated page for receipt management with mapping workflow
+- **Transaction Selector**: Scrollable table with green amount match highlighting and auto-scroll to best match
+
+### Technical Improvements
+- Made `Receipt.TransactionId` nullable with filtered unique index
+- Added `Receipt.DueDate` column for bill payment tracking
+- Enhanced OpenAI prompt to extract due dates from bills
+- Implemented `ReceiptAutoMapper` service with intelligent matching algorithm
+- Updated `ReceiptManager` with unmapped receipt support and duplicate detection
+- Added `MerchantService` for merchant management