FinBuddy:AI-Powered Personal Finance Tracker

Multi-Stage Vision and Semantic Extraction Pipeline: Developed a sophisticated OCR processing route that utilizes GPT-4o Vision to extract structured financial data (amount, category, and line items) from receipt images. The pipeline implements a secondary LLM step to generate high-dimensional vector embeddings using the text-embedding-3-small model, effectively transforming unstructured physical receipts into searchable semantic data.

Vector Search and Semantic Retrieval Architecture: Engineered a semantic search functionality by integrating pgvector within a Supabase PostgreSQL database. This includes the development of a custom PL/pgSQL function, match_expenses, which performs cosine similarity searches to retrieve contextually relevant financial records based on natural language queries, featuring adjustable precision thresholds and optimized indexing via ivfflat.

Automated Financial Insight Generation: Built an asynchronous insights engine that leverages GPT-4-preview to analyze raw OCR text and generate actionable financial intelligence. The system autonomously identifies unusual spending patterns, provides category-level summaries, and suggests optimizations, delivering the output as structured JSON for immediate frontend consumption.

Semantic Caching and Performance Optimization: Implemented a Semantic Cache layer using a dedicated PostgreSQL table to store LLM-generated summaries and their corresponding 1536-dimensional embeddings. This architecture reduces redundant API calls and latency by allowing the system to perform vector similarity lookups on previous AI summaries before triggering expensive re-generation.