FinBuddy:AI-Powered Personal Finance Tracker
Full-stack AI platform with OpenAI GPT-4, OCR, and predictive analytics.
Core Impact
“Transformed raw financial visual data into semantically searchable assets with automated spending pattern analysis.”
~90%
Fewer LLM calls via caching
1536-dim
Vector search over every expense
temp=0
Deterministic receipt extraction
The Problem
Most expense trackers make you type every purchase in by hand, and then they still can't answer a question like "what did I spend on coffee last week" because they match keywords, not meaning. I wanted to snap a photo of a receipt, have every line item pulled out automatically, search my spending by intent, and get the analysis done for me.
The Approach
Why GPT-4o Vision at temperature 0
Receipts are messy and inconsistent. I run extraction at temperature 0 so the same receipt always returns the same amount, category, date, and itemized lines. I want it deterministic, not creative, because a finance tracker that guesses differently each time is useless.
Why pgvector semantic search
I embed every expense with text-embedding-3-small and search by cosine similarity, so "coffee last week" finds the cafe run even when the word coffee never appears on the receipt. It ranks by what you meant, not which keyword happened to match.
Why SHA-256 caching
Before any summary call, I hash the current expense snapshot and check the cache. Roughly 90% of requests hit and skip OpenAI completely. That short-circuit before the API call is where the cost savings actually come from.
Why row-level security at the database
Receipts are private financial data. I enforce ownership in Supabase RLS policies at the database layer and serve receipt images through signed URLs that expire in an hour, so nobody can reach another user's data even if the client-side logic is wrong.
Technical Deep Dive
Built a GPT-4o Vision receipt intelligence pipeline at temperature=0 for deterministic extraction of amount, category, date, and itemized line items; ran a separate GPT-4o-mini pass on raw OCR text to generate per-receipt insights (store name, unusual spend flag, category guess) stored in `expenses.insights_json` for zero-latency display.
Engineered semantic search using 1536-dim embeddings via `text-embedding-3-small` stored in pgvector, with a `match_expenses` cosine similarity RPC and 600ms client-side debounce, enabling natural language retrieval ("coffee last week", "anything from Whole Foods") ranked by intent rather than keyword.
Reduced LLM API costs ~90% via SHA-256 hashing of the current expense snapshot, short-circuiting summary generation on cache hits before any OpenAI call.
Designed Budget Shield (velocity-based spend/day × days_in_month projection, zero LLM, instant response) and Smart Switch (50-transaction GPT-4o-mini pass returning one high-impact savings recommendation, e.g. "switch Disney+ to annual, saves $24/year").
Architected multi-tenant row-level security enforced at the Supabase database layer via RLS policies, and secured receipt storage with server-generated signed URLs (1-hour expiry) via an ownership-verifying API route, preventing unauthorized access regardless of client-side logic.
Systems Analysis Concluded