FinReason:Financial QA LLM (SFT + DPO)
Fine-tuned Qwen2.5-7B-Instruct on FinQA (SEC filings) using QLoRA SFT followed by DPO alignment, targeting multi-step numerical reasoning over financial tables. Model published to HuggingFace Hub with automated eval metric injection, plus a production K8s serving layer (vLLM + AWQ, HPA autoscaling, Prometheus/Grafana).
Core Impact
“0.3% → 58.5% accuracy and 6.46 → 1.71 perplexity drop on FinQA SEC earnings via QLoRA SFT + DPO, training only 0.67% of Qwen2.5-7B parameters, with a K8s vLLM+AWQ serving layer load-tested at 326 req/s.”
0.3% → 58.5%
Accuracy after SFT + DPO
6.46 → 1.71
Perplexity drop on FinQA test set
326 req/s
Serving load test, p99 14ms, 0% errors
The Problem
General-purpose LLMs are bad at multi-step arithmetic over financial tables. Ask base Qwen2.5-7B a FinQA question about revenue growth across SEC filings and it gets 0.3% right. I wanted to teach a 7B model to actually reason over those tables without renting a giant frontier model to do it.
The Approach
Why QLoRA
I fine-tuned in 4-bit with LoRA adapters on all seven projections, training only 0.67% of the weights. That's what let a 7B model fine-tune on a single A100 (Colab Pro) instead of a cluster, and it took accuracy from 0.3% to 56.5% on held-out FinQA questions before alignment.
Why add a DPO stage
After supervised fine-tuning I aligned the model with DPO on synthetic preference pairs. It came back with a 0.625 win rate over the SFT checkpoint and pushed accuracy from 56.5% to 58.5%, so the alignment stage earned its place rather than just being there for show.
Why automate the model card
The publish script injects the real eval numbers into the README right before upload, so the accuracy and perplexity shown on HuggingFace always match what the evaluation actually produced. No stale numbers, no rounding up.
Why build a K8s serving layer
Training a model is half the job. I quantized the 7B to AWQ 4-bit, published it, and wrote a vLLM serving stack on Kubernetes with a Service, HPA autoscaling (2→8 pods), and Prometheus/Grafana. A Locust load test held 326 req/s at p99 14ms with zero errors. Honest scope: the full mechanic runs end-to-end on a local CPU cluster with a small model, and the 7B AWQ manifest plus runbook are ready for any GPU node.
Technical Deep Dive
Fine-tuned Qwen2.5-7B-Instruct on 8K FinQA SEC earnings samples via QLoRA (NF4 4-bit, double quant, LoRA r=16 α=32 across all 7 projections). Accuracy jumped 0.3% → 58.5% (+58.2pp, base → DPO) while training only 0.67% of parameters.
Perplexity dropped 6.46 → 1.71 (base → SFT) on FinQA test set (313 examples, ±1% numeric tolerance), indicating strong adaptation to multi-step arithmetic reasoning over financial tables.
DPO alignment (TRL DPOTrainer, synthetic preference pairs on the SFT checkpoint) lifted accuracy 56.5% → 58.5% with a 0.625 win rate vs SFT, confirmed preferred on 62.5% of contested pairs.
Automated HuggingFace Hub publishing pipeline that injects real eval metrics (accuracy, perplexity) into model card README before upload, ensuring published numbers always match actual results.
Built a production serving layer: AWQ 4-bit quantization (published to `finreason-qwen2.5-7b-awq`), vLLM inference, Kubernetes Deployment + Service + HPA (2→8 pods), Prometheus/Grafana, and a Locust load test sustaining 326 req/s at p99 14ms, 0% errors.
Systems Analysis Concluded