FinReason:Financial QA LLM (SFT + DPO)
Two-stage QLoRA post-training pipeline on Qwen2.5-7B-Instruct over FinQA SEC earnings filings: supervised fine-tuning, then DPO alignment, then AWQ 4-bit compression served on vLLM. Measured end to end on the official 1,147-example FinQA test split, not a sampled subset.
Core Impact
“52.2% → 59.2% accuracy on the official 1,147-example FinQA test split and perplexity 6.60 → 2.87 via QLoRA SFT + DPO, training only 0.67% of Qwen2.5-7B parameters, served as AWQ 4-bit on vLLM at ~3,000 tok/s with zero failures under load.”
52.2% → 59.2%
Accuracy on official 1,147-example split
6.60 → 2.87
Perplexity after SFT + DPO
18.9 req/s
vLLM on A40, p99 1.5s, 0 failures
The Problem
General-purpose LLMs are shaky at multi-step arithmetic over financial tables. Base Qwen2.5-7B gets about half of FinQA right, and the half it misses are the multi-hop questions that actually matter. I wanted to teach a 7B model to reason over SEC filings properly, measure it on the full official test split instead of a friendly subset, and then find out what serving it in 4-bit really costs.
The Approach
Why QLoRA
I fine-tuned in 4-bit with LoRA adapters on all seven projections, training only 0.67% of the weights. That's what let a 7B model fine-tune on a single A100 instead of a cluster, and it's where most of the accuracy gain over the base model comes from.
Why add a DPO stage
After supervised fine-tuning I aligned the model with DPO at β=0.1 on mined preference pairs, selecting the checkpoint by dev loss rather than by whichever run looked best. The alignment stage is what carries accuracy to 59.2%, within 2 points of the FinQANet baseline.
Why score the full 1,147-example split
Sampling a few hundred examples makes the number noisy enough to flatter itself. I ran every question in the official FinQA test split, before and after each stage, so the 52.2% to 59.2% move is a real measurement and not a lucky draw.
Why quantify the AWQ cost
AWQ 4-bit calibrated on real FinQA prompts cuts the weights to 5.2 GB, which is what makes it servable on one GPU. Compression is usually reported as free. I re-scored the live vLLM endpoint on all 1,147 examples and measured the actual price: 4.1 points of accuracy.
Why load-test and instrument it
A Locust run with full SEC-table prompts held 18.9 req/s at 50 concurrent users, p99 1.5s, zero failures across 3,337 requests, and p50 latency actually dropped as concurrency rose thanks to continuous batching. Prometheus and Grafana capture ~3,000 tok/s and 60ms time-to-first-token, so the serving claims come from instrumentation rather than a stopwatch.
Why a CI gate on the metrics
A 30-test suite on GitHub Actions guards the answer parser, the data contract, and accuracy floors. Eval bugs rarely crash: they return a plausible wrong number, and this is what catches that before it reaches the model card.
Technical Deep Dive
Fine-tuned Qwen2.5-7B-Instruct on FinQA SEC filings via QLoRA (NF4 4-bit, double quant, LoRA r=16 α=32 across all 7 projections), training only 0.67% of weights. Accuracy on the official 1,147-example test split moved 52.2% → 59.2% through SFT then DPO, landing within 2 points of the FinQANet baseline.
Engineered two-stage alignment: SFTTrainer on chain-of-thought targets, then DPOTrainer at β=0.1 on mined preference pairs. Perplexity dropped 6.60 → 2.87, with dev-loss checkpoint selection tracked in Weights & Biases.
Compressed the merged adapter to AWQ 4-bit calibrated on real FinQA prompts, cutting weights to 5.2 GB; re-scored the live vLLM endpoint on all 1,147 examples and quantified the 4.1pp accuracy cost of compression rather than assuming it was free.
Load-tested the vLLM server on an A40 with full SEC-table prompts: 18.9 req/s at 50 concurrent users, p99 1.5s, 0 failures across 3,337 requests, with p50 latency falling as concurrency rose under continuous batching.
Deployed Prometheus and Grafana observability provisioned from files, capturing ~3,000 tok/s generation throughput and 60ms time-to-first-token on the served endpoint.
Built a 30-test CI suite on GitHub Actions gating the answer parser, data contract, and accuracy floors, catching silent metric regressions that produce plausible wrong numbers instead of exceptions.
Automated HuggingFace Hub publishing that injects real eval metrics into the model card README before upload, so published numbers always match actual results.
Systems Analysis Concluded