AuditAI:Agentic RAG Compliance Engine
A production agentic compliance engine using LangGraph and Corrective RAG (CRAG) to audit organizational policies against 4 major cybersecurity frameworks simultaneously: NIST CSF 2.0, NIST SP 800-53 Rev 5, ISO 27001:2022, and SOC 2 Trust Services Criteria.
Core Impact
“Audits policies against 4 frameworks (NIST CSF 2.0, SP 800-53, ISO 27001, SOC 2) simultaneously — achieving 96.7% RAGAS Faithfulness and 100% Context Recall via Corrective RAG with LLM-as-judge grading.”
Architecture Breakdown
Engineered Corrective RAG pipeline using LangGraph state machine with LLM-as-judge document grading and autonomous query reformulation loop (up to 3 retries before fallback), auditing policies simultaneously against NIST CSF 2.0, NIST SP 800-53, ISO 27001:2022, and SOC 2.
Parallelized retrieval across all 4 framework indexes via ThreadPoolExecutor and document grading via asyncio.gather — worst-case grading latency equals 1 LLM call regardless of chunk count; equal framework representation guaranteed.
Built semantic cache in second Qdrant Cloud collection at cosine similarity threshold 0.93 — near-duplicate queries skip the full graph, delivering near-instant response at zero LLM cost.
Engineered page-level citation system linking every answer to exact source document and page number from official PDFs — zero phantom citations via 6-pattern refusal-phrase detection on low-confidence generations.
Built automated RAGAS evaluation harness across 33 adversarial questions spanning all 4 frameworks — Faithfulness 96.7%, Context Recall 100%, Context Precision 79.4%, Answer Relevancy 77.6%.
Systems Analysis Concluded