Abstract

The rapid integration of generative AI in finance introduces both opportunities and challenges, particularly when analyzing sensitive data such as Securities and Exchange Commission (SEC) filings. This study investigates the use of open-source Small Large Language Models (SLLMs), deployed locally through the Ollama and LangChain frameworks, combined with Retrieval-Augmented Generation (RAG) for extracting financial insights relevant to index performance and reporting quality. Two key objectives guide this work: (1) benchmarking multiple open-source SLLMs for sentiment analysis, multiple-choice reasoning, and financial question answering, and (2) assessing the feasibility of locally deployed SLLMs for domain-specific financial queries. A standardized set of 50 finance-related questions was applied to each SEC filing through a RAG pipeline. Six SLLMs: Gemma3, Phi4, Llama3.1, Gpt-oss, Qwen3, and DeepSeek-r1 (10b–20b parameters) were evaluated on Financial PhraseBank, FinanceBench, and the finance subset of MMLU. Performance metrics included accuracy, BLEU, and ROUGE score to measure factuality, fluency, and reasoning quality. Results reveal trade-offs between accuracy, computational efficiency, and token throughput, highlighting model-specific strengths. The findings provide a reproducible, privacy-preserving framework for deploying locally run AI systems in financial analytics on university HPC infrastructure.

Recommended Citation

Vu, Tue; Austin, Mark; and Tuijn, Marcel (2025) "Application of Open-Source Small Large Language Models for Finance report analysis," SMU Data Science Review: Vol. 9: No. 3, Article 2.
Available at: https://scholar.smu.edu/datasciencereview/vol9/iss3/2