Evaluating Language Models (LMs) in specialized, high-stakes domains such as finance remains a significant challenge due to the scarcity of open, high-quality, and domain-specific datasets. Existing general-purpose benchmarks provide broad coverage but lack the depth and domain fidelity needed to assess LMs' capabilities for real-world financial reasoning, which requires both conceptual understanding and quantitative rigor. To address this gap, we introduce FinForge, a scalable, semi-synthetic pipeline for constructing finance-specific evaluation benchmarks through a hybrid of expert-guided data curation and controlled LM-based synthesis. FinForge combines manual and programmatic corpus construction from authoritative financial sources with structured question generation and validation using Gemini 2.5 Flash. To demonstrate the pipeline's efficacy, we produce FinForge-5k, a snapshot benchmark comprising over 5,000 human-validated question-answer pairs across 11 finance subdomains, derived from a curated corpus of 100,000 verified documents totaling 143M tokens. Evaluation of state-of-the-art open-source and closed-source models on FinForge-5k reveals significant differences in financial reasoning, with leading models achieving accuracy levels near 80%. These findings underscore the framework's utility for diagnosing current model limitations and guiding future improvements in financial domain competence. All code and data are available at https://github.com/gtfintechlab/FinForge.

翻译：在金融等高风险专业领域评估语言模型（LM）仍面临重大挑战，主要源于公开、高质量且领域特定的数据集稀缺。现有通用基准虽覆盖广泛，但缺乏评估语言模型在现实金融推理能力所需的深度与领域保真度——这种推理既需要概念理解，也要求定量严谨性。为填补这一空白，我们提出FinForge：一个可扩展的半合成流程，通过专家引导的数据策展与基于语言模型的可控合成相结合，构建金融专用评估基准。FinForge融合了权威金融来源的手动与程序化语料构建，并利用Gemini 2.5 Flash进行结构化问题生成与验证。为验证该流程的有效性，我们构建了FinForge-5k基准快照，包含来自10万份经审核文档（总计1.43亿词元）的5,000余个人工验证问答对，覆盖11个金融子领域。基于FinForge-5k对当前最优开源与闭源模型的评估显示，各模型在金融推理能力上存在显著差异，领先模型的准确率接近80%。这些发现印证了该框架在诊断现有模型局限、指导未来金融领域能力改进方面的实用价值。所有代码与数据公开于https://github.com/gtfintechlab/FinForge。