FinAutoRubric: Expert-Guided Automatic Rubric Generation for Evaluating Financial Research Agents
By Hoyoung Lee, Suyeol Yun, Jack Haverty, Yunju Cho, Meesong Kim, Daekyung Park, Sumin Kim, Jihoon Kwon, Jasmine Jia Geng, Andrew Chin, Yin Luo, Edward Tong, Yu Yu, Zach Golkhou, Minkyu Kim, Igor Halperin, Young Cha, Alejandro Lopez-Lira, Chanyeol Choi, Yongjae Lee
Rating
1500
Battle Count: 0
Relevance
6/10
Highly relevant for evaluating the quality of AI agents performing fundamental analysis, earnings analysis, and valuation, which are inputs to quantitative strategies. It does not directly address trading execution or market microstructure.
Implementation Complexity
8/10
High complexity due to the multi-stage pipeline involving task bank construction, contextualization, and iterative writer-reviewer loops with code validation and human escalation mechanisms.
Reproducibility
4/5
Code and data are stated to be publicly released soon. The paper provides detailed prompts, model configurations, and evaluation protocols in the appendices.
About this paper
Methodology: FinAutoRubric. Problem types: Natural Language Processing, Evaluation, Rubric Generation, Financial Analysis.
The interactive Everscope explorer (charts, battles, favorites) loads below.