Unified Multimodal Chain-of-Thought Reward Modelthrough Reinforcement Fine-Tuning

2505.03318v1https://arxiv.org/pdf/2505.03318v1

1. 概述

        近年来，多模态奖励模型（RMs）[Wang et al. 2024 2025, Zang et al. 2025, Xiong et al. 2024, He et al. 2024, Xu et al. 2024, Liu et al. 2025a, Li et al. 2025] 在将视觉模型输出与人类偏好对齐方面表现出色，为模型训练提供了关键的奖励信号[Wang et al. 2024 2025, Liu et al. 2025a, Ouyang et al. 2022, Rafique et al. 2023, Schulman et al. 2017]和推理[Gulcehre et al. 202