Agentic RL · 评测→训练闭环个人项目
Agentic RL: Eval-to-Training Loop
verlGRPOALFWorld多轮 rollout可验证奖励 / RLVR
Summary
Drive multi-turn agent GRPO with verifiable rewards, turning 'evaluation finds gaps' into 'training fixes gaps', validated end-to-end on ALFWorld.
Problem
Evaluation precisely characterizes capability gaps but cannot fix them; the eval signal must flow back into training.
Solution
Use rubric-based verifiable rewards as RL signal; train a multi-turn tool-using agent with GRPO on verl, handling turn-level credit assignment, sparse trajectory rewards and long-horizon stability.
Impact
- Ran multi-turn agent GRPO on ALFWorld, closing the eval → training → re-eval loop.
- One verifiable-reward scheme serves both evaluation scoring and RL training.
Multi-turn Agentic RL
Think→call tool→read env→decide multi-turn rollout; token-level loss mask (agent tokens only); GRPO vs PPO stability on long horizons.
Key Techniques
Honest Scope
The agentic RL is truly validated on ALFWorld; migration to real office/code agent tasks is currently a design.