[feat] support opd rl

[hjh0119]

No description provided.

[gemini-code-assist]

Code Review

This pull request implements Megatron On-Policy Distillation as RL (OPD-RL) by integrating teacher KL as a GRPO advantage across local and Ray-based GKD and GRPO trainers. It also introduces OpenEnvScheduler and OpenEnvWrapper to support multi-turn rollouts in OpenEnv environments. The review feedback highlights several critical issues, including missing imports and potential AttributeErrors in gkd_helpers.py, a rank-guarding mismatch in teacher_mixin.py that could cause runtime failures, and synchronous blocking calls in OpenEnvScheduler that should be run in separate threads to avoid blocking the asyncio event loop. Additionally, defensive checks are recommended to prevent potential IndexError, StopIteration, and type promotion issues.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.