CrossHA
CVPR 2026 Highlight
Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning
CrossHA trains a single Minecraft agent to choose between heterogeneous action spaces at each step, using reinforcement learning to balance high-level efficiency with low-level precision.
Abstract
Existing native agents are usually tied to one predefined action space: APIs, GUI events, raw mouse and keyboard controls, robot actions, or other specialized interfaces. CrossHA treats the action interface itself as a learned decision. The model is trained to generate valid actions across multiple granularities and to switch between them during a trajectory, choosing coarse actions when they are efficient and fine actions when precision is needed.
The training pipeline combines cold-start supervised fine-tuning, single-turn GRPO for action-space selection, and multi-turn GRPO for long-horizon task completion. Evaluated on more than 800 Minecraft tasks, CrossHA shows stronger generalization and efficiency than fixed-action baselines and hierarchical agents whose switching behavior is not directly optimized.
Key Numbers
Method Overview
CrossHA formulates heterogeneous action selection as part of the policy rather than as an external rule. The policy receives visual observations and task instructions, then emits both an action space and the action content that should be executed by the corresponding controller.
- 1
Mixed-Space SFT
A cold-start model learns the syntax and semantics of multiple action formats from a balanced mixture of trajectories, establishing basic competence across raw, grounding, motion, language, and latent-style action spaces.
- 2
Single-Turn RL
Diversity-enhanced data collection exposes several valid action choices for the same state. GRPO then rewards actions that parse to the correct raw behavior, letting the model learn which interface is most reliable for the immediate context.
- 3
Multi-Turn RL
The final stage optimizes full trajectories with sparse task success rewards and a generation-cost penalty. This pushes CrossHA to complete long-horizon tasks while preferring concise high-level actions whenever they are dependable.
Results
| Finding | Evidence | Takeaway |
|---|---|---|
| Dynamic switching beats fixed interfaces | Single-action agents specialize in one category but lose balance across Mine Blocks, Kill Entities, and Craft Items. | Learning the interface choice is more robust than hard-coding it. |
| RL generalizes beyond the training set | Multi-turn RL is trained on 30 tasks but evaluated on more than 800 OpenHA tasks. | Action-space selection learned from limited online tasks transfers broadly. |
| STRL improves the final policy | Removing the single-turn RL stage weakens convergence and lowers OOD results, especially on Craft Items. | A local action-space warm start helps long-horizon RL discover better trajectories. |
| OOD robustness improves | The paper reports 78.8% OOD success on Craft Items and 49.1% across all OOD tasks for the full CrossHA model. | Heterogeneous actions reduce overfitting to one control style. |
Behavior Analysis
Kill Sheep
The agent first favors motion actions to search efficiently, then mixes motion, grounding, and raw controls while approaching, and uses more precise grounding or raw actions for the attack phase.
Chop Tree
Static targets make coarse navigation effective early in the task. Grounding actions become more useful once the model needs to target a specific wood block.
Craft Enchanting Table
GUI-heavy crafting relies on raw actions for stable repeated steps and grounding actions for visually locating icons and item slots.
Why It Matters
CrossHA moves action-space switching from a hand-written engineering choice into the model's learned policy. That makes the agent closer to a generalist system: it can use high-level abstractions for speed, fall back to lower-level control for precision, and adapt this choice within a single trajectory.
Resources
BibTeX
@inproceedings{he2026crossha,
title = {Training One Model to Master Cross-Level Agentic Actions via Reinforcement Learning},
author = {He, Kaichen and Wang, Zihao and Li, Muyao and Liu, Anji and Liang, Yitao},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2026}
}