From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation

Xiangyun Huang1, Xiangchen Wang2, Runfeng Lin1,3, Yihao Xu1,
Kangyu Huang4, Jiang Hengchen1,5, Xiwang Dong1, Lin Jiarong1,*
1Beihang University 2Southern University of Science and Technology 3Central South University
4Harbin Institute of Technology, Shenzhen 5Dalian University of Technology
23231088@buaa.edu.cn, zivlin@buaa.edu.cn
arXiv Video Code Models Data

TL;DR

Route2Step decouples route-level semantic progress tracking from local action execution in vision-and-language navigation. MIA identifies the active sub-instruction and execution state, while MAG generates local action chunks conditioned on that state.

Approach

Route2Step framework overview

Instruction Analysis Module (MIA)

MIA receives the full route instruction and visual history. It predicts which sub-instruction is currently active and whether the agent is in a normal or recovering execution state.

Action Generation Module (MAG)

MAG conditions on the explicit progress state and recent observations to generate local action chunks. This separates the question of which step to follow from how to execute that step.

E-SPA and level-separated correction

E-SPA aligns fine-grained sub-instructions with contiguous trajectory segments without additional manual temporal labels. State-level corrective supervision is used for semantic progress, while direct action supervision is reserved for repeated local execution failures.

Resources

CodeEvaluation scripts and utilitiesGitHub ↗
ModelsMIA and MAG checkpointsHugging Face ↗
DataRoute2Step-DataHugging Face ↗

Citation

@misc{huang2026route2step,
  title         = {From Routes to Steps: Separating Semantic Progress from Local Execution in Vision-and-Language Navigation},
  author        = {Xiangyun Huang and Xiangchen Wang and Runfeng Lin and Yihao Xu and Kangyu Huang and Jiang Hengchen and Xiwang Dong and Lin Jiarong},
  year          = {2026},
  eprint        = {2608.03143},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2608.03143}
}