M4M

arXiv preprint

Learning to Move from and for Manipulation Policies

Spatializing Latent Action Representations for Goal-Pose-Free Mobile Manipulation

Siyuan Zheng1  Zhong Cao2,*  Henry X. Liu2,3,*

1Department of Electrical and Computer Engineering, University of Michigan
2Department of Civil and Environmental Engineering, University of Michigan
3University of Michigan Transportation Research Institute
*Corresponding authors

8articulated-object tasks
47.0%average success in held-out kitchens, vs. 26.5% for N2M
4frozen manipulation policies, each improved on fridge closing

From two robot cameras and a language instruction, the robot finds the target, walks around the obstacle, and stops where the frozen policy is most likely to succeed—then hands off.

At a glance

Reach is not enough

M4M learns how to move and when to hand off from a frozen manipulation policy. It turns the policy’s internal action representations into a success field, predicting where manipulation is likely to succeed before the robot gets there. From stereo observations, the robot updates this field as it moves and decides when to begin manipulation—without committing to a target base pose.

Method

How it is trained

The frozen policy itself tells the robot where to stand. We read its action-expert features at candidate stances, score them with a probe fitted to sparse outcomes, and teach a student to predict them from wherever the robot is now.

1Rollout & DistillationTraining

The frozen policy runs from candidate stances b1…b6; its action expert exposes a feature h at each one.

2Score MatchTraining

A probe fitted to the sparse outcomes turns each feature h into a success score: the higher the score, the better the place to stand.

3Score PredictTraining

From the robot’s current view at a, a pose-conditioned student predicts z ≈ h for every candidate stance, so stances it has not visited can be scored.

4DeployDeployment

In a held-out kitchen, the navigation policy moves the robot using the predicted success, and a learned handoff policy decides when manipulation starts.

Comparisons

Head-to-head on every task

Same start, same kitchen, same frozen π0.5; only the stance differs. Pick a task below: clips play at 3× speed and hold their last frame with the outcome.

N2M
π0.5
Ours

Results

Eight tasks in held-out kitchens

On every one of the eight tasks, moving with M4M before the handoff gives the same frozen π0.5 a higher success rate than running it alone or after N2M.

Average over eight tasks

8 / 8tasks where ours beats both baselines
+20.5points over N2M, averaged over tasks
2.4×the average success rate of π0.5 alone

Success rate (%), mean ± s.d. over independent runs. Click a task to watch it side by side.

Citation

BibTeX

@article{zheng2026m4m,
  title   = {Learning to Move from and for Manipulation Policies: Spatializing Latent Action Representations for Goal-Pose-Free Mobile Manipulation},
  author  = {Zheng, Siyuan and Cao, Zhong and Liu, Henry X.},
  journal = {arXiv preprint arXiv:2611.25366},
  year    = {2026}
}