Fetching the paper…

Reading the bibliography…

2025

VFP: Variational Flow-Matching Policy for Multi-Modal Robot Manipulation

Zhai, Xuanran, Zhao, Qianyou, Yu, Qiaojun et al.

Original
PDF
Post on X

Understand

Flow-matching-based policies have recently emerged as a promising approach for learning-based robot manipulation, offering significant acceleration in action sampling compared to diffusion-based policies.

However, conventional flow-matching methods struggle with multi-modality, often collapsing to averaged or ambiguous behaviors in complex manipulation tasks.
To address this, we propose the Variational Flow-Matching Policy (VFP), which introduces a variational latent prior for mode-aware action generation and effectively captures both task-level and trajectory-level multi-modality.
VFP further incorporates Kantorovich Optimal Transport (K-OT) for distribution-level alignment and utilizes a Mixture-of-Experts (MoE) decoder for mode specialization and efficient inference.

VFP: Variational Flow-Matching Policy for Multi-Modal Robot Manipulation · Around

Built on

A. Handa et al. , “Dexpilot: Vision based teleoperation of dexterous robotic hand-arm system,” 2019. [Online]. Available: https://arxiv.org/abs/1910.03135
Original
1910
Earlier work this paper cites.
A. Gupta, V. Kumar, C. Lynch, S. Levine, and K. Hausman, “Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning,” 2019. [Online]. Available: https://arxiv.org/abs/1910.11956
Original
1910
Earlier work this paper cites.
T. Yu et al. , “Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning,” 2021. [Online]. Available: https://arxiv.org/abs/1910.10897
Original
1910
Earlier work this paper cites.
G. Papamakarios et al. , “Normalizing flows for probabilistic modeling and inference,” 2021. [Online]. Available: https://arxiv.org/abs/1912.02762
Original
1912
Earlier work this paper cites.
R. A. Jacobs et al. , “Adaptive mixtures of local experts,” Neural Computation , vol. 3, no. 1, pp. 79–87, 1991
1991
Earlier work this paper cites.
M. Jordan and R. Jacobs, “Hierarchical mixtures of experts and the em algorithm,” in Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan) , vol. 2, 1993, pp. 1339–1344 vol.2
1993
Earlier work this paper cites.
C. M. Bishop, “Mixture density networks,” 1994
1994
Earlier work this paper cites.
C.-W. Huang, L. Dinh, and A. Courville, “Augmented normalizing flows: Bridging the gap between generative flows and latent variable models,” 2020. [Online]. Available: https://arxiv.org/abs/2002.07101
Original
2002
Earlier work this paper cites.
C. Villani and A. M. Society, Topics in Optimal Transportation , ser. Graduate studies in mathematics. American Mathematical Society, 2003. [Online]. Available: https://books.google.com.sg/books?id=MyPjjgEACAAJ
2003
Earlier work this paper cites.
M. Cuturi, “Sinkhorn distances: Lightspeed computation of optimal transportation distances,” 2013. [Online]. Available: https://arxiv.org/abs/1306.0895
Original
2013
Earlier work this paper cites.
K. Hausman, Y. Chebotar, S. Schaal, G. Sukhatme, and J. Lim, “Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets,” 2017. [Online]. Available: https://arxiv.org/abs/1705.10479
Original
2017
Earlier work this paper cites.

Similar

A. Rajeswaran et al. , “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,” 2018. [Online]. Available: https://arxiv.org/abs/1709.10087
Original
2018
Cited alongside, same era.
R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, “Neural ordinary differential equations,” Advances in neural information processing systems , vol. 31, 2018
2018
Cited alongside, same era.
G. Peyré and M. Cuturi, “Computational optimal transport,” 2020. [Online]. Available: https://arxiv.org/abs/1803.00567
Original
2020
Cited alongside, same era.
I. Kobyzev, S. J. Prince, and M. A. Brubaker, “Normalizing flows: An introduction and review of current methods,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 43, no. 11, p. 3964–3979, Nov. 2021. [Online]. Available: http://dx.doi.org/10.1109/TPAMI.2020.2992934
2020
Cited alongside, same era.

Then

G. Lu et al. , “Manicm: Real-time 3d diffusion policy via consistency model for robotic manipulation,” arXiv preprint arXiv:2406.01586 , 2024
Original
2024
Later among the works it cites.
Z. Wang et al. , “One-step diffusion policy: Fast visuomotor policies via diffusion distillation,” 2024. [Online]. Available: https://arxiv.org/abs/2410.21257
Original
2024
Later among the works it cites.
Q. Rouxel, A. Ferrari, S. Ivaldi, and J.-B. Mouret, “Flow matching imitation learning for multi-support manipulation,” 2024. [Online]. Available: https://arxiv.org/abs/2407.12381
Original
2024
Later among the works it cites.
Q. Zhang et al. , “Flowpolicy: Enabling fast and robust 3d flow-based policy via consistency flow matching for robot manipulation,” 2024. [Online]. Available: https://arxiv.org/abs/2412.04987
Original
2024
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

M. Shridhar, L. Manuelli, and D. Fox, “Perceiver-actor: A multi-task transformer for robotic manipulation,” in Proceedings of the 6th Conference on Robot Learning (CoRL) , 2022

2022

Cited alongside, same era.

X. Liu, C. Gong, and Q. Liu, “Flow straight and fast: Learning to generate and transfer data with rectified flow,” 2022. [Online]. Available: https://arxiv.org/abs/2209.03003

2022

Cited alongside, same era.

C. Wang et al. , “Mimicplay: Long-horizon imitation learning by watching human play,” 2023. [Online]. Available: https://arxiv.org/abs/2302.12422

2023

Cited alongside, same era.

C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” in Proceedings of Robotics: Science and Systems (RSS) , 2023

2023

Cited alongside, same era.

Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le, “Flow matching for generative modeling,” 2023. [Online]. Available: https://arxiv.org/abs/2210.02747

2023

Cited alongside, same era.

Y. Ze et al. , “3d diffusion policy: Generalizable visuomotor policy learning via simple 3d representations,” in Proceedings of Robotics: Science and Systems (RSS) , 2024

2024

Cited alongside, same era.

Y. Wang, Y. Zhang, M. Huo, R. Tian, X. Zhang, Y. Xie, C. Xu, P. Ji, W. Zhan, M. Ding, and M. Tomizuka, “Sparse diffusion policy: A sparse, reusable, and flexible policy for robot learning,” 2024. [Online]. Available: https://arxiv.org/abs/2407.01531

2024

Cited alongside, same era.

M. Braun et al. , “Riemannian flow matching policy for robot motion learning,” 2024. [Online]. Available: https://arxiv.org/abs/2403.10672

2024

Later among the works it cites.

X. Jia, D. Blessing, X. Jiang, M. Reuss, A. Donat, R. Lioutikov, and G. Neumann, “Towards diverse behaviors: A benchmark for imitation learning with human demonstrations,” in The Twelfth International Conference on Learning Representations , 2024. [Online]. Available: https://openreview.net/forum?id=6pPYRXKPpw

2024

Later among the works it cites.

L. Yang et al. , “Consistency flow matching: Defining straight flows with velocity consistency,” 2024. [Online]. Available: https://arxiv.org/abs/2407.02398

2024

Later among the works it cites.

F. Zhang and M. Gienger, “Affordance-based robot manipulation with flow matching,” 2025. [Online]. Available: https://arxiv.org/abs/2409.01083

2025

Closest in time.

A. Samaddar, Y. Sun, V. Nilsson, and S. Madireddy, “Efficient flow matching using latent variables,” 2025. [Online]. Available: https://arxiv.org/abs/2505.04486

2025

Closest in time.

H. Chen et al. , “Gaussian mixture flow matching models,” 2025. [Online]. Available: https://arxiv.org/abs/2504.05304

2025

Closest in time.

P. Guo and A. G. Schwing, “Variational rectified flow matching,” 2025. [Online]. Available: https://arxiv.org/abs/2502.09616

2025

Closest in time.