FDPP: Fine-tune Diffusion Policy with Human Preference

R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs: I. the method of paired comparisons,” Biometrika , vol. 39, no. 3/4, pp. 324–345, 1952

1952

Earlier work this paper cites.

R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” Machine learning , vol. 8, pp. 229–256, 1992

1992

Earlier work this paper cites.

R. S. Sutton, D. McAllester, S. Singh, and Y. Mansour, “Policy gradient methods for reinforcement learning with function approximation,” Advances in neural information processing systems , vol. 12, 1999

1999

Earlier work this paper cites.

A. Y. Ng, S. Russell et al. , “Algorithms for inverse reinforcement learning.” in Icml , vol. 1, no. 2, 2000, p. 2

2000

Earlier work this paper cites.

D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, and M. Riedmiller, “Deterministic policy gradient algorithms,” in International conference on machine learning . Pmlr, 2014, pp. 387–395

2014

Earlier work this paper cites.

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning . PMLR, 2015, pp. 2256–2265

2015

Earlier work this paper cites.

P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems , vol. 30, 2017

2017

Earlier work this paper cites.

D. Sadigh, A. Dragan, S. Sastry, and S. Seshia, Active preference-based learning of reward functions , 2017

2017

Earlier work this paper cites.

C. Wirth, R. Akrour, G. Neumann, and J. Fürnkranz, “A survey of preference-based reinforcement learning methods,” Journal of Machine Learning Research , vol. 18, no. 136, pp. 1–46, 2017

2017

Earlier work this paper cites.

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347 , 2017

Original

2017

Earlier work this paper cites.

E. Biyik and D. Sadigh, “Batch active preference-based learning of reward functions,” in Conference on robot learning . PMLR, 2018, pp. 519–528

2018

Earlier work this paper cites.

D. Brown, W. Goo, P. Nagarajan, and S. Niekum, “Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,” in International conference on machine learning . PMLR, 2019, pp. 783–792

2019

Earlier work this paper cites.

X. B. Peng, E. Coumans, T. Zhang, T.-W. Lee, J. Tan, and S. Levine, “Learning agile robotic locomotion skills by imitating animals,” arXiv preprint arXiv:2004.00784 , 2020

Original

2020

Cited alongside, same era.

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840–6851, 2020

2020

Cited alongside, same era.

Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456 , 2020

Original

2020

Cited alongside, same era.

N. Stiennon, L. Ouyang, J. Wu, D. Ziegler, R. Lowe, C. Voss, A. Radford, D. Amodei, and P. F. Christiano, “Learning to summarize with human feedback,” Advances in Neural Information Processing Systems , vol. 33, pp. 3008–3021, 2020

2020

Cited alongside, same era.

C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,” arXiv preprint arXiv:2303.04137 , 2023

Original

2023

Later among the works it cites.

S. P. Arunachalam, S. Silwal, B. Evans, and L. Pinto, “Dexterous imitation made easy: A learning-based framework for efficient dexterous manipulation,” in 2023 ieee international conference on robotics and automation (icra) . IEEE, 2023, pp. 5954–5961

2023

Later among the works it cites.

D. Shin, A. D. Dragan, and D. S. Brown, “Benchmarks and algorithms for offline preference-based reward learning,” arXiv preprint arXiv:2301.01392 , 2023

Original

2023

Later among the works it cites.

R. Tian, C. Xu, M. Tomizuka, J. Malik, and A. Bajcsy, “What matters to you? towards visual representation alignment for robot learning,” arXiv preprint arXiv:2310.07932 , 2023

Original

2023

Later among the works it cites.