PlanLLM: Video Procedure Planning with Refinable Large Language Models

Language models are few-shot learners

Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020 · 1901

Earlier work this paper cites.

The materials science procedural text corpus: Annotating materials synthesis procedures with shallow semantic structures

Original

Mysore, S.; Jensen, Z.; Kim, E.; Huang, K.; Chang, H.-S.; Strubell, E.; Flanigan, J.; McCallum, A.; and Olivetti, E. 2019 · 1905

Earlier work this paper cites.

A language-first approach for procedure planning

Liu, J.; Li, S.; Wang, Z.; Li, M.; and Ji, H. 2023b · 1954

Earlier work this paper cites.

Visually-grounded planning without vision: Language models infer detailed plans from high-level instructions

Original

Jansen, P. A. 2020 · 2009

Earlier work this paper cites.

Understanding natural language commands for robotic navigation and mobile manipulation

Tellex, S.; Kollar, T.; Dickerson, S.; Walter, M.; Banerjee, A.; Teller, S.; and Roy, N. 2011 · 2011

Earlier work this paper cites.

Unsupervised learning from narrated instruction videos

Alayrac, J.-B.; Bojanowski, P.; Agrawal, N.; Sivic, J.; Laptev, I.; and Lacoste-Julien, S. 2016 · 2016

Earlier work this paper cites.

Who let the dogs out? modeling dog behavior from visual data

Ehsani, K.; Bagherinezhad, H.; Redmon, J.; Mottaghi, R.; and Farhadi, A. 2018 · 2018

Earlier work this paper cites.

Tracking state changes in procedural text: a challenge dataset and models for process paragraph comprehension

Original

Mishra, B. D.; Huang, L.; Tandon, N.; Yih, W.-t.; and Clark, P. 2018 · 2018

Earlier work this paper cites.

Universal planning networks: Learning generalizable representations for visuomotor control

Srinivas, A.; Jabri, A.; Abbeel, P.; Levine, S.; and Finn, C. 2018 · 2018

Earlier work this paper cites.

Uncertainty-aware anticipation of activities

Abu Farha, Y.; and Gall, J. 2019 · 2019

Earlier work this paper cites.

Coin: A large-scale dataset for comprehensive instructional video analysis

Tang, Y.; Ding, D.; Rao, Y.; Zheng, Y.; Zhang, D.; Zhao, L.; Lu, J.; and Zhou, J. 2019 · 2019

Earlier work this paper cites.

Cross-task weakly supervised learning from instructional videos

Zhukov, D.; Alayrac, J.-B.; Cinbis, R. G.; Fouhey, D.; Laptev, I.; and Sivic, J. 2019 · 2019

Cited alongside, same era.

Procedure planning in instructional videos

Chang, C.-Y.; Huang, D.-A.; Xu, D.; Adeli, E.; Fei-Fei, L.; and Niebles, J. C. 2020 · 2020

Cited alongside, same era.

End-to-end learning of visual representations from uncurated instructional videos

Miech, A.; Alayrac, J.-B.; Smaira, L.; Laptev, I.; Sivic, J.; and Zisserman, A. 2020 · 2020

Cited alongside, same era.

Procedure planning in instructional videos via contextual modeling and model-based policy learning

Bi, J.; Luo, J.; and Xu, C. 2021 · 2021

Cited alongside, same era.

Learning transferable visual models from natural language supervision

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021 · 2021

Cited alongside, same era.

Palm: Scaling language modeling with pathways

Chowdhery, A.; Narang, S.; Devlin, J.; Bosma, M.; Mishra, G.; Roberts, A.; Barham, P.; Chung, H. W.; Sutton, C.; Gehrmann, S.; et al. 2023 · 2023

Later among the works it cites.

Llm-planner: Few-shot grounded planning for embodied agents with large language models

Song, C. H.; Wu, J.; Washington, C.; Sadler, B. M.; Chao, W.-L.; and Su, Y. 2023 · 2023

Later among the works it cites.

Translating natural language to planning goals with large-language models

Original

Xie, Y.; Yu, C.; Zhu, T.; Bai, J.; Gong, Z.; and Soh, H. 2023 · 2023

Later among the works it cites.

Generating Structured Pseudo Labels for Noise-resistant Zero-shot Video Sentence Localization

Zheng, M.; Gong, S.; Jin, H.; Peng, Y.; and Liu, Y. 2023 · 2023

Later among the works it cites.

Exploring Conditional Multi-Modal Prompts for Zero-shot HOI Detection

Lei, T.; Yin, S.; Peng, Y.; and Liu, Y. 2024 · 2024

Closest in time.