Unleashing the Temporal-Spatial Reasoning Capacity of GPT for Training-Free Audio and Language Referenced Video Object Segmentation

The 2017 davis challenge on video object segmentation

Original

Pont-Tuset, J.; Perazzi, F.; Caelles, S.; Arbeláez, P.; Sorkine-Hornung, A.; and Van Gool, L. 2017 · 2017

Earlier work this paper cites.

Youtube-vos: Sequence-to-sequence video object segmentation

Xu, N.; Yang, L.; Fan, Y.; Yang, J.; Yue, D.; Liang, Y.; Price, B.; Cohen, S.; and Huang, T. 2018 · 2018

Earlier work this paper cites.

Exploring Pre-trained Text-to-Video Diffusion Models for Referring Video Object Segmentation

Original

Zhu, Z.; Feng, X.; Chen, D.; Yuan, J.; Qiao, C.; and Hua, G. 2024 · 2018

Earlier work this paper cites.

Video object segmentation with language referring expressions

Khoreva, A.; Rohrbach, A.; and Schiele, B. 2019 · 2019

Earlier work this paper cites.

Referring image segmentation via cross-modal progressive comprehension

Huang, S.; Hui, T.; Liu, S.; Li, G.; Wei, Y.; Han, J.; Liu, L.; and Li, B. 2020 · 2020

Earlier work this paper cites.

Linguistic structure guided context modeling for referring image segmentation

Hui, T.; Liu, S.; Huang, S.; Li, G.; Yu, S.; Zhang, F.; and Han, J. 2020 · 2020

Earlier work this paper cites.

Urvos: Unified referring video object segmentation network with a large-scale benchmark

Seo, S.; Lee, J.-Y.; and Han, B. 2020 · 2020

Earlier work this paper cites.

Collaborative spatial-temporal modeling for language-queried video actor segmentation

Hui, T.; Huang, S.; Liu, S.; Ding, Z.; Li, G.; Wang, W.; Han, J.; and Wang, F. 2021 · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021 · 2021

Earlier work this paper cites.

Human-centric spatio-temporal video grounding with visual transformers

Tang, Z.; Liao, Y.; Liu, S.; Li, G.; Jin, X.; Jiang, H.; Yu, Q.; and Xu, D. 2021 · 2021

Earlier work this paper cites.

End-to-end referring video object segmentation with multimodal transformers

Botach, A.; Zheltonozhskii, E.; and Baskin, C. 2022 · 2022

Earlier work this paper cites.

Beats: Audio pre-training with acoustic tokenizers

Original

Chen, S.; Wu, Y.; Wang, C.; Liu, S.; Tompkins, D.; Chen, Z.; and Wei, F. 2022 · 2022

Earlier work this paper cites.

Language-bridged spatial-temporal interaction for referring video object segmentation

Ding, Z.; Hui, T.; Huang, J.; Wei, X.; Han, J.; and Liu, S. 2022 · 2022

Earlier work this paper cites.

High-resolution image synthesis with latent diffusion models

Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022 · 2022

Earlier work this paper cites.

Language as queries for referring video object segmentation

Wu, J.; Jiang, Y.; Sun, P.; Yuan, Z.; and Luo, P. 2022 · 2022

Earlier work this paper cites.

Opt: Open pre-trained transformer language models

Original

Zhang, S.; Roller, S.; Goyal, N.; Artetxe, M.; Chen, M.; Chen, S.; Dewan, C.; Diab, M.; Li, X.; Lin, X. V.; et al. 2022 · 2022

Earlier work this paper cites.

Audio–visual segmentation

Zhou, J.; Wang, J.; Zhang, J.; Sun, W.; Zhang, J.; Birchfield, S.; Guo, D.; Kong, L.; Wang, M.; and Zhong, Y. 2022 · 2022

Cited alongside, same era.

Gpt-4 technical report

Original

Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023 · 2023

Cited alongside, same era.

Segment and track anything

Original

Cheng, Y.; Li, L.; Xu, Y.; Li, X.; Yang, Z.; Wang, W.; and Yang, Y. 2023 · 2023

Cited alongside, same era.

MeViS: A large-scale benchmark for video segmentation with motion expressions

Ding, H.; Liu, C.; He, S.; Jiang, X.; and Loy, C. C. 2023 · 2023

Cited alongside, same era.

Html: Hybrid temporal-scale multimodal learning framework for referring video object segmentation

Han, M.; Wang, Y.; Li, Z.; Yao, L.; Chang, X.; and Qiao, Y. 2023 · 2023

Cited alongside, same era.

Audio-visual segmentation with semantics

Original

Zhou, J.; Shen, X.; Wang, J.; Zhang, J.; Sun, W.; Zhang, J.; Birchfield, S.; Guo, D.; Kong, L.; Wang, M.; et al. 2023 · 2023

Later among the works it cites.

Languagebind: Extending video-language pretraining to n-modality by language-based semantic alignment

Original

Zhu, B.; Lin, B.; Ning, M.; Yan, Y.; Cui, J.; Wang, H.; Pang, Y.; Jiang, W.; Zhang, J.; Li, Z.; et al. 2023 · 2023

Later among the works it cites.

Unsupervised Audio-Visual Segmentation with Modality Alignment

Original

Bhosale, S.; Yang, H.; Kanojia, D.; Deng, J.; and Zhu, X. 2024 · 2024

Closest in time.

Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation

Chen, Y.; Liu, Y.; Wang, H.; Liu, F.; Wang, C.; Frazer, H.; and Carneiro, G. 2024 · 2024

Closest in time.

Avsegformer: Audio-visual segmentation with transformer