Fetching the paper…

Reading the bibliography…

2024

SurGen: Text-Guided Diffusion Model for Surgical Video Generation

Cho, Joseph, Schmidgall, Samuel, Zakka, Cyril et al.

Original
PDF
Post on X

Understand

Diffusion-based video generation models have made significant strides, producing outputs with improved visual fidelity, temporal coherence, and user control.

These advancements hold great promise for improving surgical education by enabling more realistic, diverse, and interactive simulation environments.
In this study, we introduce SurGen, a text-guided diffusion model tailored for surgical video synthesis.
SurGen produces videos with the highest resolution and longest duration among existing surgical video generation models.

SurGen: Text-Guided Diffusion Model for Surgical Video Generation · Around

Built on

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer (2019) https://doi.org/10.48550/arXiv.1910.10683 [cs.LG]
1910
Earlier work this paper cites.
Ho, J., Jain, A., Abbeel, P.: Denoising diffusion probabilistic models (2020) arXiv:2006.11239 [cs.LG]
Original
2006
Earlier work this paper cites.
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013) https://doi.org/10.48550/arXiv.1312.6114 . Fixes a typo in the abstract, no other changes
Original
2013
Earlier work this paper cites.
Twinanda, A.P., Shehata, S., Mutter, D., Marescaux, J., Mathelin, M., Padoy, N.: Endonet: A deep architecture for recognition tasks on laparoscopic videos. arXiv preprint 1602.03012
Original
2016
Earlier work this paper cites.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
2016
Earlier work this paper cites.
Badash, I., Burtt, K., Solorzano, C.A., Carey, J.N.: Innovations in surgery simulation: a review of past, current and future techniques. Annals of Translational Medicine 4
2016
Earlier work this paper cites.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S.: Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in Neural Information Processing Systems, vol. 30 (2017). https://doi.org/10.48550/arXiv.1706.08500 . arXiv:1706.08500. https://arxiv.org/abs/1706.08500
Original
2017
Earlier work this paper cites.
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
Original
2017
Earlier work this paper cites.
Hara, K., Kataoka, H., Satoh, Y.: Learning spatio-temporal features with 3d residual networks for action recognition. arXiv preprint arXiv:1708.07632 (2017) https://doi.org/10.48550/arXiv.1708.07632 . To appear in ICCV 2017 Workshop (Chalearn)
Original
2017
Earlier work this paper cites.
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F., Green, T., Back, T., Natsev, P., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)
Original
2017
Earlier work this paper cites.
Unterthiner, T., Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Towards accurate generative models of video: A new metric & challenges (2018) arXiv:1812.01717 [cs.CV]
Original
2018
Earlier work this paper cites.
Dhariwal, P., Nichol, A.: Diffusion models beat gans on image synthesis. arXiv preprint arXiv:2105.05233 (2021). arXiv:2105.05233 [cs.LG]
Original
2021
Earlier work this paper cites.
Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision (2021) https://doi.org/10.48550/arXiv.2103.00020 [cs.CV]
2021
Earlier work this paper cites.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models (2021) arXiv:2112.10752 [cs.CV]
Original
2021
Earlier work this paper cites.

Similar

Arnab, A., Dehghani, M., Heigold, G., Sun, C., Lučić, M., Schmid, C.: Vivit: A video vision transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) (2021). https://doi.org/10.48550/arXiv.2103.15691 . ICCV 2021. Code available at https://github.com/yourlink . https://arxiv.org/abs/2103.15691
Original
2021
Cited alongside, same era.
Kim, S.W., Philion, J., Torralba, A., Fidler, S.: Drivegan: Towards a controllable high-quality neural simulation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021). https://doi.org/10.48550/arXiv.2104.15060 . Oral. https://doi.org/10.48550/arXiv.2104.15060
2021
Cited alongside, same era.
Chambon, P., Bluethgen, C., Delbrouck, J.-B., Sluijs, R.V., Połacin, M., Chaves, J.M.Z., Abraham, T.M., Purohit, S., Langlotz, C.P., Chaudhari, A.: Roentgen: Vision-language foundation model for chest x-ray generation (2022) arXiv:2211.12737 [cs.CV]
Original
2022
Cited alongside, same era.
Peebles, W., Xie, S.: Scalable diffusion models with transformers. arXiv preprint 2212.09748

Then

Veen, D.V., Uden, C.V., Blankemeier, L., Delbrouck, J.-B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E.P., Seehofnerova, A., Rohatgi, N., Hosamani, P., Collins, W., Ahuja, N., Langlotz, C.P., Hom, J., Gatidis, S., Pauly, J., Chaudhari, A.S.: Adapted large language models can outperform medical experts in clinical text summarization. arXiv preprint arXiv:2309.07430 (2024) https://doi.org/10.1038/s41591-024-02855-5 . arXiv:2309.07430 [cs.CL]
Original
2024
Closest in time.
Aali, A., Veen, D.V., Arefeen, Y.I., Hom, J., Bluethgen, C., Reis, E.P., Gatidis, S., Clifford, N., Daws, J., Tehrani, A.S., Kim, J., Chaudhari, A.S.: A benchmark of domain-adapted large language models for generating brief hospital course summaries. arXiv preprint arXiv:2403.05720 (2024). arXiv:2403.05720 [cs.CL]
Original
2024
Closest in time.
Cho, J., Zakka, C., Kaur, D., Shad, R., Wightman, R., Chaudhari, A., Hiesinger, W.: Medisyn: Text-guided diffusion models for broad medical 2d and 3d image synthesis (2024) arXiv:2405.09806 [cs.CV]
Original
2024
Closest in time.
Li, C., Huang, D., Lu, Z., Xiao, Y., Pei, Q., Bai, L.: A survey on long video generation: Challenges, methods, and prospects. arXiv preprint arXiv:2403.16407 (2024). arXiv:2403.16407 [cs.CV]

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2022

Cited alongside, same era.

Chambon, P., Bluethgen, C., Langlotz, C.P., Chaudhari, A.: Adapting pretrained vision-language foundational models to medical imaging domains. Foundation Models for Decision Making Workshop at Neural Information Processing Systems arXiv:2210.04133

2022

Cited alongside, same era.

Yan, B., Liu, R., Kuo, D.E., Adithan, S., Reis, E.P., Kwak, S., Venugopal, V.K., O’Connell, C.P., Saenz, A., Rajpurkar, P., Moor, M.: Style-aware radiology report generation with radgraph and few-shot prompting. arXiv preprint arXiv:2310.17811 (2023) https://doi.org/10.48550/arXiv.2310.17811 . Accepted to Findings of EMNLP 2023

2023

Cited alongside, same era.

Allmendinger, S., Hemmer, P., Queisner, M., Sauer, I., Müller, L., Jakubik, J., Vössing, M., Kühl, N.: Navigating the synthetic realm: Harnessing diffusion-based models for laparoscopic text-to-image generation. arXiv preprint arXiv:2312.03043 (2023). arXiv:2312.03043 [eess.IV]

2023

Cited alongside, same era.

Kaleta, J., Dall’Alba, D., Płotka, S., Korzeniowski, P.: Minimal data requirement for realistic endoscopic image generation with stable diffusion. International Journal of Computer Assisted Radiology and Surgery 19

2023

Cited alongside, same era.

Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S.W., Fidler, S., Kreis, K.: Align your latents: High-resolution video synthesis with latent diffusion models (2023) arXiv:2304.08818 [cs.CV]

2023

Cited alongside, same era.

Lu, H., Yang, G., Fei, N., Huo, Y., Lu, Z., Luo, P., Ding, M.: Vdt: General-purpose video diffusion transformers via mask modeling. arXiv preprint arXiv:2305.13311 (2023). Project page: https://VDT-2023.github.io

Original

2023

Cited alongside, same era.

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019) https://doi.org/10.48550/arXiv.1910.10683 . Version 4, updated on 19 Sep 2023

2023

Cited alongside, same era.

Hu, A., Russell, L., Yeo, H., Murez, Z., Fedoseev, G., Kendall, A., Shotton, J., Corrado, G.: Gaia-1: A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080 (2023)

2023

Cited alongside, same era.

2024

Closest in time.

Varghese, C., Harrison, E.M., O’Grady, G., et al

2024

Closest in time.

Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., Li, Y., Rubinstein, M., Michaeli, T., Wang, O., Sun, D., Dekel, T., Mosseri, I.: Lumiere: A space-time diffusion model for video generation. arXiv preprint arXiv:2401.12945 (2024). arXiv:2401.12945 [cs.CV]

Original

2024

Closest in time.

Nwoye, C.I., Bose, R., Elgohary, K., Arboit, L., Carlino, G., Lavanchy, J.L., Mascagni, P., Padoy, N.: Surgical text-to-image generation. arXiv preprint arXiv:2407.09230 (2024). arXiv:2407.09230 [cs.CV]

2024

Closest in time.

Li, C., Liu, H., Liu, Y., Feng, B.Y., Li, W., Liu, X., Chen, Z., Shao, J., Yuan, Y.: Endora: Video generation models as endoscopy simulators. arXiv preprint arXiv:2403.11050 (2024) https://doi.org/10.48550/arXiv.2403.11050 . Project page: this https URL

2024

Closest in time.

Iliash, I., Allmendinger, S., Meissen, F., Kühl, N., Rückert, D.: Interactive generation of laparoscopic videos with diffusion models. arXiv preprint arXiv:2406.06537 (2024). arXiv:2406.06537 [eess.IV]

2024

Closest in time.

Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al.: Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072 (2024)

2024

Closest in time.

Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., Levi, Y., Lorenz, D., Sauer, A., Boesel, F., Podell, D., Dockhorn, T., English, Z., Lacey, K., Goodwin, A., Marek, Y., Rombach, R.: Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint 2403.03206

2024

Closest in time.

Yu, L., Lezama, J., Gundavarapu, N.B., Versari, L., Sohn, K., Minnen, D., Cheng, Y., Birodkar, V., Gupta, A., Gu, X., Hauptmann, A.G., Gong, B., Yang, M.-H., Essa, I., Ross, D.A., Jiang, L.: Language model beats diffusion – tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737 (2023) https://doi.org/10.48550/arXiv.2310.05737 . ICLR 2024

2024

Closest in time.

Abaid, A., Farooq, M.A., Hynes, N., Corcoran, P., Ullah, I.: Synthesizing cta image data for type-b aortic dissection using stable diffusion models. arXiv preprint arXiv:2402.06969 (2024) arXiv:2402.06969 [cs.CV]. Submitted in IEEE EMBC 2024 Conference

2024

Closest in time.