Fetching the paper…
Reading the bibliography…
Foundation Models (FMs) such as GPT-4 encoded with vast knowledge and powerful emergent abilities have achieved remarkable success in various natural language processing and computer vision tasks.
Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
J.C. Spall. 1992 · 1992
Earlier work this paper cites.
A one-measurement form of simultaneous perturbation stochastic approximation
James C. Spall. 1997 · 1997
Earlier work this paper cites.
A Survey on Transfer Learning
Sinno Jialin Pan and Qiang Yang. 2010 · 2010
Earlier work this paper cites.
Broadening the Scope of Differential Privacy Using Metrics. In Privacy Enhancing Technologies
Konstantinos Chatzikokolakis, Miguel E. Andrés, Nicolás Emilio Bordenabe, and Catuscia Palamidessi. 2013 · 2013
Earlier work this paper cites.
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al · 2014
Earlier work this paper cites.
Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy . IEEE, 463–480
Yinzhi Cao and Junfeng Yang. 2015 · 2015
Earlier work this paper cites.
Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security . 308–318
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016 · 2016
Earlier work this paper cites.
Practical Secure Aggregation for Federated Learning on User-Held Data. In NIPS Workshop on Private Multi-Party Machine Learning
K. A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2016 · 2016
Earlier work this paper cites.
The CMA evolution strategy: A tutorial
Nikolaus Hansen. 2016 · 2016
Earlier work this paper cites.
Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics . PMLR, 1273–1282
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017 · 2017
Earlier work this paper cites.
Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. In Research in Attacks, Intrusions, and Defenses . 273–294
Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018 · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al · 2018
Earlier work this paper cites.
Privacy loss classes: The central limit theorem in differential privacy
David Sommer, Sebastian Meiser, and Esfandiar Mohammadi. 2018 · 2018
Earlier work this paper cites.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . 4171–4186
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Earlier work this paper cites.
Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning . PMLR, 2790–2799
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019 · 2019
Earlier work this paper cites.
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 2019
Earlier work this paper cites.
Language Models are Unsupervised Multitask Learners
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019 · 2019
Earlier work this paper cites.
Overlearning reveals sensitive attributes
Congzheng Song and Vitaly Shmatikov. 2019 · 2019
Earlier work this paper cites.
Federated learning
Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. 2019 · 2019
Earlier work this paper cites.
Eugene: Towards deep intelligence as a service. In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) . 1630–1640
Shuochao Yao, Yifan Hao, Yiran Zhao, Ailing Piao, Huajie Shao, Dongxin Liu, Shengzhong Liu, Shaohan Hu, Dulanga Weerakoon, Kasthuri Jayarajah, et al · 2019
Earlier work this paper cites.
Continual Lifelong Learning in Natural Language Processing: A Survey. In Proceedings of the 28th International Conference on Computational Linguistics . 6523–6541
Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. 2020 · 2020
Earlier work this paper cites.
Group knowledge transfer: Federated learning of large cnns at the edge
Chaoyang He, Murali Annavaram, and Salman Avestimehr. 2020 · 2020
Earlier work this paper cites.
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (Eds.). Association for Computational Linguistics, 7871–7880
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020 · 2020
Earlier work this paper cites.
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020 · 2020
Earlier work this paper cites.
Information Leakage in Embedding Models. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security . 377–390
Congzheng Song and Ananth Raghunathan. 2020 · 2020
Earlier work this paper cites.
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . 7859–7869
Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei, and Ming Zhou. 2020 · 2020
Earlier work this paper cites.
Deep leakage from gradients
Ligeng Zhu and Song Han. 2020 · 2020
Earlier work this paper cites.
Your fairness may vary: Pretrained language model fairness in toxic text classification
Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Mikhail Yurochkin, and Moninder Singh. 2021 · 2021
Earlier work this paper cites.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Earlier work this paper cites.
Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 141–159
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021 · 2021
Earlier work this paper cites.
Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21) . 2633–2650
Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al · 2021
Earlier work this paper cites.
Knowledge distillation: A survey
Jianping Gou, Baosheng Yu, Stephen J Maybank, and Dacheng Tao. 2021 · 2021
Earlier work this paper cites.
Towards a unified view of parameter-efficient transfer learning
Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2021 · 2021
Earlier work this paper cites.
Practical and private (deep) learning without sampling or shuffling. In International Conference on Machine Learning . PMLR, 5213–5225
Peter Kairouz, Brendan McMahan, Shuang Song, Om Thakkar, Abhradeep Thakurta, and Zheng Xu. 2021 · 2021
Earlier work this paper cites.
Invbert: Reconstructing text from contextualized word embeddings by inverting the bert pipeline
Kai Kugler, Simon Münker, Johannes Höhmann, and Achim Rettinger. 2021 · 2021
Earlier work this paper cites.
The power of scale for parameter-efficient prompt tuning
Brian Lester, Rami Al-Rfou, and Noah Constant. 2021 · 2021
Earlier work this paper cites.
Prefix-tuning: Optimizing continuous prompts for generation
Xiang Lisa Li and Percy Liang. 2021 · 2021
Earlier work this paper cites.
CAPE: Context-Aware Private Embeddings for Private Language Learning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing . 7970–7978
Richard Plant, Dimitra Gkatzia, and Valerio Giuffrida. 2021 · 2021
Earlier work this paper cites.
Natural language understanding with privacy-preserving bert. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management . 1488–1497
Chen Qu, Weize Kong, Liu Yang, Mingyang Zhang, Michael Bendersky, and Marc Najork. 2021 · 2021
Earlier work this paper cites.
Learning transferable visual models from natural language supervision. In International conference on machine learning . 8748–8763
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al · 2021
Earlier work this paper cites.
Backdoor Pre-Trained Models Can Transfer to All. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security . 3141–3158
Lujia Shen, Shouling Ji, Xuhong Zhang, Jinfeng Li, Jing Chen, Jie Shi, Chengfang Fang, Jianwei Yin, and Ting Wang. 2021 · 2021
Earlier work this paper cites.
Differential Privacy for Text Analytics via Natural Text Sanitization. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 , Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.). 3853–3866
Xiang Yue, Minxin Du, Tianhao Wang, Yaliang Li, Huan Sun, and Sherman S. M. Chow. 2021 · 2021
Earlier work this paper cites.
Red Alarm for Pre-trained Models: Universal Vulnerability to Neuron-level Backdoor Attacks
Zhengyan Zhang, Guangxuan Xiao, Yongwei Li, Tian Lv, Fanchao Qi, Yasheng Wang, Xin Jiang, Zhiyuan Liu, and Maosong Sun. 2021 · 2021
Earlier work this paper cites.
Efficient Neural Network Training via Forward and Backward Propagation Sparsification. In Advances in Neural Information Processing Systems
Xiao Zhou, WEIZHONG ZHANG, Zonghao Chen, SHIZHE DIAO, and Tong Zhang. 2021 · 2021
Earlier work this paper cites.
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, 5723–5738
Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. 2022 · 2022
Earlier work this paper cites.
LAMP: Extracting Text from Gradients with Language Model Priors. In Advances in Neural Information Processing Systems , Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.)
Mislav Balunovic, Dimitar Iliev Dimitrov, Nikola Jovanović, and Martin Vechev. 2022 · 2022
Earlier work this paper cites.
EW-Tune: A Framework for Privately Fine-Tuning Large Language Models with Differential Privacy. In 2022 IEEE International Conference on Data Mining Workshops (ICDMW) . IEEE, 560–566
Rouzbeh Behnia, Mohammadreza Reza Ebrahimi, Jason Pacheco, and Balaji Padmanabhan. 2022 · 2022
Earlier work this paper cites.
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Association for Computational Linguistics, 1–9
Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022 · 2022
Earlier work this paper cites.
Differentially Private Bias-Term only Fine-tuning of Foundation Models. In Workshop on Trustworthy and Socially Responsible Machine Learning, NeurIPS 2022
Zhiqi Bu, Yu-Xiang Wang, Sheng Zha, and George Karypis. 2022 · 2022
Earlier work this paper cites.
BadPrompt: Backdoor Attacks on Continuous Prompts. In Advances in Neural Information Processing Systems , Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.)
Xiangrui Cai, Haidong Xu, Sihan Xu, Ying Zhang, and Xiaojie Yuan. 2022 · 2022
Earlier work this paper cites.
THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption. In Findings of the Association for Computational Linguistics: ACL 2022 . Association for Computational Linguistics, 3510–3520
Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, and Furu Wei. 2022a · 2022
Earlier work this paper cites.
Georgios Chochlakis, Tejas Srinivasan, Jesse Thomason, and Shrikanth Narayanan. 2022 · 2022
Earlier work this paper cites.
Palm: Scaling language modeling with pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al · 2022
Earlier work this paper cites.
LaMDA: Language Models for Dialog Applications
Aaron Daniel Cohen, Adam Roberts, Alejandra Molina, Alena Butryna, Alicia Jin, Apoorv Kulshreshtha, Ben Hutchinson, Ben Zevenbergen, Blaise Hilary Aguera-Arcas, Chung ching Chang, Claire Cui, Cosmo Du, Daniel De Freitas Adiwardana, Dehao Chen, Dmitry (Dima) Lepikhin, Ed H. Chi, Erin Hoffman-John, Heng-Tze Cheng, Hongrae Lee, Igor Krivokon, James Qin, Jamie Hall, Joe Fenton, Johnny Soraker, Kathy Meier-Hellstern, Kristen Olson, Lora Mois Aroyo, Maarten Paul Bosma, Marc Joseph Pickett, Marcelo Amorim Menegali, Marian Croak, Mark Díaz, Matthew Lamm, Maxim Krikun, Meredith Ringel Morris, Noam Shazeer, Quoc V. Le, Rachel Bernstein, Ravi Rajakumar, Ray Kurzweil, Romal Thoppilan, Steven Zheng, Taylor Bos, Toju Duke, Tulsee Doshi, Vincent Y. Zhao, Vinodkumar Prabhakaran, Will Rusch, YaGuang Li, Yanping Huang, Yanqi Zhou, Yuanzhong Xu, and Zhifeng Chen. 2022 · 2022
Earlier work this paper cites.
Recovering Private Text in Federated Learning of Language Models. In Advances in Neural Information Processing Systems , Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (Eds.)
Samyak Gupta, Yangsibo Huang, Zexuan Zhong, Tianyu Gao, Kai Li, and Danqi Chen. 2022 · 2022
Earlier work this paper cites.
Iron: Private inference on transformers
Meng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing, Guowen Xu, and Tianwei Zhang. 2022 · 2022
Earlier work this paper cites.
Invernet: An Inversion Attack Framework to Infer Fine-Tuning Datasets through Word Embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2022 . 5009–5018
Ishrak Hayet, Zijun Yao, and Bo Luo. 2022 · 2022
Earlier work this paper cites.
LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022 · 2022
Earlier work this paper cites.
DP-Rewrite: Towards Reproducibility and Transparency in Differentially Private Text Rewriting. In Proceedings of the 29th International Conference on Computational Linguistics
Timour Igamberdiev, Thomas Arnold, and Ivan Habernal. 2022 · 2022
Earlier work this paper cites.
You Don’t Know My Favorite Color: Preventing Dialogue Representations from Revealing Speakers’ Private Personas. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, Seattle, United States, 5858–5870
Haoran Li, Yangqiu Song, and Lixin Fan. 2022c · 2022
Earlier work this paper cites.
Explanations from large language models make small reasoners better
Shiyang Li, Jianshu Chen, Yelong Shen, Zhiyu Chen, Xinlu Zhang, Zekun Li, Hong Wang, Jing Qian, Baolin Peng, Yi Mao, et al · 2022
Earlier work this paper cites.
Large language models can be strong differentially private learners
Xuechen Li, Florian Tramer, Percy Liang, and Tatsunori Hashimoto. 2022d · 2022
Earlier work this paper cites.
Federated split bert for heterogeneous text classification. In 2022 International Joint Conference on Neural Networks (IJCNN) . IEEE, 1–8
Zhengyang Li, Shijing Si, Jianzong Wang, and Jing Xiao. 2022b · 2022
Earlier work this paper cites.
FedNLP: Benchmarking Federated Learning Methods for Natural Language Processing Tasks. In Findings of the Association for Computational Linguistics: NAACL 2022 . 157–175
Bill Yuchen Lin, Chaoyang He, Zihang Ze, Hulin Wang, Yufen Hua, Christophe Dupuy, Rahul Gupta, Mahdi Soltanolkotabi, Xiang Ren, and Salman Avestimehr. 2022 · 2022
Cited alongside, same era.
Vertical Federated Learning: Concepts, Advances and Challenges
Yang Liu, Yan Kang, Tianyuan Zou, Yanhong Pu, Yuanqin He, Xiaozhou Ye, Ye Ouyang, Ya-Qin Zhang, and Qiang Yang. 2022 · 2022
Cited alongside, same era.
Differentially Private Language Models for Secure Data Sharing. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
Justus Mattern, Zhijing Jin, Benjamin Weggenmann, Bernhard Schoelkopf, and Mrinmaya Sachan. 2022 · 2022
Cited alongside, same era.
Differentially Private Model Compression. In Advances in Neural Information Processing Systems . 29468–29483
FatemehSadat Mireshghallah, Arturs Backurs, Huseyin A. Inan, Lukas Wutschitz, and Janardhan Kulkarni. 2022 · 2022
Cited alongside, same era.
Who Wrote this Code? Watermarking for Code Generation
Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2023 · 2023
Closest in time.
Privacy in Large Language Models: Attacks, Defenses and Future Directions
Haoran Li, Yulin Chen, Jinglong Luo, Yan Kang, Xiaojin Zhang, Qi Hu, Chunkit Chan, and Yangqiu Song. 2023a · 2023
Closest in time.
Backdoor Threats from Compromised Foundation Models to Federated Learning. In International Workshop on Federated Learning in the Age of Foundation Models in Conjunction with NeurIPS 2023
Xi Li, Songhe Wang, Chen Wu, Hao Zhou, and Jiaqi Wang. 2023e · 2023
Closest in time.
Privacy-preserving prompt tuning for large language model services
Yansong Li, Zhixing Tan, and Yang Liu. 2023c · 2023
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Cited alongside, same era.
Red Teaming Language Models with Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (Eds.). 3419–3448
Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022 · 2022
Cited alongside, same era.
High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 10684–10695
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022 · 2022
Cited alongside, same era.
Bloom: A 176b-parameter open-access multilingual language model
Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al · 2022
Cited alongside, same era.
Selective Differential Privacy for Language Modeling. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Association for Computational Linguistics, 2848–2859
Weiyan Shi, Aiqi Cui, Evan Li, Ruoxi Jia, and Zhou Yu. 2022a · 2022
Cited alongside, same era.
Just Fine-tune Twice: Selective Differential Privacy for Large Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . 6327–6340
Weiyan Shi, Ryan Shea, Si Chen, Chiyuan Zhang, Ruoxi Jia, and Zhou Yu. 2022b · 2022
Cited alongside, same era.
BBTv2: Towards a Gradient-Free Future with Large Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Abu Dhabi, United Arab Emirates
Tianxiang Sun, Zhengfu He, Hong Qian, Yunhua Zhou, Xuanjing Huang, and Xipeng Qiu. 2022a · 2022
Cited alongside, same era.
Lst: Ladder side-tuning for parameter and memory efficient transfer learning
Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. 2022 · 2022
Cited alongside, same era.
Guiding Large Language Models via Directional Stimulus Prompting. In NeurIPS 2023
Zekun Li, Baolin Peng, Pengcheng He, Michel Galley, Jianfeng Gao, and Xifeng Yan. 2023b · 2023
Closest in time.
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers. In The Eleventh International Conference on Learning Representations
Chen Liang, Haoming Jiang, Zheng Li, Xianfeng Tang, Bing Yin, and Tuo Zhao. 2023 · 2023
Closest in time.
Efficient Federated Prompt Tuning for Black-box Large Pre-trained Models
Zihao Lin, Yan Sun, Yifan Shi, Xueqian Wang, Lifu Huang, Li Shen, and Dacheng Tao. 2023 · 2023
Closest in time.
Beyond One-Model-Fits-All: A Survey of Domain Specialization for Large Language Models
Chen Ling, Xujiang Zhao, Jiaying Lu, Chengyuan Deng, Can Zheng, Junxiang Wang, Tanmoy Chowdhury, Yun Li, Hejie Cui, Tianjiao Zhao, et al · 2023
Closest in time.
Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023c · 2023
Closest in time.
LLMs Can Understand Encrypted Prompt: Towards Privacy-Computing Friendly Transformers
Xuanqi Liu and Zhuotao Liu. 2023 · 2023
Closest in time.
Differentially Private Low-Rank Adaptation of Large Language Model Using Federated Learning
Xiao-Yang Liu, Rongyi Zhu, Daochen Zha, Jiechao Gao, Shan Zhong, and Meikang Qiu. 2023d · 2023
Closest in time.
ZooPFL: Exploring Black-box Foundation Models for Personalized Federated Learning
Wang Lu, Hao Yu, Jindong Wang, Damien Teney, Haohan Wang, Yiqiang Chen, Qiang Yang, Xing Xie, and Xiangyang Ji. 2023 · 2023
Closest in time.
Yuhan Ma, Haiqi Jiang, and Chenyou Fan. 2023 · 2023
Closest in time.
Teaching Small Language Models to Reason. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) . Toronto, Canada, 1773–1781
Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2023 · 2023
Closest in time.
Text embeddings reveal (almost) as much as text
John X Morris, Volodymyr Kuleshov, Vitaly Shmatikov, and Alexander M Rush. 2023 · 2023
Closest in time.
A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3
Ha-Thanh Nguyen. 2023 · 2023
Closest in time.
BlackVIP: Black-Box Visual Prompting for Robust Transfer Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 24224–24235
Changdae Oh, Hyeji Hwang, Hee-young Lee, YongTaek Lim, Geunyoung Jung, Jiyoung Jung, Hosik Choi, and Kyungwoo Song. 2023 · 2023
Closest in time.
OpenAI. 2023 · 2023
Closest in time.
Differentially Private In-Context Learning
Ashwinee Panda, Tong Wu, Jiachen T Wang, and Prateek Mittal. 2023 · 2023
Closest in time.
Fairness in language models beyond English: Gaps and challenges
Krithika Ramesh, Sunayana Sitaram, and Monojit Choudhury. 2023 · 2023
Closest in time.
Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Kang Gu, Najrin Sultana, and Shagufta Mehnaz. 2023 · 2023
Closest in time.
A Split-and-Privatize Framework for Large Language Model Fine-Tuning
Xicong Shen, Yang Liu, Huiqi Liu, Jue Hong, Bing Duan, Zirui Huang, Yunlong Mao, Ye Wu, and Di Wu. 2023 · 2023
Closest in time.
Towards expert-level medical question answering with large language models
Karan Singhal, Tao Tu, Juraj Gottweis, Rory Sayres, Ellery Wulczyn, Le Hou, Kevin Clark, Stephen Pfohl, Heather Cole-Lewis, Darlene Neal, et al · 2023
Closest in time.
Joint Prompt Optimization of Stacked LLMs using Variational Inference. In Thirty-seventh Conference on Neural Information Processing Systems
Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, and Nicolas Le Roux. 2023 · 2023
Closest in time.
FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models
Jingwei Sun, Ziyue Xu, Hongxu Yin, Dong Yang, Daguang Xu, Yiran Chen, and Holger R Roth. 2023 · 2023
Closest in time.
Breaching FedMD: Image Recovery via Paired-Logits Inversion Attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 12198–12207
Hideaki Takahashi, Jingjing Liu, and Yang Liu. 2023 · 2023
Closest in time.
InferDPT: Privacy-preserving inference for black-box large language model
Meng Tong, Kejiang Chen, Yuang Qi, Jie Zhang, Weiming Zhang, and Nenghai Yu. 2023 · 2023
Closest in time.
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al · 2023
Closest in time.
Can Public Large Language Models Help Private Cross-device Federated Learning?
Boxin Wang, Yibo Jacky Zhang, Yuan Cao, Bo Li, H Brendan McMahan, Sewoong Oh, Zheng Xu, and Manzil Zaheer. 2023c · 2023
Closest in time.
Promptagent: Strategic planning with language models enables expert-level prompt optimization
Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P Xing, and Zhiting Hu. 2023a · 2023
Closest in time.
PrivateLoRA For Efficient Privacy Preserving LLM
Yiming Wang, Yu Lin, Xiaodong Zeng, and Guannan Zhang. 2023b · 2023
Closest in time.
Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly
Herbert Woisetschläger, Alexander Isenko, Shiqiang Wang, Ruben Mayer, and Hans-Arno Jacobsen. 2023 · 2023
Closest in time.
Lamini-lm: A diverse herd of distilled models from large-scale instructions
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, and Alham Fikri Aji. 2023c · 2023
Closest in time.
Bloomberggpt: A large language model for finance
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023a · 2023
Closest in time.
Leveraging Foundation Models to Improve Lightweight Clients in Federated Learning. In International Workshop on Federated Learning in the Age of Foundation Models in Conjunction with NeurIPS 2023
Xidong Wu, Wan-Yi Lin, Devin Willmott, Filipe Condessa, Yufei Huang, Zhenzhen Li, and Madan Ganesh. 2023b · 2023
Closest in time.
Defending pre-trained language models as few-shot learners against backdoor attacks
Zhaohan Xi, Tianyu Du, Changjiang Li, Ren Pang, Shouling Ji, Jinghui Chen, Fenglong Ma, and Ting Wang. 2023 · 2023
Closest in time.
Offsite-tuning: Transfer learning without full model
Guangxuan Xiao, Ji Lin, and Song Han. 2023 · 2023
Closest in time.
Shuffled Transformer for Privacy-Preserving Split Learning
Hengyuan Xu, Liyao Xiang, Hangyu Ye, Dixi Yao, Pengzhi Chu, and Baochun Li. 2023c · 2023
Closest in time.
Training Large-Vocabulary Neural Language Models by Private Federated Learning for Resource-Constrained Devices. In 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Mingbin Xu, Congzheng Song, Ye Tian, Neha Agrawal, Filip Granqvist, Rogier van Dalen, Xiao Zhang, Arturo Argueta, Shiyi Han, Yaqiao Deng, Leo Liu, Anmol Walia, and Alex Jin. 2023a · 2023
Closest in time.
Federated fine-tuning of billion-sized language models across mobile devices
Mengwei Xu, Yaozong Wu, Dongqi Cai, Xiang Li, and Shangguang Wang. 2023b · 2023
Closest in time.
FinGPT: Open-Source Financial Large Language Models
Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023 · 2023
Closest in time.
Privacy-Preserving Knowledge Transfer through Partial Parameter Sharing. In Proceedings of the 5th Clinical Natural Language Processing Workshop . 19–23
Paul Youssef, Jörg Schlötterer, and Christin Seifert. 2023 · 2023
Closest in time.
Selective Pre-training for Private Fine-tuning
Da Yu, Sivakanth Gopi, Janardhan Kulkarni, Zinan Lin, Saurabh Naik, Tomasz Lukasz Religa, Jian Yin, and Huishuai Zhang. 2023a · 2023
Closest in time.
Multimodal Federated Learning via Contrastive Representation Ensemble
Qiying Yu, Yang Liu, Yimu Wang, Ke Xu, and Jingjing Liu. 2023b · 2023
Closest in time.
Rethinking Mobile AI Ecosystem in the LLM Era
Jinliang Yuan, Chen Yang, Dongqi Cai, Shihe Wang, Xin Yuan, Zeling Zhang, Xiang Li, Dingge Zhang, Hanzi Mei, Xianqing Jia, et al · 2023
Closest in time.
Synthetic Text Generation with Differential Privacy: A Simple and Practical Recipe. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Xiang Yue, Huseyin Inan, Xuechen Li, Girish Kumar, Julia McAnallen, Hoda Shajari, Huan Sun, David Levitan, and Robert Sim. 2023 · 2023
Closest in time.
Is chatgpt fair for recommendation? evaluating fairness in large language model recommendation
Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023a · 2023
Closest in time.
Towards Building the Federated GPT: Federated Instruction Tuning
Jianyi Zhang, Saeed Vahidian, Martin Kuo, Chunyuan Li, Ruiyi Zhang, Guoyin Wang, and Yiran Chen. 2023c · 2023
Closest in time.
GPT-FL: Generative pre-trained model-assisted federated learning
Tuo Zhang, Tiantian Feng, Samiul Alam, Mi Zhang, Shrikanth S Narayanan, and Salman Avestimehr. 2023b · 2023
Closest in time.
FedPETuning: When Federated Learning Meets the Parameter-Efficient Tuning Methods of Pre-trained Language Models. In Findings of the Association for Computational Linguistics: ACL 2023 . 9963–9977
Zhuo Zhang, Yuanhang Yang, Yong Dai, Qifan Wang, Yue Yu, Lizhen Qu, and Zenglin Xu. 2023d · 2023
Closest in time.
FedPrompt: Communication-Efficient and Privacy-Preserving Prompt Tuning in Federated Learning. In ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . 1–5
Haodong Zhao, Wei Du, Fangqi Li, Peixuan Li, and Gongshen Liu. 2023b · 2023
Closest in time.
Provable robust watermarking for ai-generated text
Xuandong Zhao, Prabhanjan Ananth, Lei Li, and Yu-Xiang Wang. 2023a · 2023
Closest in time.
Primer: Fast Private Transformer Inference on Encrypted Data
Mengxin Zheng, Qian Lou, and Lei Jiang. 2023 · 2023
Closest in time.
TextObfuscator: Making Pre-trained Language Model a Privacy Protector via Obfuscating Word Representations. In Findings of the Association for Computational Linguistics: ACL 2023 . 5459–5473
Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Yuran Wang, Yong Ding, Yibo Zhang, Qi Zhang, and Xuanjing Huang. 2023a · 2023
Closest in time.
A survey on model compression for large language models
Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. 2023a · 2023
Closest in time.
PaD: Program-aided Distillation Specializes Large Models in Reasoning
Xuekai Zhu, Biqing Qi, Kaiyan Zhang, Xingwei Long, and Bowen Zhou. 2023b · 2023
Closest in time.
When foundation model meets federated learning: Motivations, challenges, and future directions
Weiming Zhuang, Chen Chen, and Lingjuan Lyu. 2023 · 2023
Closest in time.
Mutual Information Regularization for Vertical Federated Learning
Tianyuan Zou, Yang Liu, and Ya-Qin Zhang. 2023 · 2023
Closest in time.
FedDAT: An Approach for Foundation Model Finetuning in Multi-Modal Heterogeneous Federated Learning
Haokun Chen, Yao Zhang, Denis Krompass, Jindong Gu, and Volker Tresp. 2024 · 2024
Closest in time.
A Fast, Performant, Secure Distributed Training Framework For Large Language Model
Wei Huang, Yinggui Wang, Anda Cheng, Aihui Zhou, Chaofan Yu, and Lei Wang. 2024 · 2024
Closest in time.
Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions
Thuy Dung Nguyen, Tuan Nguyen, Phi Le Nguyen, Hieu H Pham, Khoa D Doan, and Kok-Seng Wong. 2024 · 2024
Closest in time.
CLIP-guided Federated Learning on Heterogeneous and Long-Tailed Data
Jiangming Shi, Shanshan Zheng, Xiangbo Yin, Yang Lu, Yuan Xie, and Yanyun Qu. 2024 · 2024
Closest in time.
Beyond Memorization: Violating Privacy via Inference with Large Language Models. In The Twelfth International Conference on Learning Representations
Robin Staab, Mark Vero, Mislav Balunović, and Martin Vechev. 2024 · 2024
Closest in time.