Fetching the paper…
Reading the bibliography…
LLMs have been found to memorize training textual sequences and regurgitate verbatim said sequences during text generation time.
Membership inference attacks from first principles
Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., and Tramer, F · 1914
Earlier work this paper cites.
The enron email dataset database schema and brief statistical report
Shetty, J. and Adibi, J · 2004
Earlier work this paper cites.
Semeval-2012 task 7: Choice of plausible alternatives: An evaluation of commonsense causal reasoning
Gordon, A., Kozareva, Z., and Roemmele, M · 2012
Earlier work this paper cites.
The eu proposal for a general data protection regulation and the roots of the ‘right to be forgotten’
Mantelero, A · 2013
Earlier work this paper cites.
Deep learning with differential privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., and Zhang, L · 2016
Earlier work this paper cites.
The lambada dataset: Word prediction requiring a broad discourse context
Paperno, D., Kruszewski, G., Lazaridou, A., Pham, Q. N., Bernardi, R., Pezzelle, S., Baroni, M., Boleda, G., and Fernández, R · 2016
Earlier work this paper cites.
Think you have solved question answering? try arc, the ai2 reasoning challenge
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O · 2018
Earlier work this paper cites.
Wizard of wikipedia: Knowledge-powered conversational agents
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., and Weston, J · 2018
Earlier work this paper cites.
Leveraging random label memorization for unsupervised pre-training
Pondenkandath, V., Alberti, M., Puran, S., Ingold, R., and Liwicki, M · 2018
Earlier work this paper cites.
Towards empathetic open-domain conversation models: A new benchmark and dataset
Rashkin, H., Smith, E. M., Li, M., and Boureau, Y.-L · 2018
Earlier work this paper cites.
Humans forget, machines remember: Artificial intelligence and the right to be forgotten
Villaronga, E. F., Kieseberg, P., and Li, T · 2018
Earlier work this paper cites.
Mathqa: Towards interpretable math word problem solving with operation-based formalisms
Amini, A., Gabriel, S., Lin, P., Koncel-Kedziorski, R., Choi, Y., and Hajishirzi, H · 2019
Earlier work this paper cites.
SAMSum corpus: A human-annotated dialogue dataset for abstractive summarization
Gliwa, B., Mochol, I., Biesek, M., and Wawer, A · 2019
Earlier work this paper cites.
Pubmedqa: A dataset for biomedical research question answering
Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W., and Lu, X · 2019
Earlier work this paper cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al · 2019
Earlier work this paper cites.
Hellaswag: Can a machine really finish your sentence?
Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y · 2019
Earlier work this paper cites.
Piqa: Reasoning about physical commonsense in natural language
Bisk, Y., Zellers, R., Gao, J., Choi, Y., et al · 2020
Earlier work this paper cites.
Language models are few-shot learners
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al · 2020
Earlier work this paper cites.
Does learning require memorization? a short tale about a long tail
Feldman, V · 2020
Earlier work this paper cites.
The pile: An 800gb dataset of diverse text for language modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al · 2020
Earlier work this paper cites.
Eternal sunshine of the spotless net: Selective forgetting in deep networks
Golatkar, A., Achille, A., and Soatto, S · 2020
Earlier work this paper cites.
Neurologic decoding:(un) supervised neural text generation with predicate logic constraints
Lu, X., West, P., Zellers, R., Bras, R. L., Bhagavatula, C., and Choi, Y · 2020
Earlier work this paper cites.
Can you put it all together: Evaluating conversational agents’ ability to blend skills
Smith, E. M., Williamson, M., Shuster, K., Weston, J., and Boureau, Y.-L · 2020
Cited alongside, same era.
Large-scale differentially private bert
Anil, R., Ghazi, B., Gupta, V., Kumar, R., and Manurangsi, P · 2021
Cited alongside, same era.
Gpt-neo: Large scale autoregressive language modeling with mesh-tensorflow
Black, S., Gao, L., Wang, P., Leahy, C., and Biderman, S · 2021
Cited alongside, same era.
Extracting training data from large language models
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., Herbert-Voss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al · 2021
Cited alongside, same era.
Amnesiac machine learning
Graves, L., Nagisetty, V., and Ganesh, V · 2021
Cited alongside, same era.
Locating and editing factual associations in gpt
Meng, K., Bau, D., Andonian, A., and Belinkov, Y · 2022
Later among the works it cites.
A survey of machine unlearning
Nguyen, T. T., Huynh, T. T., Nguyen, P. L., Liew, A. W.-C., Yin, H., and Nguyen, Q. V. H · 2022
Later among the works it cites.
Unrolling sgd: Understanding factors influencing machine unlearning
Thudi, A., Deza, G., Chandrasekaran, V., and Papernot, N · 2022
Later among the works it cites.
Memorization without overfitting: Analyzing the training dynamics of large language models
Tirumala, K., Markosyan, A., Zettlemoyer, L., and Aghajanyan, A · 2022
Later among the works it cites.
Prompt certified machine unlearning with randomized gradient smoothing and quantization
Zhang, Z., Zhou, Y., Zhao, X., Che, T., and Lyu, L · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Komeili, M., Shuster, K., and Weston, J · 2021
Cited alongside, same era.
Deduplicating training data makes language models better
Lee, K., Ippolito, D., Nystrom, A., Zhang, C., Eck, D., Callison-Burch, C., and Carlini, N · 2021
Cited alongside, same era.
Large language models can be strong differentially private learners
Li, X., Tramer, F., Liang, P., and Hashimoto, T · 2021
Cited alongside, same era.
Winogrande: An adversarial winograd schema challenge at scale
Sakaguchi, K., Bras, R. L., Bhagavatula, C., and Choi, Y · 2021
Cited alongside, same era.
Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Language Model with JAX
Wang, B · 2021
Cited alongside, same era.
Differentially private fine-tuning of language models
Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al · 2021
Cited alongside, same era.
What does it mean for a language model to preserve privacy?
Brown, H., Lee, K., Mireshghallah, F., Shokri, R., and Tramèr, F · 2022
Cited alongside, same era.
Chen, J. and Yang, D · 2023
Later among the works it cites.
Palm: Scaling language modeling with pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al · 2023
Later among the works it cites.
Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher
Chundawat, V. S., Tarun, A. K., Mandal, M., and Kankanhalli, M · 2023
Later among the works it cites.
Preserving privacy through dememorization: An unlearning technique for mitigating memorization risks in language models
Kassem, A., Mahmoud, O., and Saad, S · 2023
Later among the works it cites.
Towards unbounded machine unlearning
Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E · 2023
Later among the works it cites.
Forgetting private textual sequences in language models via leave-one-out ensemble
Liu, Z. and Kalinli, O · 2023
Later among the works it cites.
Membership inference attacks against language models via neighbourhood comparison
Mattern, J., Mireshghallah, F., Jin, Z., Schölkopf, B., Sachan, M., and Berg-Kirkpatrick, T · 2023
Later among the works it cites.
Privacy issues in large language models: A survey
Neel, S. and Chang, P · 2023
Later among the works it cites.
In-context unlearning: Language models as few shot unlearners
Pawelczyk, M., Neel, S., and Lakkaraju, H · 2023
Later among the works it cites.
Neurips 2023 - machine unlearning, 2023
Triantafillou, E., Pedregosa, F., Hayes, J., Kairouz, P., Guyon, I., Kurmanji, M., Dziugaite, G. K., Triantafillou, P., Zhao, K., Hosoya, L. S., Junior, J. C. S. J., Dumoulin, V., Mitliagkas, I., Escalera, S., Wan, J., Dane, S., Demkin, M., and Reade, W · 2023
Later among the works it cites.
Knowledge editing for large language models: A survey
Wang, S., Zhu, Y., Liu, H., Zheng, Z., Chen, C., et al · 2023
Later among the works it cites.
Composing parameter-efficient modules with arithmetic operations, 2023
Zhang, J., Chen, S., Liu, J., and He, J · 2023
Later among the works it cites.
Machine unlearning in learned databases: An experimental analysis
Kurmanji, M., Triantafillou, E., and Triantafillou, P · 2024
Closest in time.
Tofu: A task of fictitious unlearning for llms
Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C., and Kolter, J. Z · 2024
Closest in time.
Large language model unlearning, 2024
Yao, Y., Xu, X., and Liu, Y · 2024
Closest in time.
A comprehensive study of knowledge editing for large language models
Zhang, N., Yao, Y., Tian, B., Wang, P., Deng, S., Wang, M., Xi, Z., Mao, S., Zhang, J., Ni, Y., et al · 2024
Closest in time.