Confidential Prompting: Privacy-preserving LLM Inference on Cloud · Around

Fetching the paper…

Confidential Prompting: Privacy-preserving LLM Inference on Cloud · Around

Built on

M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro, “Megatron-LM: Training multi-billion parameter language models using model parallelism,” CoRR , vol. abs/1909.08053, 2019. [Online]. Available: http://arxiv.org/abs/1909.08053
Original
1909
Earlier work this paper cites.
O. Goldreich, Foundations of cryptography: volume 2, basic applications . Cambridge university press, 2001, vol. 2
2001
Earlier work this paper cites.
F. McKeen, I. Alexandrovich, A. Berenzon, C. V. Rozas, H. Shafi, V. Shanbhogue, and U. R. Savagaonkar, “Innovative instructions and software model for isolated execution.” Hasp@ isca , vol. 10, no. 1, 2013
2013
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems (NeurIPS 2017) , vol. 30, 2017
2017
Earlier work this paper cites.
M. Milakov and N. Gimelshein, “Online normalizer calculation for softmax,” arXiv preprint arXiv:1805.02867 , 2018
Original
2018
Earlier work this paper cites.
J. Vig and Y. Belinkov, “Analyzing the structure of attention in a transformer language model,” arXiv preprint arXiv:1906.04284 , 2019
Original
2019
Earlier work this paper cites.
K. Clark, U. Khandelwal, O. Levy, and C. D. Manning, “What does bert look at? an analysis of bert’s attention,” arXiv preprint arXiv:1906.04341 , 2019
Original
2019
Earlier work this paper cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019
2019
Earlier work this paper cites.
M. Ott, S. Edunov, A. Baevski, A. Fan, S. Gross, N. Ng, D. Grangier, and M. Auli, “fairseq: A fast, extensible toolkit for sequence modeling,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Demonstrations , W. Ammar, A. Louis, and N. Mostafazadeh, Eds. Association for Computational Linguistics, 2019, pp. 48–53. [Online]. Available: https://doi.org/10.18653/v1/n19-4009
2019
Earlier work this paper cites.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in Neural Information Processing Systems (NeurIPS 2020) , vol. 33, pp. 1877–1901, 2020
2020
Earlier work this paper cites.
T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, and A. M. Rush, “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations . Online: Association for Computational Linguistics, Oct. 2020, pp. 38–45. [Online]. Available: https://www.aclweb.org/anthology/2020.emnlp-demos.6
2020
Earlier work this paper cites.
M. Li, Y. Zhang, H. Wang, K. Li, and Y. Cheng, “CIPHERLEAKS: Breaking constant-time cryptography on AMD SEV via the ciphertext side channel,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 717–732
2021
Earlier work this paper cites.
Z. Huang, W.-j. Lu, C. Hong, and J. Ding, “Cheetah: Lean and fast secure two-party deep neural network inference,” in 31st USENIX Security Symposium (USENIX Security 22) , 2022, pp. 809–826
2022
Earlier work this paper cites.
M. Hao, H. Li, H. Chen, P. Xing, G. Xu, and T. Zhang, “Iron: Private inference on transformers,” Advances in Neural Information Processing Systems (NeurIPS 2022) , vol. 35, pp. 15 718–15 731, 2022
2022
Earlier work this paper cites.
S. Zhao, M. Li, Y. Zhang, and Z. Lin, “vSGX: Virtualizing SGX enclaves on AMD SEV,” in 2022 IEEE Symposium on Security and Privacy (SP) . IEEE, 2022, pp. 321–336
2022
Earlier work this paper cites.
W. Diffie and M. E. Hellman, “New directions in cryptography,” in Democratizing cryptography: the work of Whitfield Diffie and Martin Hellman , 2022, pp. 365–390
2022
Earlier work this paper cites.
D. Li, R. Shao, H. Wang, H. Guo, E. P. Xing, and H. Zhang, “MPCFormer: fast, performant and private transformer inference with MPC,” arXiv preprint arXiv:2211.01452 , 2022
Original
2022
Earlier work this paper cites.
T. Chen, H. Bao, S. Huang, L. Dong, B. Jiao, D. Jiang, H. Zhou, J. Li, and F. Wei, “THE-X: Privacy-preserving transformer inference with homomorphic encryption,” arXiv preprint arXiv:2206.00216 , 2022
Original
2022
Earlier work this paper cites.
T. Wu, A. Panda, J. T. Wang, and P. Mittal, “Privacy-preserving in-context learning for large language models,” arXiv preprint arXiv:2305.01639 , 2023
Original
2023
Earlier work this paper cites.
X. Tang, R. Shin, H. A. Inan, A. Manoel, F. Mireshghallah, Z. Lin, S. Gopi, J. Kulkarni, and R. Sim, “Privacy-preserving in-context learning with differentially private few-shot generation,” arXiv preprint arXiv:2309.11765 , 2023
Original
2023
Earlier work this paper cites.
Y. Chen, T. Li, H. Liu, and Y. Yu, “Hide and Seek (HaS): A lightweight framework for prompt privacy protection,” arXiv preprint arXiv:2309.03057 , 2023
Original
2023
Earlier work this paper cites.
Z. Kan, L. Qiao, H. Yu, L. Peng, Y. Gao, and D. Li, “Protecting user privacy in remote conversational systems: A privacy-preserving framework based on text sanitization,” arXiv preprint arXiv:2306.08223 , 2023
Original
2023
Earlier work this paper cites.
J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “GPT-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023
Original
2023
Earlier work this paper cites.
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “LLaMA: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023
Original
2023
Earlier work this paper cites.
R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean, “Efficiently scaling transformer inference,” Proceedings of Machine Learning and Systems , vol. 5, pp. 606–624, 2023
2023
Earlier work this paper cites.
I. Gim, G. Chen, S.-s. Lee, N. Sarda, A. Khandelwal, and L. Zhong, “Prompt Cache: Modular attention reuse for low-latency inference,” arXiv preprint arXiv:2311.04934 , 2023
Original
2023
Earlier work this paper cites.
H. Qin, Z. Song, W. Zhang, S. Huang, W. Yao, G. Liu, X. Jia, and H. Du, “Protecting encrypted virtual machines from nested page fault controlled channel,” in Proceedings of the Thirteenth ACM Conference on Data and Application Security and Privacy , 2023, pp. 165–175
2023
Earlier work this paper cites.
E. Perot, “Running stable diffusion on gpu with gvisor,” 2025, Accessed: 2025-11-10. [Online]. Available: https://gvisor.dev/blog/2023/06/20/gpu-pytorch-stable-diffusion/
2023
Earlier work this paper cites.

Similar

A. Panda, T. Wu, J. T. Wang, and P. Mittal, “Differentially private in-context learning,” arXiv preprint arXiv:2305.01639 , 2023
Original
2023
Cited alongside, same era.
Y. Akimoto, K. Fukuchi, Y. Akimoto, and J. Sakuma, “Privformer: Privacy-preserving transformer with MPC,” in 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P) . IEEE, 2023, pp. 392–410
2023
Cited alongside, same era.
X. Liu and Z. Liu, “LLMs can understand encrypted prompt: Towards privacy-computing friendly transformers,” arXiv preprint arXiv:2305.18396 , 2023
Original
2023
Cited alongside, same era.
P. Mai, Y. Yang, R. Yan, R. Ye, and Y. Pang, “ConfusionPrompt: Practical private inference for online large language models,” Available at SSRN 5046754 , 2023
2023
Cited alongside, same era.

Then

——, “Publish, Audit, Attest: How Tinfoil Builds Trust,” 2025, Accessed: 2025-11-10. [Online]. Available: https://tinfoil.sh/blog/2025-01-13-how-tinfoil-builds-trust
2025
Closest in time.
——, “Detailed Attestation Architecture,” 2025, Accessed: 2025-11-10. [Online]. Available: https://docs.tinfoil.sh/verification/attestation-architecture
2025
Closest in time.
Nvidia, “Nvidia confidential computing,” 2023, Accessed: 2025-11-10. [Online]. Available: https://www.nvidia.com/en-us/data-center/solutions/confidential-computing
2025
Closest in time.
——, “NVIDIA Confidential Computing Whitepaper,” 2023, Accessed: 2025-11-10. [Online]. Available: https://images.nvidia.com/aem-dam/en-zz/Solutions/data-center/HCC-Whitepaper-v1.0.pdf
2025
Closest in time.
——, “NVIDIA Blackwell Architecture,” 2025, Accessed: 2025-11-10. [Online]. Available: https://www.nvidia.com/en-us/data-center/technologies/blackwell-architecture/

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…