Can't Hide Behind the API: Stealing Black-Box Commercial Embedding Models

Distilling Task-Specific Knowledge From BERT Into Simple Neural Networks

Original

Raphael Tang, Yao Lu, Linqing Liu, Lili Mou, Olga Vechtomova, and Jimmy Lin. 2019 · 1903

Earlier work this paper cites.

Thieves on Sesame Street! Model Extraction of BERT-based APIs

Original

Kalpesh Krishna, Gaurav Singh Tomar, Ankur P Parikh, Nicolas Papernot, and Mohit Iyyer. 2019 · 1910

Earlier work this paper cites.

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Original

Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 1910

Earlier work this paper cites.

Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval

Original

Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. 2020 · 2007

Earlier work this paper cites.

Distilling the Knowledge in a Neural Network

Original

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015

Earlier work this paper cites.

MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

Original

Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2016 · 2016

Earlier work this paper cites.

Stealing Machine Learning Models via Prediction APIs

Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. 2016 · 2016

Earlier work this paper cites.

Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring

Yossi Adi, Carsten Baum, Moustapha Cisse, Benny Pinkas, and Joseph Keshet. 2018 · 2018

Earlier work this paper cites.

SentEval: An Evaluation Toolkit for Universal Sentence Representations

Alexis Conneau and Douwe Kiela. 2018 · 2018

Earlier work this paper cites.

Overview of the TREC 2019 deep learning track

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2019 · 2019

Earlier work this paper cites.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019

Earlier work this paper cites.

Sentence-BERT: Sentence embeddings using Siamese BERT-networks

Nils Reimers and Iryna Gurevych. 2019 · 2019

Cited alongside, same era.

Overview of the TREC 2020 deep learning track

Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2020 · 2020

Cited alongside, same era.

Dense Passage Retrieval for Open-Domain Question Answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020

Cited alongside, same era.

Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

Nils Reimers and Iryna Gurevych. 2020 · 2020

Cited alongside, same era.

CloudLeak: Large-Scale Deep Learning Models Stealing Through Adversarial Examples

Original

Honggang Yu, Kaichen Yang, Teng Zhang, Yun-Yun Tsai, Tsung-Yi Ho, and Yier Jin. 2020 · 2020

Cited alongside, same era.

Large Dual Encoders Are Generalizable Retrievers

Jianmo Ni, Chen Qu, Jing Lu, Zhuyun Dai, Gustavo Hernandez Abrego, Ji Ma, Vincent Zhao, Yi Luan, Keith Hall, Ming-Wei Chang, and Yinfei Yang. 2022 · 2022

Later among the works it cites.

Sentence Embedding Encoders are Easy to Steal but Hard to Defend

Adam Dziedzic, Franziska Boenisch, Mingjian Jiang, Haonan Duan, and Nicolas Papernot. 2023 · 2023

Later among the works it cites.

Towards General Text Embeddings with Multi-stage Contrastive Learning

Original

Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023 · 2023

Later among the works it cites.

Can’t steal? Cont-steal! Contrastive stealing attacks against image encoders

Zeyang Sha, Xinlei He, Ning Yu, Michael Backes, and Yang Zhang. 2023 · 2023

Later among the works it cites.

SimTDE: Simple Transformer Distillation for Sentence Embeddings

Jian Xie, Xin He, Jiyang Wang, Zimeng Qiu, Ali Kebarighotbi, and Farhad Ghassemi. 2023 · 2023

Later among the works it cites.