Fetching the paper…
Reading the bibliography…
Current pre-trained cross-lingual sentence encoders approaches use sentence-level objectives only.
Yinfei Yang, Gustavo Hernandez Abrego, Steve Yuan, Mandy Guo, Qinlan Shen, Daniel Cer, Yun hsuan Sung, Brian Strope, and Ray Kurzweil · 1902
Earlier work this paper cites.
Sentence-bert: Sentence embeddings using siamese bert-networks
Nils Reimers and Iryna Gurevych · 1908
Earlier work this paper cites.
Ccmatrix: Mining billions of high-quality parallel sentences on the web, 2020
Holger Schwenk, Guillaume Wenzek, Sergey Edunov, Edouard Grave, and Armand Joulin · 1911
Earlier work this paper cites.
Nonparametric entropy estimation: An overview
Jan Beirlant, E. Dudewicz, L. Gyor, and E.C. Meulen · 1997
Earlier work this paper cites.
Making monolingual sentence embeddings multilingual using knowledge distillation
Nils Reimers and Iryna Gurevych · 2004
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
Raia Hadsell, Sumit Chopra, and Yann Lecun · 2006
Earlier work this paper cites.
Beyond english-centric multilingual machine translation, 2020
Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin · 2010
Earlier work this paper cites.
DGT-TM: A freely available translation memory in 22 languages
Ralf Steinberger, Andreas Eisele, Szymon Klocek, Spyridon Pilos, and Patrick Schlüter · 2012
Earlier work this paper cites.
Parallel data, tools and interfaces in OPUS
Jörg Tiedemann · 2012
Earlier work this paper cites.
The AMARA corpus: Building parallel language resources for the educational domain
Ahmed Abdelali, Francisco Guzman, Hassan Sajjad, and Stephan Vogel · 2014
Earlier work this paper cites.
A massively parallel corpus: the bible in 100 languages
Christos Christodouloupoulos and Mark Steedman · 2015
Earlier work this paper cites.
Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel, Antonio Torralba, Raquel Urtasun, and Sanja Fidler · 2015
Earlier work this paper cites.
OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles
Pierre Lison and Jörg Tiedemann · 2016
Earlier work this paper cites.
The United Nations parallel corpus v1.0
Michał Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen · 2016
Earlier work this paper cites.
Supervised learning of universal sentence representations from natural language inference data
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loïc Barrault, and Antoine Bordes · 2017
Earlier work this paper cites.
Overview of the second BUCC shared task: Spotting parallel sentences in comparable corpora
Pierre Zweigenbaum, Serge Sharoff, and Reinhard Rapp · 2017
Cited alongside, same era.
Senteval: An evaluation toolkit for universal sentence representations
Alexis Conneau and Douwe Kiela · 2018
Cited alongside, same era.
Effective parallel corpus mining using bilingual sentence embeddings
Mandy Guo, Qinlan Shen, Yinfei Yang, Heming Ge, Daniel Cer, Gustavo Hernandez Abrego, Keith Stevens, Noah Constant, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil · 2018
Cited alongside, same era.
Overview of the Third BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora
Pierre Zweigenbaum, Serge Sharoff, and Reinhard Rapp · 2018
Cited alongside, same era.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2019
xformers: A modular and hackable transformer modelling library
Benjamin Lefaudeux, Francisco Massa, Diana Liskovich, Wenhan Xiong, Vittorio Caggiano, Sean Naren, Min Xu, Jieru Hu, Marta Tintore, Susan Zhang, Patrick Labatut, Daniel Haziza, Luca Wehrstedt, Jeremy Reizenstein, and Grigory Sizov · 2022
Later among the works it cites.
No language left behind: Scaling human-centered machine translation, 2022
NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang · 2022
Later among the works it cites.
RetroMAE: Pre-training retrieval-oriented language models via masked auto-encoder
Shitao Xiao, Zheng Liu, Yingxia Shao, and Zhao Cao · 2022
Later among the works it cites.
ibot: Image bert pre-training with online tokenizer
Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan Yuille, and Tao Kong · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Spreading vectors for similarity search
Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, and Hervé Jégou · 2019
Cited alongside, same era.
Representation learning with contrastive predictive coding, 2019
Aaron van den Oord, Yazhe Li, and Oriol Vinyals · 2019
Cited alongside, same era.
A multiscale visualization of attention in the transformer model
Jesse Vig · 2019
Cited alongside, same era.
TICO-19: the translation initiative for COvid-19
Antonios Anastasopoulos, Alessandro Cattelan, Zi-Yi Dou, Marcello Federico, Christian Federmann, Dmitriy Genzel, Franscisco Guzmán, Junjie Hu, Macduff Hughes, Philipp Koehn, Rosie Lazar, Will Lewis, Graham Neubig, Mengmeng Niu, Alp Öktem, Eric Paquin, Grace Tang, and Sylwia Tur · 2020
Cited alongside, same era.
Unsupervised cross-lingual representation learning at scale
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov · 2020
Cited alongside, same era.
XTREME: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation
Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson · 2020
Cited alongside, same era.
BEit: BERT pre-training of image transformers
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei · 2022
Cited alongside, same era.
xSIM++: An improved proxy to bitext mining performance for low-resource languages
Mingda Chen, Kevin Heffernan, Onur Çelebi, Alexandre Mourachko, and Holger Schwenk · 2023
Later among the works it cites.
Sonar: Sentence-level multimodal and language-agnostic representations, 2023
Paul-Ambroise Duquenne, Holger Schwenk, and Benoît Sagot · 2023
Later among the works it cites.
Dual-alignment pre-training for cross-lingual sentence embedding
Ziheng Li, Shaohan Huang, Zihan Zhang, Zhi-Hong Deng, Qiang Lou, Haizhen Huang, Jian Jiao, Furu Wei, Weiwei Deng, and Qi Zhang · 2023
Later among the works it cites.
MTEB: Massive text embedding benchmark
Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers · 2023
Later among the works it cites.
Revisiting feature prediction for learning visual representations from video
Adrien Bardes, Quentin Garrido, Jean Ponce, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas · 2024
Closest in time.
Vision transformers need registers, 2024
Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski · 2024
Closest in time.
Anisotropy Is Inherent to Self-Attention in Transformers
Nathan Godey, Éric Clergerie, and Benoît Sagot · 2024
Closest in time.
DINOv2: Learning robust visual features without supervision
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski · 2024
Closest in time.
Massive activations in large language models, 2024
Mingjie Sun, Xinlei Chen, J. Zico Kolter, and Zhuang Liu · 2024
Closest in time.
Efficient streaming language models with attention sinks, 2024
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis · 2024
Closest in time.
Universal sentence encoder for English
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, and Ray Kurzweil · 2029
Closest in time.