Built on

J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” arXiv preprint arXiv:1801.06146 , 2018
Original
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A pretrained language model for scientific text,” arXiv preprint arXiv:1903.10676 , 2019
Original
2019
Earlier work this paper cites.
R. Puri and B. Catanzaro, “Zero-shot text classification with generative language models,” arXiv preprint arXiv:1912.10165 , 2019
Original
2019
Earlier work this paper cites.
B. Myagmar, J. Li, and S. Kimura, “Cross-domain sentiment classification with bidirectional contextualized transformer language models,” IEEE Access , vol. 7, pp. 163 219–163 230, 2019
2019
Earlier work this paper cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019
2019
Earlier work this paper cites.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to fine-tune bert for text classification?” in Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings 18 . Springer, 2019, pp. 194–206
2019
Earlier work this paper cites.

Similar

J.-S. Lee and J. Hsiang, “Patent classification by fine-tuning BERT language model,” World Patent Information , vol. 61, p. 101965, 2020
2020
Cited alongside, same era.
M. Fabien, E. Villatoro-Tello, P. Motlicek, and S. Parida, “BertAA: BERT fine-tuning for authorship attribution,” in Proceedings of the 17th International Conference on Natural Language Processing (ICON) , 2020, pp. 127–137
2020
Cited alongside, same era.
W. Antoun, F. Baly, and H. Hajj, “Arabert: Transformer-based model for Arabic language understanding,” arXiv preprint arXiv:2003.00104 , 2020
Original
2020
Cited alongside, same era.
Y. He, Z. Zhu, Y. Zhang, Q. Chen, and J. Caverlee, “Infusing disease knowledge into BERT for health question answering, medical inference and disease name recognition,” arXiv preprint arXiv:2010.03746 , 2020
Original
2020
Cited alongside, same era.

Then

R. Bommasani et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258 , 2021
Original
2021
Later among the works it cites.
X. Chen, P. Cong, and S. Lv, “A long-text classification method of Chinese news based on BERT and CNN,” IEEE Access , vol. 10, pp. 34 046–34 057, 2022
2022
Later among the works it cites.
K. Lagutina, “Topical text classification of Russian news: a comparison of BERT and standard models,” in Conference of Open Innovations Association (FRUCT) . IEEE, 2022, pp. 160–166
2022
Later among the works it cites.
C.-X. Wang, X. You, X. Gao, X. Zhu, Z. Li, C. Zhang, H. Wang, Y. Huang, Y. Chen, H. Haas et al. , “On the road to 6G: Visions, requirements, key technologies and testbeds,” IEEE Commun. Surveys Tuts. , 2023
2023
Closest in time.
J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” arXiv preprint arXiv:2304.03442 , 2023

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2023

Understanding Telecom Language Through Large Language Models

Bariah, Lina, Zou, Hang, Zhao, Qiyang et al.

Understand

The recent progress of artificial intelligence (AI) opens up new frontiers in the possibility of automating many tasks involved in Telecom networks design, implementation, and deployment.

This has been further pushed forward with the evolution of generative artificial intelligence (AI), including the emergence of large language models (LLMs), which is believed to be the cornerstone toward realizing self-governed, interactive AI agents.
Motivated by this, in this paper, we aim to adapt the paradigm of LLMs to the Telecom domain.
In particular, we fine-tune several LLMs including BERT, distilled BERT, RoBERTa and GPT-2, to the Telecom domain languages, and demonstrate a use case for identifying the 3rd Generation Partnership Project (3GPP) standard working groups.

Built on

J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” arXiv preprint arXiv:1801.06146 , 2018
Original
2018
Earlier work this paper cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805 , 2018
Original
2018
Earlier work this paper cites.
I. Beltagy, K. Lo, and A. Cohan, “SciBERT: A pretrained language model for scientific text,” arXiv preprint arXiv:1903.10676 , 2019
Original
2019
Earlier work this paper cites.
R. Puri and B. Catanzaro, “Zero-shot text classification with generative language models,” arXiv preprint arXiv:1912.10165 , 2019
Original
2019
Earlier work this paper cites.
B. Myagmar, J. Li, and S. Kimura, “Cross-domain sentiment classification with bidirectional contextualized transformer language models,” IEEE Access , vol. 7, pp. 163 219–163 230, 2019
2019
Earlier work this paper cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019
2019
Earlier work this paper cites.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to fine-tune bert for text classification?” in Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings 18 . Springer, 2019, pp. 194–206
2019
Earlier work this paper cites.

Similar

J.-S. Lee and J. Hsiang, “Patent classification by fine-tuning BERT language model,” World Patent Information , vol. 61, p. 101965, 2020
2020
Cited alongside, same era.
M. Fabien, E. Villatoro-Tello, P. Motlicek, and S. Parida, “BertAA: BERT fine-tuning for authorship attribution,” in Proceedings of the 17th International Conference on Natural Language Processing (ICON) , 2020, pp. 127–137
2020
Cited alongside, same era.
W. Antoun, F. Baly, and H. Hajj, “Arabert: Transformer-based model for Arabic language understanding,” arXiv preprint arXiv:2003.00104 , 2020
Original
2020
Cited alongside, same era.
Y. He, Z. Zhu, Y. Zhang, Q. Chen, and J. Caverlee, “Infusing disease knowledge into BERT for health question answering, medical inference and disease name recognition,” arXiv preprint arXiv:2010.03746 , 2020
Original
2020
Cited alongside, same era.

Then

R. Bommasani et al. , “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258 , 2021
Original
2021
Later among the works it cites.
X. Chen, P. Cong, and S. Lv, “A long-text classification method of Chinese news based on BERT and CNN,” IEEE Access , vol. 10, pp. 34 046–34 057, 2022
2022
Later among the works it cites.
K. Lagutina, “Topical text classification of Russian news: a comparison of BERT and standard models,” in Conference of Open Innovations Association (FRUCT) . IEEE, 2022, pp. 160–166
2022
Later among the works it cites.
C.-X. Wang, X. You, X. Gao, X. Zhu, Z. Li, C. Zhang, H. Wang, Y. Huang, Y. Chen, H. Haas et al. , “On the road to 6G: Visions, requirements, key technologies and testbeds,” IEEE Commun. Surveys Tuts. , 2023
2023
Closest in time.
J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” arXiv preprint arXiv:2304.03442 , 2023

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

S. Prabhu, M. Mohamed, and H. Misra, “Multi-class text classification using bert-based active learning,” arXiv preprint arXiv:2104.14289 , 2021

Original

2021

Cited alongside, same era.

H. Holm, “Bidirectional encoder representations from transformers (BERT) for question answering in the telecom domain.: Adapting a BERT-like language model to the telecom domain using the ELECTRA pre-training approach,” 2021

2021

Cited alongside, same era.

Original

2023

Closest in time.

“Apache tika - a content analysis toolkit,” https://tika.apache.org

Original

2023

Closest in time.