Revolutionizing Mobile Interaction: Enabling a 3 Billion Parameter GPT LLM on Mobile

Built on

Attention is all you need
Original
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I., 2017 · 2017
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
Original
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., 2021 · 2021
Earlier work this paper cites.
Openai: An ai research laboratory URL: https://openai.com/
OpenAI, 2021 · 2021
Earlier work this paper cites.
Robust speech recognition via large-scale weak supervision
Original
Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I., 2022 · 2022
Earlier work this paper cites.

ggerganov/ggml/gpt-neox
ggerganov, 2023a · 2023
Cited alongside, same era.
PEFT: Practical entity framework for transformer
Hugging Face, 2023 · 2023
Cited alongside, same era.
Awq: Activation-aware weight quantization for llm compression and acceleration
Original
Lin, J., Tang, J., Tang, H., Yang, S., Dang, X., Han, S., 2023 · 2023
Cited alongside, same era.
Gpt-4 technical report
Original
OpenAI, 2023 · 2023
Cited alongside, same era.
llama.cpp
ggerganov, 2023b
Cited in the paper.

RedPajama-INCITE-Chat-3B-v1 model on hugging face
togethercomputer, 2023 · 2023
Closest in time.
Llama: Open and efficient foundation language models
Original
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G., 2023 · 2023
Closest in time.
Bloom: A 176b-parameter open-access multilingual language model
Original
Workshop, B., 2023 · 2023
Closest in time.