Built on

Zhang, Y., Li, Y., Chin, D., Xia, G.: Adaptive multimodal music learning via interactive haptic instrument. In: Queiroz, M., Sedó, A.X. (eds.) Proceedings of the International Conference on New Interfaces for Musical Expression. pp. 140–145. UFRGS, Porto Alegre, Brazil (June 2019). https://doi.org/10.5281/zenodo.3672900,
2019
Earlier work this paper cites.
Chin, D., Zhang, Y., Zhang, T., Zhao, J., Xia, G.: Interactive rainbow score: A visual-centered multimodal flute tutoring system. In: Michon, R., Schroeder, F. (eds.) Proceedings of the International Conference on New Interfaces for Musical Expression. pp. 208–213. Birmingham City University, Birmingham, UK (July 2020). https://doi.org/10.5281/zenodo.4813324,
2020
Earlier work this paper cites.
Wang, Z., Chen, K., Jiang, J., Zhang, Y., Xu, M., Dai, S., Xia, G.: Pop909: A pop-song dataset for music arrangement generation. In: Proceedings of the 21st International Society for Music Information Retrieval Conference. pp. 38–45. ISMIR, Montreal, Canada (2020). https://doi.org/10.5281/zenodo.4245366,
2020
Earlier work this paper cites.
Chien, C.M., Lin, J.H., Huang, C.y., Hsu, P.c., Lee, H.y.: Investigating on incorporating pretrained and learnable speaker representations for multi-speaker multi-style text-to-speech. In: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 8588–8592 (2021). https://doi.org/10.1109/ICASSP39728.2021.9413880
2021
Earlier work this paper cites.
Chin, D., Xia, G.: A computer-aided multimodal music learning system with curriculum: A pilot study. In: Proceedings of the International Conference on New Interfaces for Musical Expression. The University of Auckland, New Zealand (jun 2022). https://doi.org/10.21428/92fbeb44.c6910363,
2022
Earlier work this paper cites.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). https://doi.org/10.48550/arXiv.2303.08774
Original
2023
Earlier work this paper cites.
Gao, D., Ji, L., Zhou, L., Lin, K.Q., Chen, J., Fan, Z., Shou, M.Z.: Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn. arXiv preprint arXiv:2306.08640 (2023). https://doi.org/10.48550/arXiv.2306.08640
Original
2023
Earlier work this paper cites.
Gao, S., Shi, Z., Zhu, M., Fang, B., Xin, X., Ren, P., Chen, Z., Ma, J.: Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. arXiv preprint arXiv:2308.14034 (2023). https://doi.org/10.48550/arXiv.2308.14034
Original
2023
Earlier work this paper cites.
Liu, Z., Lai, Z., Gao, Z., Cui, E., Li, Z., Zhu, X., Lu, L., Chen, Q., Qiao, Y., Dai, J., et al.: Controlllm: Augment language models with tools by searching on graphs. arXiv preprint arXiv:2310.17796 (2023). https://doi.org/10.48550/arXiv.2310.17796
Original
2023
Earlier work this paper cites.
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al.: Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789 (2023). https://doi.org/10.48550/arXiv.2307.16789
Original
2023
Earlier work this paper cites.
Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Robust speech recognition via large-scale weak supervision. In: International Conference on Machine Learning. pp. 28492–28518. PMLR (2023)
2023
Earlier work this paper cites.

Similar

Then

ddupont808: GPT-4V-Act
2024
Closest in time.
Liang, Y., Wu, C., Song, T., Wu, W., Xia, Y., Liu, Y., Ou, Y., Lu, S., Ji, L., Mao, S., et al.: Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. Intelligent Computing
2024
Closest in time.
Microsoft: Microsoft copilot: Your everyday ai companion
2024
Closest in time.
Qian, C., He, B., Zhuang, Z., Deng, J., Qin, Y., Cong, X., Lin, Y., Zhang, Z., Liu, Z., Sun, M.: Tell me more! towards implicit user intention understanding of language model driven agents. arXiv preprint arXiv:2402.09205 (2024). https://doi.org/10.48550/arXiv.2402.09205
Original
2024
Closest in time.
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Human-Centered LLM-Agent User Interface: A Position Paper

Chin, Daniel, Wang, Yuxuan, Xia, Gus

Understand

Large Language Model (LLM) -in-the-loop applications have been shown to effectively interpret the human user's commands, make plans, and operate external tools/systems accordingly.

Still, the operation scope of the LLM agent is limited to passively following the user, requiring the user to frame his/her needs with regard to the underlying tools/systems.
We note that the potential of an LLM-Agent User Interface (LAUI) is much greater.
A user mostly ignorant to the underlying tools/systems should be able to work with a LAUI to discover an emergent workflow.

Built on

Zhang, Y., Li, Y., Chin, D., Xia, G.: Adaptive multimodal music learning via interactive haptic instrument. In: Queiroz, M., Sedó, A.X. (eds.) Proceedings of the International Conference on New Interfaces for Musical Expression. pp. 140–145. UFRGS, Porto Alegre, Brazil (June 2019). https://doi.org/10.5281/zenodo.3672900,
2019
Earlier work this paper cites.
Chin, D., Zhang, Y., Zhang, T., Zhao, J., Xia, G.: Interactive rainbow score: A visual-centered multimodal flute tutoring system. In: Michon, R., Schroeder, F. (eds.) Proceedings of the International Conference on New Interfaces for Musical Expression. pp. 208–213. Birmingham City University, Birmingham, UK (July 2020). https://doi.org/10.5281/zenodo.4813324,
2020
Earlier work this paper cites.
Wang, Z., Chen, K., Jiang, J., Zhang, Y., Xu, M., Dai, S., Xia, G.: Pop909: A pop-song dataset for music arrangement generation. In: Proceedings of the 21st International Society for Music Information Retrieval Conference. pp. 38–45. ISMIR, Montreal, Canada (2020). https://doi.org/10.5281/zenodo.4245366,
2020
Earlier work this paper cites.
Chien, C.M., Lin, J.H., Huang, C.y., Hsu, P.c., Lee, H.y.: Investigating on incorporating pretrained and learnable speaker representations for multi-speaker multi-style text-to-speech. In: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 8588–8592 (2021). https://doi.org/10.1109/ICASSP39728.2021.9413880
2021
Earlier work this paper cites.
Chin, D., Xia, G.: A computer-aided multimodal music learning system with curriculum: A pilot study. In: Proceedings of the International Conference on New Interfaces for Musical Expression. The University of Auckland, New Zealand (jun 2022). https://doi.org/10.21428/92fbeb44.c6910363,
2022
Earlier work this paper cites.
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). https://doi.org/10.48550/arXiv.2303.08774
Original
2023
Earlier work this paper cites.
Gao, D., Ji, L., Zhou, L., Lin, K.Q., Chen, J., Fan, Z., Shou, M.Z.: Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn. arXiv preprint arXiv:2306.08640 (2023). https://doi.org/10.48550/arXiv.2306.08640
Original
2023
Earlier work this paper cites.
Gao, S., Shi, Z., Zhu, M., Fang, B., Xin, X., Ren, P., Chen, Z., Ma, J.: Confucius: Iterative tool learning from introspection feedback by easy-to-difficult curriculum. arXiv preprint arXiv:2308.14034 (2023). https://doi.org/10.48550/arXiv.2308.14034
Original
2023
Earlier work this paper cites.
Liu, Z., Lai, Z., Gao, Z., Cui, E., Li, Z., Zhu, X., Lu, L., Chen, Q., Qiao, Y., Dai, J., et al.: Controlllm: Augment language models with tools by searching on graphs. arXiv preprint arXiv:2310.17796 (2023). https://doi.org/10.48550/arXiv.2310.17796
Original
2023
Earlier work this paper cites.
Qin, Y., Liang, S., Ye, Y., Zhu, K., Yan, L., Lu, Y., Lin, Y., Cong, X., Tang, X., Qian, B., et al.: Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789 (2023). https://doi.org/10.48550/arXiv.2307.16789
Original
2023
Earlier work this paper cites.
Radford, A., Kim, J.W., Xu, T., Brockman, G., McLeavey, C., Sutskever, I.: Robust speech recognition via large-scale weak supervision. In: International Conference on Machine Learning. pp. 28492–28518. PMLR (2023)
2023
Earlier work this paper cites.

Similar

Then

ddupont808: GPT-4V-Act
2024
Closest in time.
Liang, Y., Wu, C., Song, T., Wu, W., Xia, Y., Liu, Y., Ou, Y., Lu, S., Ji, L., Mao, S., et al.: Taskmatrix. ai: Completing tasks by connecting foundation models with millions of apis. Intelligent Computing
2024
Closest in time.
Microsoft: Microsoft copilot: Your everyday ai companion
2024
Closest in time.
Qian, C., He, B., Zhuang, Z., Deng, J., Qin, Y., Cong, X., Lin, Y., Zhang, Z., Liu, Z., Sun, M.: Tell me more! towards implicit user intention understanding of language model driven agents. arXiv preprint arXiv:2402.09205 (2024). https://doi.org/10.48550/arXiv.2402.09205
Original
2024
Closest in time.
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., Scialom, T.: Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Original

2023

Cited alongside, same era.

Wu, C., Yin, S., Qi, W., Wang, X., Tang, Z., Duan, N.: Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671 (2023). https://doi.org/10.48550/arXiv.2303.04671

Original

2023

Cited alongside, same era.

Yan, A., Yang, Z., Zhu, W., Lin, K., Li, L., Wang, J., Yang, J., Zhong, Y., McAuley, J., Gao, J., et al.: Gpt-4v in wonderland: Large multimodal models for zero-shot smartphone gui navigation. arXiv preprint arXiv:2311.07562 (2023). https://doi.org/10.48550/arXiv.2311.07562

Original

2023

Cited alongside, same era.

Yang, Z., Liu, J., Han, Y., Chen, X., Huang, Z., Fu, B., Yu, G.: Appagent: Multimodal agents as smartphone users. arXiv preprint arXiv:2312.13771 (2023). https://doi.org/10.48550/arXiv.2312.13771

Original

2023

Cited alongside, same era.

Zhan, Z., Zhang, A.: You only look at screens: Multimodal chain-of-action agents. arXiv preprint arXiv:2309.11436 (2023). https://doi.org/10.48550/arXiv.2309.11436

Original

2023

Cited alongside, same era.

Zhang, Y., Maezawa, A., Xia, G., Yamamoto, K., Dixon, S.: Loop copilot: Conducting ai ensembles for music generation and iterative editing. arXiv preprint arXiv:2310.12404 (2023). https://doi.org/10.48550/arXiv.2310.12404

Original

2023

Cited alongside, same era.

2024

Closest in time.

Shen, W., Li, C., Chen, H., Yan, M., Quan, X., Chen, H., Zhang, J., Huang, F.: Small llms are weak tool learners: A multi-llm agent. arXiv preprint arXiv:2401.07324 (2024). https://doi.org/10.48550/arXiv.2401.07324

Original

2024

Closest in time.

Shen, Y., Song, K., Tan, X., Li, D., Lu, W., Zhuang, Y.: Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems

2024

Closest in time.

Suzzi, M.: Teotronico

2024

Closest in time.

Yang, K., Liu, J., Wu, J., Yang, C., Fung, Y.R., Li, S., Huang, Z., Cao, X., Wang, X., Wang, Y., et al.: If llm is the wizard, then code is the wand: A survey on how code empowers large language models to serve as intelligent agents. arXiv preprint arXiv:2401.00812 (2024). https://doi.org/10.48550/arXiv.2401.00812

Original

2024

Closest in time.

Yang, Z., Zeng, W., Jin, S., Qian, C., Luo, P., Liu, W.: Autommlab: Automatically generating deployable models from language instructions for computer vision tasks. arXiv preprint arXiv:2402.15351 (2024). https://doi.org/10.48550/arXiv.2402.15351

Original

2024

Closest in time.