2023

WeaverBird: Empowering Financial Decision-Making with Large Language Model, Knowledge Base, and Search Engine

Xue, Siqiao, Zhou, Fan, Xu, Yi et al.

Understand

We present WeaverBird, an intelligent dialogue system designed specifically for the finance domain.

Our system harnesses a large language model of GPT architecture that has been tuned using extensive corpora of finance-related text.
As a result, our system possesses the capability to understand complex financial queries, such as "How should I manage my investments during inflation?", and provide informed responses.
Furthermore, our system incorporates a local knowledge base and a search engine to retrieve relevant information.

Built on

Squad: 100,000+ questions for machine comprehension of text. In
Original
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016
Earlier work this paper cites.
Billion-scale similarity search with GPUs
Original
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019 · 2019
Earlier work this paper cites.
Language models are few-shot learners. In
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 2020
Earlier work this paper cites.
Dense passage retrieval for open-domain question answering. In
Original
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020 · 2020
Earlier work this paper cites.
Generalization through memorization: Nearest neighbor language models. In
Original
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020 · 2020
Earlier work this paper cites.
How Much Knowledge Can You Pack Into the Parameters of a Language Model?. In
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020 · 2020
Earlier work this paper cites.
Lora: Low-rank adaptation of large language models
Original
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021 · 2021
Earlier work this paper cites.
Webgpt: Browser-assisted question-answering with human feedback
Original
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al · 2021
Earlier work this paper cites.
Improving language models by retrieving from trillions of tokens. In
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al · 2022
Earlier work this paper cites.
LangChain
Harrison Chase. 2022 · 2022
Earlier work this paper cites.
GLM: General Language Model Pretraining with Autoregressive Blank Infilling. In
Original
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022 · 2022
Earlier work this paper cites.
Unsupervised Dense Information Retrieval with Contrastive Learning
Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022a · 2022
Earlier work this paper cites.

Similar

Few-shot learning with retrieval augmented language models
Original
Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022b · 2022
Cited alongside, same era.
StreamingQA: A Benchmark for Adaptation to New Knowledge over Time in Question Answering Models
Original
Adam Liška, Tomáš Kočiský, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, Cyprien de Masson d’Autume, Tim Scholtes, Manzil Zaheer, Susannah Young, Ellen Gilsenan-McMahon Sophia Austin, Phil Blunsom, and Angeliki Lazaridou. 2022 · 2022
Cited alongside, same era.
PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods
Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, and Sayak Paul. 2022 · 2022
Cited alongside, same era.
Teaching language models to support answers with verified quotes
Original
Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al · 2022
Cited alongside, same era.
Nearest neighbor zero-shot inference. In
Weijia Shi, Julian Michael, Suchin Gururangan, and Luke Zettlemoyer. 2022 · 2022
Cited alongside, same era.
Text Embeddings by Weakly-Supervised Contrastive Pre-training
Original
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022 · 2022
Cited alongside, same era.
Memorizing Transformers. In
Yuhuai Wu, Markus Norman Rabe, DeLesley Hutchins, and Christian Szegedy. 2022 · 2022
Cited alongside, same era.
Glm-130b: An open bilingual pre-trained model
Original
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al · 2022
Cited alongside, same era.
Training language models with memory augmentation. In
Original
Zexuan Zhong, Tao Lei, and Danqi Chen. 2022 · 2022
Cited alongside, same era.
Palm 2 technical report
Original
Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al · 2023
Cited alongside, same era.
wealth-alpaca
Gaurang Bharti. 2023 · 2023
Cited alongside, same era.
Textbooks Are All You Need
Original
Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al · 2023
Cited alongside, same era.

Then

Is GPT-4 a reliable rater? Evaluating Consistency in GPT-4 Text Ratings
Original
Veronika Hackl, Alexandra Elena Müller, Michael Granitzer, and Maximilian Sailer. 2023 · 2023
Closest in time.
Time-LLM: Time Series Forecasting by Reprogramming Large Language Models
Original
Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. 2023a · 2023
Closest in time.
LangChain-ChatGLM
LangChain-ChatGLM. 2023 · 2023
Closest in time.
WebGLM: Towards An Efficient Web-Enhanced Question Answering System with Human Preferences
Original
Xiao Liu, Hanyu Lai, Hao Yu, Yifan Xu, Aohan Zeng, Zhengxiao Du, Peng Zhang, Yuxiao Dong, and Jie Tang. 2023 · 2023
Closest in time.
GPT-4 Technical Report
Original
OpenAI. 2023 · 2023
Closest in time.
In-context retrieval-augmented language models
Original
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023 · 2023
Closest in time.
Toolformer: Language models can teach themselves to use tools
Original
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023 · 2023
Closest in time.
Replug: Retrieval-augmented black-box language models
Original
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2023 · 2023
Closest in time.
Bloomberggpt: A large language model for finance
Original
Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023 · 2023
Closest in time.
FinGPT: Open-Source Financial Large Language Models
Original
Hongyang Yang, Xiao-Yang Liu, and Christina Dan Wang. 2023 · 2023
Closest in time.
Docprompting: Generating code by retrieving the docs. In
Original
Shuyan Zhou, Uri Alon, Frank F Xu, Zhengbao Jiang, and Graham Neubig. 2023 · 2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…