NLP and LSTM based Prediction for Stock Market

Abstract

This study1 investigates the application of natural language processing (NLP) using pre-trained large language models (LLM) to enhance the predictive capabilities of deep neural networks for the US stock market. By demonstrating that the sentiment factor derived from the large language model can e ectively serve as a predictive factor, this paper develops a straightforward deep learning architecture. This architecture utilises the sentiment factor alongside other common nancial data to classify stock price movements. The paper concludes with a simple quantitative trading strategy based on these classi cation results, illustrating the model's practical applicability and potential for generating pro ts.

The pre-trained large language model employed in this study is DistilBERT (Sanh et al., 2019), a streamlined version of the well-known BERT (Devlin et al., 2018a) language model. This DistilBERT was further pre-trained and ne-tuned on a nancial corpus. Additionally, a legacy state-of-the-art sentiment classi er was also trained to serve as a baseline model. Testing on the dataset reveals that the DistilBERT model, which underwent further pre-training and ne-tuning on a nancial corpus (named FinDistilBERT thereinafter), outperforms the others models.

The nal nancial backtesting, based on the simple quantitative trading strategy, demonstrates that integration of the sentiment analysis with the nancial predictive deep learning models can indeed yield signi cant returns and generate pro ts.