Deep Reinforcement Learning for Algorithmic Trading Strategies

Abstract

In an ever more complex world of globalized market structures, the academic literature and practitioners in finance have been drawn into applying state-of-the-art artificial intelligence (AI) techniques to algorithmic trading tasks. Conventional methods include technical indicators and rule based systems which have short comings in changing market environment. We study exploiting DRL for building adaptive and profitable trading strategies in this paper. We utilized a DQN and PPO model to compare their performance under U.S. stock market environment. The features included in the models were computed from historical high-frequency data of the S&P 500 index, such as price returns, moving averages and volatility measures. The method was implemented as such that an agent had buy, hold or sell actions in an environment where the rewards were measured by the cumulative portfolio returns considering transaction costs. DQN was a strong performer in stable markets, while PPO overall performed better when markets were more volatile. The experimental results demonstrated that PPO significantly outperformed DQN with satisfactory Sharpe ratio 1.75 and average annualized return over 18%, comparing to the 1.12 of DQN. Both of these models outperformed the buy-and-hold and moving-average crossover type baselines. The results demonstrate that, in dynamic environment, DRL which can learn optimal trading policies adaptively is possible to achieve a notable enhancement of the risk-adjusted return. These findings also emphasize the generalization capability of DRL on the development of intelligent trading systems that address financial market volatility and uncertainty