Search Personalization Using Machine Learning

Abstract

Firms typically use query-based search to help consumers find information/products on their websites. We consider the problem of optimally ranking a set of results shown in response to a query. We propose a personalized ranking mechanism based on a user’s search and click history. Our machine-learning framework consists of three modules: (a) feature generation, (b) normalized discounted cumulative gain–based LambdaMART algorithm, and (c) feature selection wrapper. We deploy our framework on large-scale data from a leading search engine using Amazon EC2 servers and present results from a series of counterfactual analyses. We find that personalization improves clicks to the top position by 3.5% and reduces the average error in rank of a click by 9.43% over the baseline. Personalization based on short-term history or within-session behavior is shown to be less valuable than long-term or across-session personalization. We find that there is significant heterogeneity in returns to personalization as a function of user history and query type. The quality of personalized results increases monotonically with the length of a user’s history. Queries can be classified based on user intent as transactional, informational, or navigational, and the former two benefit more from personalization. We also find that returns to personalization are negatively correlated with a query’s past average performance. Finally, we demonstrate the scalability of our framework and derive the set of optimal features that maximizes accuracy while minimizing computing time. This paper was accepted by Juanjuan Zhang, marketing.