Crowdsourcing under Real-Time Constraints

Abstract

In the recent years we are experiencing the rapid growth of crowdsourcing systems, in which "human workers" are enlisted to perform tasks more effectively than computers, and get compensated for the work they provide. The common belief is that the wisdom of the "human crowd" can greatly complement many computer tasks which are assigned to machines. A significant challenge facing these systems is determining the most efficient allocation of tasks to workers to achieve successful completion of the tasks under real-time constraints. This paper presents REACT, a crowdsourcing system that seeks to address this challenge and proposes algorithms that aim to stimulate user participation and handle dynamic task assignment and execution in the crowdsourcing system. The goal is to determine the most appropriate workers to assign incoming tasks, in such a way so that the realtime demands are met and high quality results are returned. We empirically evaluate our approach and show that REACT meets the requested real-time demands, achieves good accuracy, is efficient, and improves the amount of successful tasks that meet their deadlines up to 61% compared to traditional approaches like AMT.