Extractive summarization of personal photos from life events

Abstract

Manually sifting through large collections of personal photos shot at various life events is both tedious and inefficient. In this paper, we propose a photo summarization system which creates a representative subset summary by extracting photos from a larger set shot at an event (e.g., in a trip, birthday, etc). We define three properties that are necessary to generate an effective summary: relevance, diversity and coverage. We propose methods to compute them using multimodal content and context data. The objective for automatic photo summarization is formulated as an optimization of these properties. We discuss algorithms that solve the problem efficiently. A dataset of 7,700 photos from personal life events is created with user-generated ground truth summary. We also propose objective metrics to evaluate summaries automatically. Evaluations using both objective metrics and user feedback show our models can generate summaries which are much better than baselines.