Divide and Conquer Framework with Feature Partitioning Concepts

Abstract

Divide-and-Conquer (DC) approach is a classical well-adopted paradigm for designing algorithms. In current big data scenarios, processing of voluminous and variety of data is required. One of the characteristics is, large-dimensional data that needs to be analyzed; for example, high resolution images used in social media are used for sentiment analysis. Our research is oriented towards discovering approaches where stage-by-stage processing is done to bring out most salient features from high-dimensional data. However, we observe that data block processing, in most of the conventional approaches, does not scale well for higher dimensionality. Instead, we think of making blocks along the feature set and we propose a divide-and-conquer based feature extraction framework based on feature set partitioning. We demonstrate the effectiveness of the proposed framework using various feature set partitioning based PCA approaches.