Distributed multidimensional suffix arrays for string search

Abstract

Although the classical B-tree and its derivatives are space efficient data structures for small string documents, they require many structures and an excessive amount of pointer manipulations for efficiently processing and searching large documents. In this paper, we propose a new and conceptually powerful data structure, called multidimensional suffix arrays (MSA), for string search. This data structure combines the features of suffix arrays and multilevel structures into a new approach which is very amenable to implementation on both conventional and clustered computers. Algorithms for searching, updating, reorganizing, and distributing multidimensional suffix arrays are discussed. Moreover, MSA achieve a significant improvement over string B-trees.