SQL text parsing for information retrieval

Abstract

The concept of using a relational database to perform information retrieval (IR) search functions is well established. Prior work demonstrates the capability to perform common functions and advanced ranking algorithms using standard, unchanged SQL. The previous work does not address the preprocessing of unstructured text within the relational model. In fact, the parsing of the unstructured data into a structured data set was done outside of the database, usually using sequential programming languages such as C. This work proves that IR preprocessing does not require proprietary application code to build the framework necessary for searching document databases. Furthermore, the resulting environment is relational and integrates with other data sources within an organization.