Extending ETL framework using service oriented architecture

Abstract

Extraction, Transformation and Loading (ETL) represent a big portion of a data warehouse project. Complexity of components extensibility is a main problem in the ETL area, because ETL components are tightly-coupled to each others in the current ETL framework. The missing extensibility feature causes impediments to add new components to the current ETL framework; to meet special business needs.This paper shows how to restructure the current ETL framework based on Service Oriented Architecture(SOA) to be easier to extend. This restructuring solution distributes the ETL into interoperable components. The distribution of Extraction, Transformation and Loading components while keeping interoperability amongst them; can be achieved by SOA. A Classified-Fragmentation component to enhance the report generation speed is added to the new framework; as a proof of the extensibility concept. The result of this work is an extensible ETL framework including Classified-Fragmentation component as an extension.