Category:ETL Specialist
From Business Intelligence Compendium
The responsibilities of an ETL Specialist (Application Programmer in case this done using manual coding)include:
- Sourcing the data from the operational systems
- Applying the business transformation rules
- Preparing a database-loadable file for the Data Warehouse
- Management of the deployment of the data acquisition tool(s)
- Contributing the technical metadata to the metadata repository
Roles and responsibilities
A brief description of each of these responsibilities follows.
- The Data Warehouse Application Programmer will extract data destined for the Data Warehouse from the operational systems that store the data. The specific tasks are:
- Work with the source system analysts to understand the windows available for data extraction
- Program, test, implement and maintain any data extraction programs necessary to extract the data from the operational systems needed to be moved to the Data Warehouse
- The Data Warehouse Application Programmer is responsible for applying transformation rules as necessary to keep the data clean and consistent and therefore usable by the user community. Specific tasks include:
- Participation in design sessions chaired by data stewards and/or IT personnel where decisions are made involving the transformation from source to target
- Programming the data acquisition tool with the rules to be applied to the data
- Ensuring the correct application of the business rules through data query after the data is loaded into the Data Warehouse
- Following the data extraction and rules application, the file(s) will need to be made ready for loading into the Data Warehouse. This is the responsibility of the Data Warehouse Application Programmer. Specific duties include
- Participation in design sessions chaired by data stewards and/or IT personnel where decisions are made involving the transformation from source to target
- Programming the data acquisition tool with the rules to be applied to the data
- Ensuring the correct application of the business rules through data query after the data is loaded into the Data Warehouse
- Following the data extraction and rules application, the file(s) will need to be made ready for loading into the Data Warehouse. This is the responsibility of the Data Warehouse Application Programmer. Specific duties include:
- Obtaining complete knowledge of the physical database schema
- Preparing the files needed to load each table that has been designated to receive files from operational systems rather than direct input from data stewards
- Programming the data acquisition tool(s) with the tables to load the files into
- Working with the Data Warehouse Database Administrator to ensure the file loads properly into the Data Warehouse
The metadata repository will contain technical metadata such as data sources and transformation rules applied. The Application Programmer will be responsible for the entry and maintenance of this information.
Skills
- Masters the used ETL tool or programming language used to perform ETL tasks
- Good understanding of standard BI data modeling standards and practices
- Good database knowledge
Experts
The below list of users are registered users on the Business Intelligence Compendium who have experience in this role.
This category currently contains no pages or media.
