Deletion Can Be Key for Alternative Delta Loads Using Non-R/3 Data Sources
- by Adrian Gawdiak, Manager, Global Data Warehousing, American Standard Companies
- Nilesh Mehta, Senior Developer and Analyst, Global Data Warehousing, American Standard Companies
- April 1, 2004
Integrating data from third-party systems into BW is a complex and often frustrating process. The authors developed a custom delta load system at their company based on removing data through a systematic deletion process. Sample code is included.
Every BW team knows that implementing effective delta mechanisms is an important part of a smoothly running system. We created an alternative solution for a specific delta load at our company based on removing data via a systematic deletion process. While our solution is not applicable to all non-R/3 source applications, it is well suited for those instances where users are not allowed to change data after a certain period and the data volume loaded into BW is not very large.
First, you should know more about our system before we discuss our alternative delta load. One of the custom-built BW applications developed for our IT landscape uses an Oracle database as a source system. We bring the Oracle data into BW using flat files rather than using DB Connect because the data is further formulated using SQL functions at the time it is written into the files. The files in BW are used as a data source for an InfoPackage.
When loading data, the challenge our team faced was how to bring it in on a routine basis without performing full data loads. It was a complicated problem because in BW the data needs to be stored at a higher level of aggregation than in our source application. To make matters worse, the source application does not keep a history of changes.
Would you like to see this full item?