Fully Dynamic Partitioning: Handling Data Skew in Parallel Data Cube Computation

H.J. Lu, J.X. Yu, L. Feng, Z.X. Li

Research output: Contribution to journalArticleAcademicpeer-review

10 Citations (Scopus)

Abstract

Parallel data processing is a promising approach for efficiently computing data cube in relational databases, because most aggregate functions used in OLAP (On-Line Analytical Processing) are distributive functions. This paper studies the issues of handling data skew in parallel data cube computation. We present a fully dynamic partitioning approach that can effectively distribute workload among processing nodes without priori knowledge of data distribution. As supplement, a simple and effective dynamic load balancing mechanism is also incorporated into our algorithm, which further improves the overall performance. Our experimental results indicated that the proposed techniques are effective even when high data skew exists. The results of scale-up and speedup tests are also satisfactory.
Original languageUndefined
Article number10.1023/A:1021567425133
Pages (from-to)181-202
Number of pages22
JournalDistributed and parallel databases
Volume13
Issue number2
DOIs
Publication statusPublished - Mar 2003

Keywords

  • EWI-6316
  • IR-63245
  • DB-DW: DATA WAREHOUSING

Cite this