Distributing Data in Real Time Spatial Data Warehouse

Abstract

Nowadays, there are many real-time spatial applications like location-aware services and traffic monitoring and the need for real time spatial data processing becomes more and more important. As a result, there is a tremendous amount of real-time spatial data in real-time spatial data warehouse. The continuous growth in the amount of data seems to outspeed the advance of the traditional centralized real-time spatial data warehouse. As a solution, many organizations use distributed real-time spatial data warehouse (DRTSDW) as a powerful technique to achieve OLAP (On Line Analytical Processing) analysis and business intelligence (BI). Distributing data in real time data warehouse is divided into two steps: partitioning data and their allocation into sites. Several works have proposed many algorithms for partitioning and allocation data. But with a huge amount of real-time spatial data generated, the system performance degrades rapidly, especially in overload situations. In order to deal with this volumetry and to increase query efficiency, we propose a novel approach for partitioning data in real-time spatial data warehouse to find the right number of clusters and to divides the RTSDW into partitions using the horizontal partitioning. Secondly, we suggest our allocation strategy to place the partitions on the sites where they are most used, to minimize data transfers between sites. We have evaluated those proposed approaches using the new TPC-DS (Transaction processing performance council, http://www.tpc.org, 2014) benchmark. The preliminary results show that the approach is quite interesting.