Performance Tools for Big Data Optimization

Abstract

Many big data optimizations have critical performance requirements (e.g., real-time big data analytics), as indicated by the Velocity dimension of 4Vs of big data. To accelerate the big data optimization, users typically rely on detailed performance analysis to identify potential performance bottlenecks. However, due to the large scale and high abstraction of existing big data optimization frameworks (e.g., Apache Hadoop MapReduce), it remains a major challenge to tune the massively distributed systems in a fine granularity. To alleviate the challenges of performance analysis, various performance tools have been proposed to understand the run-time behaviors of big data optimization for performance tuning. In this chapter, we introduce several performance tools for big data optimization from various aspects, including the requirements of ideal performance tools, the challenges of performance tools, and state-of-the-art performance tool examples.