Throughput-guarantee resource provisioning for streaming analytical workflows in the cloud

Abstract

Nowadays, most data analytical applications comprise of multiple tasks, which can be represented as workflow in nature. Some of data analytical applications, the data requests arrived continuously, such as fraud detection application, order application, etc. Generally, such streaming analytical workflow applications have a rigid requirement on throughput. It is critical to provisioning resource for streaming analytical workflows on a cloud platform with financial cost as minimizing as possible while still guaranteeing system throughput. We propose a cost effective resource provisioning algorithm which can guarantee system throughput. Experiments on the Alibaba cloud indicate that our proposed scheduling algorithm can guarantee the workflow throughput under different intensities of the workloads.