Adaptive Batch Scheduler: Automatically Decide Parallelism of Flink Batch Jobs
出处:
flink.apache.org
作者:
Lijie Wang & Zhu Zhu
Deciding proper parallelisms of operators is not an easy work for many users. For batch jobs, a small parallelism may result in long execution time and big failover regression. While an unnecessary large parallelism may result in resource waste and more overhead cost in task deployment and network shuffling.