ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform

Lili Xu; Edin Muharemagc; Amy Apon

doi:10.1109/BigData.2017.8258263

ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform

Lili Xu, Edin Muharemagc, Amy Apon

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

The proliferation of Big Data processing environments such as Hadoop, Apache Spark, and HPCC Systems is driving the development of performance analysis tools in these distributed systems. The goal is to achieve high performance through the optimization of Big Data applications. However, tuning performance in a fine-grained manner is quite challenging due to the high complexity and massive size of the distributed systems. ECL-Watch is a data-flow based fine-grained comprehensive Big Data performance analysis tool utilizing the high level declarative dataflow programming language ECL in HPCC Systems. As a case study, we implement and optimize the Yinyang K-Means machine learning algorithm in ECL in HPCC Systems. The experimental results show that the performance of the native ECL version of the Yinyang K-Means algorithm increased significantly after tuning: from being about three times slower than the standard K-Means implementation in ECL, to become roughly 15% faster than standard K-Means.

Original language	American English
Title of host publication	IEEE International Conference
Editors	Jian-Yun Nie, Zoran Obradovic, Toyotaro Suzumura, Rumi Ghosh, Raghunath Nambiar, Chonggang Wang, Hui Zang, Ricardo Baeza-Yates, Ricardo Baeza-Yates, Xiaohua Hu, Jeremy Kepner, Alfredo Cuzzocrea, Jian Tang, Masashi Toyoda
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	2941-2950
Number of pages	10
ISBN (Electronic)	9781538627143
DOIs	https://doi.org/10.1109/BigData.2017.8258263
State	Published - 2017
Event	5th IEEE International Conference on Big Data, Big Data 2017 - Boston, United States Duration: Dec 11 2017 → Dec 14 2017

Publication series

Name	Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017
Volume	2018-January

Conference

Conference	5th IEEE International Conference on Big Data, Big Data 2017
Country/Territory	United States
City	Boston
Period	12/11/17 → 12/14/17

Keywords

Big Data
Distributed Computing
HPCC Systems
Machine Learning
Performance Analysis
Tuning and Optimization

Access to Document

10.1109/BigData.2017.8258263

https://ieeexplore.ieee.org/document/8258263

Cite this

Xu, L., Muharemagc, E., & Apon, A. (2017). ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform. In J.-Y. Nie, Z. Obradovic, T. Suzumura, R. Ghosh, R. Nambiar, C. Wang, H. Zang, R. Baeza-Yates, R. Baeza-Yates, X. Hu, J. Kepner, A. Cuzzocrea, J. Tang, & M. Toyoda (Eds.), IEEE International Conference (pp. 2941-2950). (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/BigData.2017.8258263

Xu, Lili ; Muharemagc, Edin ; Apon, Amy. / ECL-watch: A big data application performance tuning tool in the HPCC systems platform : A big data application performance tuning tool in the HPCC systems platform. IEEE International Conference. editor / Jian-Yun Nie ; Zoran Obradovic ; Toyotaro Suzumura ; Rumi Ghosh ; Raghunath Nambiar ; Chonggang Wang ; Hui Zang ; Ricardo Baeza-Yates ; Ricardo Baeza-Yates ; Xiaohua Hu ; Jeremy Kepner ; Alfredo Cuzzocrea ; Jian Tang ; Masashi Toyoda. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 2941-2950 (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017).

@inproceedings{8984d1a5a23c45ce84295cf7d9ccba81,

title = "ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform",

abstract = "The proliferation of Big Data processing environments such as Hadoop, Apache Spark, and HPCC Systems is driving the development of performance analysis tools in these distributed systems. The goal is to achieve high performance through the optimization of Big Data applications. However, tuning performance in a fine-grained manner is quite challenging due to the high complexity and massive size of the distributed systems. ECL-Watch is a data-flow based fine-grained comprehensive Big Data performance analysis tool utilizing the high level declarative dataflow programming language ECL in HPCC Systems. As a case study, we implement and optimize the Yinyang K-Means machine learning algorithm in ECL in HPCC Systems. The experimental results show that the performance of the native ECL version of the Yinyang K-Means algorithm increased significantly after tuning: from being about three times slower than the standard K-Means implementation in ECL, to become roughly 15\% faster than standard K-Means.",

keywords = "Big Data, Distributed Computing, HPCC Systems, Machine Learning, Performance Analysis, Tuning and Optimization",

author = "Lili Xu and Edin Muharemagc and Amy Apon",

note = "Publisher Copyright: {\textcopyright} 2017 IEEE.; 5th IEEE International Conference on Big Data, Big Data 2017 ; Conference date: 11-12-2017 Through 14-12-2017",

year = "2017",

doi = "10.1109/BigData.2017.8258263",

language = "American English",

series = "Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "2941--2950",

editor = "Jian-Yun Nie and Zoran Obradovic and Toyotaro Suzumura and Rumi Ghosh and Raghunath Nambiar and Chonggang Wang and Hui Zang and Ricardo Baeza-Yates and Ricardo Baeza-Yates and Xiaohua Hu and Jeremy Kepner and Alfredo Cuzzocrea and Jian Tang and Masashi Toyoda",

booktitle = "IEEE International Conference",

}

Xu, L, Muharemagc, E & Apon, A 2017, ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform. in J-Y Nie, Z Obradovic, T Suzumura, R Ghosh, R Nambiar, C Wang, H Zang, R Baeza-Yates, R Baeza-Yates, X Hu, J Kepner, A Cuzzocrea, J Tang & M Toyoda (eds), IEEE International Conference. Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017, vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 2941-2950, 5th IEEE International Conference on Big Data, Big Data 2017, Boston, United States, 12/11/17. https://doi.org/10.1109/BigData.2017.8258263

ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform. / Xu, Lili; Muharemagc, Edin; Apon, Amy.
IEEE International Conference. ed. / Jian-Yun Nie; Zoran Obradovic; Toyotaro Suzumura; Rumi Ghosh; Raghunath Nambiar; Chonggang Wang; Hui Zang; Ricardo Baeza-Yates; Ricardo Baeza-Yates; Xiaohua Hu; Jeremy Kepner; Alfredo Cuzzocrea; Jian Tang; Masashi Toyoda. Institute of Electrical and Electronics Engineers Inc., 2017. p. 2941-2950 (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - ECL-watch: A big data application performance tuning tool in the HPCC systems platform

T2 - 5th IEEE International Conference on Big Data, Big Data 2017

AU - Xu, Lili

AU - Muharemagc, Edin

AU - Apon, Amy

PY - 2017

Y1 - 2017

N2 - The proliferation of Big Data processing environments such as Hadoop, Apache Spark, and HPCC Systems is driving the development of performance analysis tools in these distributed systems. The goal is to achieve high performance through the optimization of Big Data applications. However, tuning performance in a fine-grained manner is quite challenging due to the high complexity and massive size of the distributed systems. ECL-Watch is a data-flow based fine-grained comprehensive Big Data performance analysis tool utilizing the high level declarative dataflow programming language ECL in HPCC Systems. As a case study, we implement and optimize the Yinyang K-Means machine learning algorithm in ECL in HPCC Systems. The experimental results show that the performance of the native ECL version of the Yinyang K-Means algorithm increased significantly after tuning: from being about three times slower than the standard K-Means implementation in ECL, to become roughly 15% faster than standard K-Means.

AB - The proliferation of Big Data processing environments such as Hadoop, Apache Spark, and HPCC Systems is driving the development of performance analysis tools in these distributed systems. The goal is to achieve high performance through the optimization of Big Data applications. However, tuning performance in a fine-grained manner is quite challenging due to the high complexity and massive size of the distributed systems. ECL-Watch is a data-flow based fine-grained comprehensive Big Data performance analysis tool utilizing the high level declarative dataflow programming language ECL in HPCC Systems. As a case study, we implement and optimize the Yinyang K-Means machine learning algorithm in ECL in HPCC Systems. The experimental results show that the performance of the native ECL version of the Yinyang K-Means algorithm increased significantly after tuning: from being about three times slower than the standard K-Means implementation in ECL, to become roughly 15% faster than standard K-Means.

KW - Big Data

KW - Distributed Computing

KW - HPCC Systems

KW - Machine Learning

KW - Performance Analysis

KW - Tuning and Optimization

UR - https://www.scopus.com/pages/publications/85047834273

U2 - 10.1109/BigData.2017.8258263

DO - 10.1109/BigData.2017.8258263

M3 - Conference contribution

T3 - Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017

SP - 2941

EP - 2950

BT - IEEE International Conference

A2 - Nie, Jian-Yun

A2 - Obradovic, Zoran

A2 - Suzumura, Toyotaro

A2 - Ghosh, Rumi

A2 - Nambiar, Raghunath

A2 - Wang, Chonggang

A2 - Zang, Hui

A2 - Baeza-Yates, Ricardo

A2 - Hu, Xiaohua

A2 - Kepner, Jeremy

A2 - Cuzzocrea, Alfredo

A2 - Tang, Jian

A2 - Toyoda, Masashi

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 11 December 2017 through 14 December 2017

ER -

Xu L, Muharemagc E, Apon A. ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform. In Nie JY, Obradovic Z, Suzumura T, Ghosh R, Nambiar R, Wang C, Zang H, Baeza-Yates R, Baeza-Yates R, Hu X, Kepner J, Cuzzocrea A, Tang J, Toyoda M, editors, IEEE International Conference. Institute of Electrical and Electronics Engineers Inc. 2017. p. 2941-2950. (Proceedings - 2017 IEEE International Conference on Big Data, Big Data 2017). doi: 10.1109/BigData.2017.8258263

ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Teaching and research with LexisNexis HPCC systems with Clemson University

Cite this

ECL-watch: A big data application performance tuning tool in the HPCC systems platform: A big data application performance tuning tool in the HPCC systems platform

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

Teaching and research with LexisNexis HPCC systems with Clemson University

Cite this