Parallelizing R in hadoop (a work-in-progress study)

Yen Zhou Huang, Yu Ling Chen, Chia Ping Tsai, Hung-Chang Hsiao

研究成果: Conference contribution

1 引文 斯高帕斯(Scopus)

摘要

R is a popular programming language which is widely adopted by data scientists. However, typical R can only be executed in a single machine environment. Although R can be linked to Hadoop such as RHadoop, R users need to develop their R scripts based on the MapReduce framework. This de-mands highly skill of R programmers to parallelize their R pro-grams in terms of Map and Reduce jobs, killing the motivation of performing R computation in distributed environments out-pacing the single machine capacity. We present an implementa-tion for parallelizing R in Hadoop in this paper. Our objective is to allow R users to run their R scripts, which are developed in a single machine environment, in Hadoop without modification. While this research work is still ongoing, we report our prelim-inary experiences in this paper on how to hide the complexity of migrating and running such R scripts in Hadoop.

原文English
主出版物標題Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015
編輯Xingang Liu, Peicheng Wang, Yufeng Wang, Mianxiong Dong, Robert C. H. Hsu, Feng Xia, Yuhui Deng
發行者Institute of Electrical and Electronics Engineers Inc.
頁面1114-1116
頁數3
ISBN(電子)9781509018932
DOIs
出版狀態Published - 2015 一月 1
事件IEEE International Conference on Smart City, SmartCity 2015 - Chengdu, China
持續時間: 2015 十二月 192015 十二月 21

出版系列

名字Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015

Other

OtherIEEE International Conference on Smart City, SmartCity 2015
國家China
城市Chengdu
期間15-12-1915-12-21

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Media Technology
  • Computer Science Applications
  • Signal Processing
  • Computer Networks and Communications
  • Modelling and Simulation
  • Sociology and Political Science
  • Urban Studies

指紋 深入研究「Parallelizing R in hadoop (a work-in-progress study)」主題。共同形成了獨特的指紋。

  • 引用此

    Huang, Y. Z., Chen, Y. L., Tsai, C. P., & Hsiao, H-C. (2015). Parallelizing R in hadoop (a work-in-progress study). 於 X. Liu, P. Wang, Y. Wang, M. Dong, R. C. H. Hsu, F. Xia, & Y. Deng (編輯), Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015 (頁 1114-1116). [7463873] (Proceedings - 2015 IEEE International Conference on Smart City, SmartCity 2015, Held Jointly with 8th IEEE International Conference on Social Computing and Networking, SocialCom 2015, 5th IEEE International Conference on Sustainable Computing and Communications, SustainCom 2015, 2015 International Conference on Big Data Intelligence and Computing, DataCom 2015, 5th International Symposium on Cloud and Service Computing, SC2 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SmartCity.2015.218