Exploiting and evaluating mapreduce for large-scale graph mining

Hung Che Lai, Chcng Te Li, Yi Chen Lo, Shou De Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Graph mining is a popular tcchniquc for discovering the hidden structures or important instances in a graph, but the computational efficiency is usually a cause for concern when dealing with large-scale graphs containing billions of entities. Cloud computing is widely regarded as a feasible solution to the problem. In this work, we present an open source graph mining library called the MapReduce Graph Mining Framework (MGMF) to be a robust and efficient MapReduce-based graph mining tool. We start from dividing graph mining algorithms into four categories and designing a MapReduce framew ork for algorithms in each category. The experimental results show that MGMF is 3 to 20 times more efficient than PEGASUS, a state-of- the-art library for graph mining on MapReduce. Moreover, it provides better coverage of different graph mining algorithms. We also validate our framework on billion-scaled networks to demonstrate that it is scalable to the number of machines. Furthermore, we test and compare the feasibility between single machine and the cloud computing technique. The effects of different file input formats for MapReduce arc investigated as well. Our implemented open-source library can be downloaded from http://mslab.csie.ntu.edu.tw/-noahsark/MGMF/.

Original languageEnglish
Title of host publicationProceedings of the 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2012
Pages434-441
Number of pages8
DOIs
Publication statusPublished - 2012
Event2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2012 - Istanbul, Turkey
Duration: 2012 Aug 262012 Aug 29

Publication series

NameProceedings of the 2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2012

Other

Other2012 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2012
CountryTurkey
CityIstanbul
Period12-08-2612-08-29

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Software

Fingerprint Dive into the research topics of 'Exploiting and evaluating mapreduce for large-scale graph mining'. Together they form a unique fingerprint.

Cite this