Building a fault tolerant framework with deadline guarantee in big data stream computing environments

Dawei Sun, Guangyan Zhang, Chengwen Wu, Keqin Li, Weimin Zheng

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

Big data stream computing systems should work continuously to process streams of on-line data. Therefore, fault tolerance is one of the key metrics of quality of service in big data stream computing. In this paper, we propose a fault tolerant framework with deadline guarantee for stream computing called FTDG. First, FTDG identifies the critical path of a data stream graph at a given data stream throughput, and quantifies the system reliability of a data stream graph. Second, FTDG allocates tasks by the fault tolerance aware heuristic and critical path scheduling mechanism. Third, FTDG online optimizes the task scheduling by reallocating the critical vertices on the critical path of the data stream graph to lower the response time and reduce system fluctuations. Theoretical as well as experimental results demonstrate that the FTDG makes a desirable trade-off between high fault tolerance and low response time objectives in big data stream computing environments.

Original languageEnglish
Pages (from-to)4-23
Number of pages20
JournalJournal of Computer and System Sciences
Volume89
DOIs
Publication statusPublished - 2017 Nov

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Networks and Communications
  • Computational Theory and Mathematics
  • Applied Mathematics

Fingerprint Dive into the research topics of 'Building a fault tolerant framework with deadline guarantee in big data stream computing environments'. Together they form a unique fingerprint.

  • Cite this