A DSM-based fragmented data sharing framework for grids

Po Cheng Chen, Jyh Biau Chang, Ce Kuen Shieh, Chia Han Lin, Yi Chang Zhuang

Research output: Contribution to journalArticle

Abstract

Sharing scientific and data capture files of gigabyte and terabyte size in conventional data grid systems is inefficient because conventional approaches copy the entire shared file to a user's local storage even when only a tiny file fragment is required. Such transfer schemes consume unnecessary data transmission time and local storage space, with the additional problem of maintaining replica synchronization. Traditionally, replica consistency treats shared files as read-only, consequently sacrificing guaranteed replica consistency. This paper presents a DSM-based fragmented data sharing framework called "Spigot" which transfers only the necessary fragments of large files on user demand, thereby reducing data transmission time, wasted network bandwidth and required storage space. Data waiting time is further reduced by overlapping data transmission and data analysis. The DSM concept maintains replica synchronization. Real experiments show reduced turnaround time in data-intensive applications, particularly when fragment size is low and analysis time and network latency are high.

Original languageEnglish
Pages (from-to)668-677
Number of pages10
JournalFuture Generation Computer Systems
Volume26
Issue number4
DOIs
Publication statusPublished - 2010 Apr 1

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'A DSM-based fragmented data sharing framework for grids'. Together they form a unique fingerprint.

  • Cite this