A DSM-based fragmented data sharing framework for grids

Po Cheng Chen, Jyh Biau Chang, Ce Kuen Shieh, Chia Han Lin, Yi Chang Zhuang

研究成果: Article同行評審


Sharing scientific and data capture files of gigabyte and terabyte size in conventional data grid systems is inefficient because conventional approaches copy the entire shared file to a user's local storage even when only a tiny file fragment is required. Such transfer schemes consume unnecessary data transmission time and local storage space, with the additional problem of maintaining replica synchronization. Traditionally, replica consistency treats shared files as read-only, consequently sacrificing guaranteed replica consistency. This paper presents a DSM-based fragmented data sharing framework called "Spigot" which transfers only the necessary fragments of large files on user demand, thereby reducing data transmission time, wasted network bandwidth and required storage space. Data waiting time is further reduced by overlapping data transmission and data analysis. The DSM concept maintains replica synchronization. Real experiments show reduced turnaround time in data-intensive applications, particularly when fragment size is low and analysis time and network latency are high.

頁(從 - 到)668-677
期刊Future Generation Computer Systems
出版狀態Published - 2010 4月

All Science Journal Classification (ASJC) codes

  • 軟體
  • 硬體和架構
  • 電腦網路與通信


深入研究「A DSM-based fragmented data sharing framework for grids」主題。共同形成了獨特的指紋。