Research and Implementation of a Highly Reliable Distributed Storage Scheme Based on Wirehair Code

doi:10.3969/j.issn.1671-1122.2018.02.003

Abstract

Abstract:

In the distributed storage, the data availability of each distributed storage node is very important. Therefore, the corresponding redundance measures must be taken to ensure the data availability. However, the redundance cost of the traditional redundance strategy based on duplication will greatly increase as data availability improves. Meanwhile, with the increasing of the number of distributed storage nodes, the possibility that some nodes suffer physical attack and theft also increases, which greatly increases the security threat to sensitive data. On the basis of deep research on Raptor code, this paper presents a distributed storage scheme based on Raptor code. And through a lot of experimental tests, a distributed storage system based on wirehair code (open source implementation of RaptorQ) is designed and implemented. The system can not only achieve higher data availability with smaller redundance, but also increase the security of the original data through the coding process.

Key words: distributed storage, wirehair code, redundance, Raptor code

CLC Number:

TP309

Kai DENG, Zhihong TIAN, Danyang MA. Research and Implementation of a Highly Reliable Distributed Storage Scheme Based on Wirehair Code[J]. Netinfo Security, 2018, 18(2): 20-26.

Figures/Tables 9

References 13

[1]	简玲. 基于Hadoop的PB级海量数据处理系统的设计与实现[J].信息网络安全,2013(11):87-89.
[2]	GAZELLE D, SNYDERS J.Reliability-based Code-search Algorithms for Maximum Likelihood Decoding of Block Codes[J].IEEE Transaction on Information Theory,1997,43(1):239-249.
[3]	WEATHERSPOON H, KUBIATOWICZ J.Erasure Coding vs. Replication: A Quantitative Comparison[M]//IPTPS. Peer-to-Peer Systems. London :Springer-Verlag ,2002:328-338.
[4]	孙程,谢军.一种基于纠删码的分布式存储容灾的设计与实现[J].中国集成电路,2009,18(10):38-42.
[5]	万武南. 分布式安全存储系统纠删码技术的研究[D].北京:中国科学院大学,2006.
[6]	刘凯. 基于RS码的分布式容灾系统的研究[D].哈尔滨:哈尔滨工程大学,2013.
[7]	张明. 基于网络编码的一种分布式存储系统应用设计[D].南京:南京理工大学,2013.
[8]	周福平,谢强,丁秋林. 分布式环境中基于改进RS编码的数据存储[J].计算机工程,2013,39(7):83-85.
[9]	刘莎,楚传仁.基于文件等级的Ceph数据冗余存储策略的研究[J].信息网络安全,2016(4):50-54.
[10]	SHOKROLLAHI A.Raptor Codes[J]. IEEE Transactions on Information Theory ,2006,52(6):2551-2567.
[11]	LUBY M, SHOKROLLAHI A, WATSON M, et al.Raptor Forward Error Correction Scheme for Object Delivery[J]. Letf Rmt Working Group Work in Progress, 1997, 11(3):82-89.
[12]	LUBY M.LT Codes[C]// IEEE. The 43rd Annual IEEE Symposium Foundations of Computer Science( FOCS ), November 16-19, 2002, Vancouver, BC, Canada.NJ:IEEE,2002:271.
[13]	谢垂益,鲁向前,卿斯汉.云存储系统的分区编码冗余方法研究[J].信息网络安全,2013(6):2-6.

[1]	Zhiyan ZHAO, Xiaomo JI. Research on the Intelligent Fusion Model of Network Security Situation Awareness [J]. Netinfo Security, 2020, 20(4): 87-93.
[2]	Min LIU, Shuhui CHEN. Research on VoLTE Traffic Based on Association Fusion [J]. Netinfo Security, 2020, 20(4): 81-86.
[3]	Lingyu BIAN, Linlin ZHANG, Kai ZHAO, Fei SHI. Ethereum Malicious Account Detection Method Based on LightGBM [J]. Netinfo Security, 2020, 20(4): 73-80.
[4]	Yifeng DU, Yuanbo GUO. A Dynamic Access Control Method for Fog Computing Based on Trust Value [J]. Netinfo Security, 2020, 20(4): 65-72.
[5]	Zhizhou FU, Liming WANG, Ding TANG, Shuguang ZHANG. HBase Secondary Ciphertext Indexing Method Based on Homomorphic Encryption [J]. Netinfo Security, 2020, 20(4): 55-64.
[6]	Rong WANG, Chunguang MA, Peng WU. An Intrusion Detection Method Based on Federated Learning and Convolutional Neural Network [J]. Netinfo Security, 2020, 20(4): 47-54.
[7]	Xiaoli DONG, Shuai SHANG, Jie CHEN. Impossible Differential Attacks on 9-Round Block Cipher Rijndael-192 [J]. Netinfo Security, 2020, 20(4): 40-46.
[8]	Chun GUO, Changqing CHEN, Guowei SHEN, Chaohui JIANG. A Ransomware Classification Method Based on Visualization [J]. Netinfo Security, 2020, 20(4): 31-39.
[9]	Lu CHEN, Yajie SUN, Liqiang ZHANG, Yun CHEN. A Scheme of Measurement for Terminal Equipment Based on DICE in IoT [J]. Netinfo Security, 2020, 20(4): 21-30.
[10]	Jinfang JIANG, Guangjie HAN. Survey of Trust Management Mechanism in Wireless Sensor Network [J]. Netinfo Security, 2020, 20(4): 12-20.
[11]	Jianwei LIU, Yiran HAN, Bin LIU, Beiyuan YU. Research on 5G Network Slicing Security Model [J]. Netinfo Security, 2020, 20(4): 1-11.
[12]	Peng LIU, Qian HE, Wangyang LIU, Xu CHENG. CP-ABE Scheme Supporting Attribute Revocation and Outsourcing Decryption [J]. Netinfo Security, 2020, 20(3): 90-97.
[13]	Yubo SONG, Ming FAN, Junjie YANG, Aiqun HU. Multipath Solution and Blocking Method of Network Attack Traffic Based on Topology Analysis [J]. Netinfo Security, 2020, 20(3): 9-17.
[14]	Tengfei WANG, Manchun CAI, Tianliang LU, Ting YUE. IPv6 Network Attack Source Tracing Method Based on iTrace_v6 [J]. Netinfo Security, 2020, 20(3): 83-89.
[15]	Yi ZHANG, Hongyan LIU, Hequn XIAN, Chengliang TIAN. A Cloud Storage Encrypted Data Deduplication Method Based on Authorization Records [J]. Netinfo Security, 2020, 20(3): 75-82.