Efficient reliability in volunteer storage systems with random linear coding

Abstract

Volunteer systems pose difficult challenges for data storage. Because of the extremely low reliability of volunteer nodes, these systems require so high redundancy that replication is infeasible. Erasure coding has been proposed to cope with this problem as it needs much less redundancy to achieve the same reliability. Its downside is that the reparation of the system creates high overhead, as fully decoding the original data is required to generate new coded data. Random linear coding has been proposed to be used as a data storage method, as it provides a better redundancy/reliability ratio, and less control overhead. We propose that it also helps in the reparation of the system, as decoding is not required; instead, coded data can be generated from already existing coded data. However, it may be possible that this iterative reparation leads to degradation of data over time; even more so, if sparse coding is used to increase compute efficiency. This paper examines the effects of random linear coding and the iterative reparation of the system. It shows the reliability that can be achieved with random linear coding in a highly volatile distributed system. We conclude that random linear coding can achieve high reliability even in highly volatile systems.