Fast Erasure Coding for Data Storage - A Comprehensive Study of the Acceleration Techniques

Explore a comprehensive study on accelerating erasure coding techniques for data storage in this USENIX FAST '19 conference talk. Delve into various optimization methods including bitmatrix design, computation scheduling, XOR operation reduction, caching management, and vectorization. Learn how these techniques, previously proposed individually, can be combined for maximum efficiency. Discover a simple design approach that optimizes bitmatrices using a weighted cost function, and understand a straightforward erasure coding procedure that outperforms many existing techniques. Compare the performance against well-known codes like EVENODD, RDP, and STAR. Gain insights into why vectorizing XOR operations may be preferable to vectorizing finite field operations, considering both encoding throughput and CPU migration efforts. Examine individual acceleration techniques, optimization tiers, and their combinations through extensive testing results.