Light-Dedup - A Light-weight Inline Deduplication Framework for Non-Volatile Memory File Systems

Explore a 17-minute conference talk from USENIX ATC '23 that introduces Light-Dedup, a lightweight inline deduplication framework designed for non-volatile memory file systems. Learn about the innovative approach to reducing the cost of emerging NVM storage media through efficient data block elimination. Discover the Light-Redundant-Block-Identifier (LRBI) technique, which combines non-cryptographic hash with speculative-prefetch-based byte-by-byte content comparison to leverage NVM's memory interface for improved performance. Understand the benefits of the in-NVM Light-Meta-Table (LMT) for storing deduplication metadata and its region-based organization for reducing I/O amplification. Gain insights into experimental results showcasing Light-Dedup's superior I/O throughput compared to existing NVM deduplication file systems, and the significant improvements in metadata read/write amplification achieved through its novel design.