Optimizing HDFS Performance: Balanced Utilization of Hardware Accelerators and ARM64 Multi-Implementation - Transparent Encryption

Explore strategies for optimizing HDFS performance through balanced utilization of hardware accelerators and ARM64 multi-core processors in this 20-minute conference talk. Delve into the challenges of Hadoop/HDFS Transparent Encryption and learn about innovative approaches to improve encryption performance. Discover how to leverage both dedicated hardware accelerators and CPU proprietary instructions like Intel AES-NI and ARM64 Crypto-Extension to enhance encryption and decryption bandwidth. Examine a feature based on OpenSSL 3.0 that enables dynamic resource allocation between multiple algorithm implementations, potentially increasing resource utilization efficiency and supporting more users on the same hardware. Gain insights into the performance improvements achieved in HDFS Transparent Encryption using these methods, presented by Guodong Xu, Senior Tech Lead at Linaro Ltd., with extensive experience in ARM optimization, big data, and kernel development.