How to Kill Your PySpark Performance - Common Anti-patterns and Solutions

Learn about PySpark's performance optimization in this 20-minute conference talk from PyCon Israel that explores both the powerful capabilities and common pitfalls of Spark's Python interface. Discover how certain anti-patterns can severely impact PySpark applications' performance, leading to increased costs and reduced efficiency. Master native PySpark tools and patterns that help avoid these performance bottlenecks, enabling you to harness the full potential of this robust data processing framework.