Parquet Courses and Certifications

Databricks

The Parquet Format and Performance Optimization Opportunities

Explore Parquet format intricacies, columnar storage basics, and optimization techniques for Spark workloads. Learn about encoding, compression, predicate pushdown, and Delta Lake to enhance analytical performance.

• YouTube
• 41 minutes
• On-Demand
• Free Video

The ASF

Bytedance's Parquet Format Optimization for Cost Reduction and Efficiency

Optimize Parquet file operations for big data efficiency. Learn binary copy techniques to enhance file overwriting, merge small files, and implement column-level TTL, boosting performance 10+ times.

• YouTube
• 23 minutes
• On-Demand
• Free Video

Python for Data Science Tips, Tricks, & Techniques

70 ratings at LinkedIn Learning

Explore a series of tips and tricks that you can put into practice to improve your skills in Python.

• LinkedIn Learning
• 1 hour 2 minutes
• On-Demand
• Free Trial Available

Databricks

The Apache Spark File Format Ecosystem - Optimizing Storage for Performance

Explore key file formats in Spark ecosystem: Parquet, ORC, Avro. Learn core concepts, optimization techniques, and how to select the right format for performant data processing and AI/ML workflows.

• YouTube
• 29 minutes
• On-Demand
• Free Video

Databricks

Building a Streaming Microservice Architecture with Apache Spark Structured Streaming

Explore building a scalable, fault-tolerant streaming microservice architecture using Apache Spark, Kafka, gRPC, and Delta Lake for real-time data processing and reliable insights.

• YouTube
• 39 minutes
• On-Demand
• Free Video

Databricks

Optimizing Spark SQL Jobs with Parallel and Asynchronous IO

Optimizing Spark SQL jobs through parallel and async IO techniques, including file-level and row group-level parallel read, asynchronous spill, and parquet column family, resulting in 5-30% performance improvement.

• YouTube
• 21 minutes
• On-Demand
• Free Video

Databricks

Degrading Performance - Understanding and Solving Small Files Syndrome

Learn how small files impact data pipeline performance, understand cloud storage and Apache Spark interactions, and discover DeltaLake solutions for improved efficiency in handling massive datasets.

• YouTube
• 27 minutes
• On-Demand
• Free Video

Databricks

Sound Data Engineering in Rust - From Bits to DataFrames

Explore Rust for efficient data engineering, from low-level bit manipulation to high-performance DataFrames using Apache Arrow. Learn optimization techniques and benchmarks for improved data processing.

• YouTube
• 35 minutes
• On-Demand
• Free Video

Databricks

Recent Parquet Improvements in Apache Spark - Vectorized Complex Types and Column Index Support

Explore recent improvements in Apache Spark's Parquet performance, including vectorized read support for complex types and column index support, leading to significant speed enhancements and efficient data processing.

• YouTube
• 37 minutes
• On-Demand
• Free Video

OLAP on Cassandra Data with Arrow, Flight SQL, ADBC, and DuckDB

Explore modern OLAP techniques for Cassandra data using Arrow, Flight SQL, ADBC, and DuckDB. Learn to convert sstable snapshots to Parquet, enable real-time analytics, and leverage Flight SQL for efficient querying.

• YouTube
• 30 minutes
• On-Demand
• Free Video

Analyze Semi-structured Data in Snowflake

This course introduces Snowflake’s capabilities for working with semi-structured data. These are critical for performing data analysis tasks when working with popular semi-structured formats like JSON.

• Pluralsight
• 20 minutes
• On-Demand
• Free Trial Available

Confluent

Maximizing Real-Time Data Processing with Apache Kafka and InfluxDB - A Comprehensive Guide

Learn to create powerful data pipelines by combining Apache Kafka and InfluxDB for real-time processing, analysis, and visualization of large-scale data, especially in IoT applications.

• YouTube
• 30 minutes
• On-Demand
• Free Video

The ASF

Caching Framework for Exabyte-Scale Data Lakes

Optimize exabyte-scale data lakes with open-source caching framework. Improve performance, reduce costs, and tackle challenges in complex data environments using Hadoop, Parquet, Hudi, and Alluxio.

• YouTube
• 40 minutes
• On-Demand
• Free Video

The ASF

Apache Arrow and Go: A Match Made in Data

Explore Apache Arrow's Go implementation, covering Arrow and Parquet libraries, Flight server/client, C Data API integration, and performance optimization techniques.

• YouTube
• 42 minutes
• On-Demand
• Free Video

Data Council

The Future Roadmap for the Composable Data Stack

Explore advancements in data processing with Wes McKinney. Learn about Parquet, Arrow, and interface libraries like Ibis and tidyverse. Gain insights into current project states and future innovations.

• YouTube
• 33 minutes
• On-Demand
• Free Video