Go Profiling and Observability from Scratch

Overview

Explore profiling and observability techniques for Go applications in this comprehensive conference talk from GopherCon 2021. Learn how to reduce costs, improve latency, and debug issues like memory leaks and performance regressions. Gain insights into the Go runtime's unique features and understand the built-in CPU, Memory, Block, Mutex, and Goroutine Profilers, as well as the Runtime Tracer. Discover advanced tools such as eBPF, Linux Perf, and Delve for optimizing and debugging Go applications. Follow along as the speaker covers topics including scheduling, memory management, profiling techniques, tracing, overhead analysis, and observability tools, providing a solid foundation for improving Go application performance and troubleshooting skills.

Syllabus

Intro
Target audience
Agenda
Scheduling & Memory Management
Memory Management: Stack
Memory Management: Heap
CPU Profiler: SIGPROF for every 10ms of CPU Time
CPU Profiler: Add Stacktrace to Profile
CPU Profiler: Waiting for Go 1.18
Block Profiler: Channels
Block Profiler: Mutexes
Mutex Profiler
Block vs Mutex
Quick Note on Time: 1 Request - 3 Goroutines
Memory Profiling
Goroutine Profiling
Tracing Manually
Runtime Tracer
Profiling and Tracing: A Faustian Bargain?
Profiling and Tracing Overhead Analysis
Overhead Analysis: SQL Workload
Overhead Analysis: HTTP Workload
Overhead Analysis: JSON Workload
JsoN Workload (Concurrency = 8)
Overhead Analysis: Chan Workload
Overhead Analysis: Error Sources
Metrics
Linux perf
bpftrace
Scheduling / Execution Observability
Memory Management Observability
Recap