Explore common performance and architectural issues in distributed applications through this 37-minute conference talk from SREcon20 Americas. Learn to identify patterns such as N+1 Call & Query, Payload Flood, Too Granular, Tight Coupling, Bad Timeouts/Retries/Backoff, and Inefficient Dependencies. Discover how to automatically detect these patterns in CI/CD pipelines using the CNCF project Keptn and its SLI/SLO-based Quality Gate approach. Gain insights from real-world examples, including a distributed trace from StepStone, and understand the importance of answering questions like "Who is depending on me?" and "What is the risk of change?" in distributed architectures. Delve into topics such as cascading N+1 query patterns, payload flood issues, and refactoring strategies. Learn about Keptn's declarative workflows and event-triggered actions, and see a demonstration of automated SLI/SLO validation based on Dynatrace Dashboards. Explore the concept of Keptn-Closed-Loop Remediation and discover how to automate distributed problem detection and remediation in your own systems.
Automatically Detect the Top Performance & Scalability Issues in Distributed Architectures
Overview
Syllabus
Intro
Distributed Trace Example from StepStone (AWS Summit Berlin 2019)
In distributed architectures we need to answer: Who is depending on me? What is the risk of change?
N+1 Call Pattern across distributed "Product Service"
LANDBAY
Cascading N+1 Query Pattern: This is a single End-2-End Distributed Trace
Payload Flood: "Doc Creation" sequential across distributed services
Refactor: Only send relevant data to specialized services
Keptn from 10000ft: Declarative Workflows + Event-Triggered Actions
Root Cause: Lengthy manual approval in existing delivery pipelines
Inspired by Dynatrace's internal „Performance Signature as Code
SLI/SLO-based evaluation implementation in Keptn
Solution: Automate Approval through SLI/SLO-based Quality Gates
Demo: Automated SLI/SLO Validation based on Dynatrace Dashboards
Keptn-Closed-Loop Remediation with Keptn 0.7
Automate Distributed Problem Detection & Remediation
Taught by
USENIX
