Experimenting in Sociotechnical Systems

Watch a conference talk exploring how experimentation works within complex sociotechnical systems, delivered by Jessica Kerr, Principal Developer Advocate at Honeycomb.io. Learn why traditional scientific methods and A/B testing may not be sufficient for understanding modern software systems and team dynamics. Discover alternative approaches to framing, selecting, and conducting experiments, along with new ways to evaluate outcomes beyond simple success/failure metrics. Gain insights into how careful observation and curiosity can help teams better understand their systems, even when objective measurements aren't possible. Drawing from 20 years of software development experience across multiple languages and platforms, Kerr shares practical wisdom about learning systems composed of both evolving software and enthusiastic people. The talk challenges conventional thinking about experimentation while providing frameworks for more effective system probing and organizational learning.