Formal Specification and Testing for Reinforcement Learning

Explore a 29-minute video presentation from ICFP 2023 that introduces a formal specification and testing framework for reinforcement learning applications. Delve into the systematic approach proposed by researchers from IT University of Copenhagen and University of Oslo to address the exploratory nature of reinforcement learning development. Learn about their formal specification of reinforcement learning problems and algorithms, focusing on temporal difference methods and backup diagrams. Discover the test harness developed for temporal difference learning applications, including SARSA and Q-learning, rooted in functional programming methods. Examine the use of compositional interpreters and property-based testing to create a robust testing methodology. Gain insights into the effectiveness of their test suite through mutation testing results, demonstrating high mutant kill rates and the potential for write-once-use-everywhere tests applicable to various reinforcement learning problems.