The Abuse Uncertainty Principle, and Other Lessons Learned from Measuring Abuse on the Internet - David Freeman - USENIX Enigma Conference - 2020
USENIX Enigma Conference via YouTube
Overview
Syllabus
Intro
Sometimes I wish I worked on ad ranking...
Abusive content on Facebook
Measuring abuse is hard!
Measuring abuse: The Dark Ages
Measuring Abuse: The Renaissance
Benefits of Measurement (in addition to a quieter office)
How do we get there?
Count what you blocked These things have to be bad, right?
Have users do the labeling Crowdsource the work via reporting/appeals flows.
Human labeling Get experts to decide for you.
Deployment at Facebook Used for internal direction and/or external reporting of
Deployment at Facebook Used for internal direction on
Comparison of approaches
Example: Rate Limiting
"Proof" of Goodhart's Law
Corollary: "Abuse Uncertainty Principle"
Living with the Uncertainty Principle For spam detection at Facebook, we split signals into two classes
Other coping mechanisms
Challenge: Build an Abuse Adapter!
Open questions
Thank you!
Taught by
USENIX Enigma Conference