Automating Technical Privacy Reviews Using LLMs

Explore how Uber and HERE Technologies leverage Large Language Models (LLMs) to automate and enhance technical privacy review processes in this 16-minute conference talk from PEPR '24. Discover the challenges of conducting privacy reviews in agile environments with frequent software releases and the impact of documentation quality on review effectiveness. Learn about the application of LLMs in assessing documentation completeness and classifying projects into risk categories. Understand the limitations of LLMs in performing full reviews without supervision and the strategies employed to improve efficiency, including policy writing, dataset validation, and prompt engineering with decision trees. Gain insights into balancing performance metrics like model accuracy and false negatives with cost considerations such as workload reduction and computational expenses. Examine the use of gates in GPT-4 prompts to achieve high accuracy rates and discuss future directions in automating technical privacy reviews.