Rousillon - Scraping Distributed Hierarchical Web Data

Explore a conference talk presenting Rousillon, a novel programming system for web scraping complex hierarchical data across multiple pages. Learn how this Programming by Demonstration (PBD) tool addresses limitations in existing solutions for collecting structured web data, as identified through interviews with social scientists. Discover the innovative "universal table" demonstration model and the underlying relation selection and generalization algorithms that enable users to efficiently scrape hierarchical datasets. Examine the results of a user study comparing Rousillon to traditional programming methods, showcasing an 8-fold increase in web scraper development speed. Gain insights into the potential impact of this tool on data collection for social science research and other fields requiring complex web data extraction.