Handshake's Approach to Content Tagging with vLLM and Anyscale

Explore Handshake's innovative approach to content tagging using vLLM and Anyscale in this insightful conference talk from Ray Summit 2024. Discover how Kyle Gallatin and Nabil Ahmed implemented an internally hosted LLM on Anyscale to enhance Handshake's job feed. Learn about the significant improvements in product performance achieved through LLM-based content tagging, and gain valuable insights into their methodology and workflow for deploying LLM services internally. Delve into Handshake's development of an in-house tool designed to abstract LLM use cases and backends, addressing their growing LLM requirements. Gain practical knowledge for integrating LLM technologies into product ecosystems, particularly for content optimization and management. This 24-minute presentation offers valuable lessons for companies looking to leverage LLM technologies effectively.