Delivering Domain-Specific LLMs with GPU Serving - Case of IFC MALENA

Explore how the International Finance Corporation (IFC) leverages AI and data to combat poverty and climate change in this 32-minute conference talk. Learn about IFC's successful scaling of the AI-powered MALENA platform using Lakehouse to accelerate custom large language model development. Discover the team's journey in enhancing real-time inferencing through Databricks' model serving for both internal IFC users and external B2B REST API users. Gain insights into their LLM Ops process and performance comparisons between Azure Functions and CPU model serving, particularly for fine-tuned models based on Google BERT. Understand the potential benefits of optimized GPU model serving for fine-tuned models trained on foundation models like Llama 2 or Mistral. Presented by Blaise Sandwidi, Lead Data Scientist and ESG Officer at IFC, and Jonathan Lorentz, Data Scientist at IFC, this talk offers valuable insights into domain-specific LLM deployment and GPU serving in the context of international development.