Presenters

Source

Level Up Your Video Quality: Scaling AI-Powered Super Resolution at Meta 🚀

Video is everywhere. It dominates internet traffic, drives user engagement, and fuels business growth. At Meta, we process a staggering amount of video daily – over 1 billion uploads and a trillion view requests! As video becomes increasingly vital, ensuring optimal quality and user experience is paramount. Today, we’re diving into how Meta is tackling this challenge by deploying Video Super Resolution (VSR) at scale.

Join us as Ryan Lei, Video Tech Lead and Software Engineer in Meta’s Media Foundation team, shares the journey, learnings, and future directions of this exciting initiative.

Why Video Super Resolution? 🎯

The need for VSR stems from three primary sources of low-quality video:

  • User-Generated Content: Videos created with lower-quality cameras, poor lighting, or heavy compression during upload.
  • Cross-Platform Content: Videos downloaded from other platforms and uploaded to Meta apps.
  • Legacy Inventory: Older videos originally created at lower resolutions and qualities.

VSR, powered by sophisticated machine learning models, transforms these blurry and grainy videos into sharper, clearer visuals. Think of it as magic, breathing new life into older content and making it shine on modern, high-resolution displays.

Scaling VSR: The Meta Approach 🛠️

Meta’s approach to deploying VSR is multifaceted, focusing on quality, efficiency, and scalability. Here’s a breakdown:

  • Server-Side VSR: Applying VSR to lower-resolution videos before encoding them for delivery. This creates a high-quality source for ABR (Adaptive Bitrate) encoding.
  • Client-Side VSR: Enhancing the quality of videos during playback, especially valuable when bandwidth is limited.
  • Lightweight Solutions for Mobile: Developing VSR models optimized for mobile devices, balancing quality with battery life.
  • End-to-End Quality Logging: Monitoring the quality improvements from VSR deployments.
  • A Portfolio of Solutions: Supporting a range of VSR models to cater to diverse needs.

Intel Partnership: Unlocking CPU Power 🌐

A key breakthrough came with a partnership with Intel. By leveraging Intel’s RVSR SDK and openVINO 2K, Meta can now deploy advanced VSR models on standard x86 CPU infrastructure. This is a game-changer because it:

  • Reduces GPU Dependency: Decreases reliance on scarce and expensive GPU resources.
  • Increases Operational Scalability: Enables more flexible and cost-effective deployments.
  • Hides Complexity: A middleware stack simplifies integration with existing processes.

Real-World Impact: Ads, Geni Videos, and Restyle ✨

The impact of VSR is already being felt:

  • Improved Ad Quality: Lower-resolution ads are now significantly sharper and more appealing.
  • Meta Restyle Feature: The cutting-edge Restyle feature (available in Meta AI and Instagram editor) uses VSR to transform photos and videos, allowing users to change backgrounds, lighting, and artistic styles.
  • Enhanced Video Calling: Future plans include exploring VSR for video calling use cases.

Key Learnings & Future Directions 💾

Scaling VSR isn’t without its challenges. Here’s what Meta has learned:

  • Subjective Evaluation is Key: Reliable quality metrics are crucial for benchmarking VSR models and identifying potential artifacts. Meta utilizes a framework for large-scale, automated subjective evaluation.
  • UQ Correlation: Data UQ score correlates well with human subject rating.
  • Targeted Application: VSR is most effective on videos that are already of medium to high quality. Applying it to very low-quality videos yields minimal improvement.
  • Media Understanding: Combining VSR with advanced media understanding will enable safer and more intelligent applications.

Want to learn more?

Meta’s journey with VSR demonstrates the power of AI to elevate video quality and user experience. As technology evolves, expect even more innovative solutions to emerge, further transforming how we create, consume, and share video content.

Appendix