Write For Us

Demo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM

E-Commerce Solutions SEO Solutions Marketing Solutions
39 Views

  • Thanks! Share it with your friends!

    You disliked this video. Thanks for the feedback!

ava
Published
Even the smallest of Large Language Models are compute intensive significantly affecting the cost of your Generative AI application. Your ability to increase the throughput and reduce latency can make or break many business cases. NVIDIA TensorRT-LLM is an open-source tool that allows you to considerably speed up execution of your models and in this talk we will demonstrate its application to Gemma.

Subscribe to Google for Developers → https://goo.gle/developers

#Gemma #GemmaDeveloperDay
Category
Project
Tags
Google, developers, pr_pr: Core DevRel DEI;
Sign in or sign up to post comments.
Be the first to comment

Up Next