Write For Us

Scaling Up (Part 2)

E-Commerce Solutions SEO Solutions Marketing Solutions
22 Views

  • Thanks! Share it with your friends!

    You disliked this video. Thanks for the feedback!

ava
Published
Welcome back to part two of our three part series on sharding and parallelism! Let’s explore how to scale your Flax NNX models using JAX's powerful distributed computing capabilities, specifically its SPMD paradigm. If you're coming from PyTorch and have started using JAX and Flax NNX, you know that modern models often outgrow single accelerators. We’ll discuss JAX's approach to parallelism, and how NNX integrates with it seamlessly. This episode focuses on the main workflow: how to integrate JAX's sharding primitives with Flax NNX, with a special focus on the critical sharded initialization pattern.

Resources:
Learn more → https://goo.gle/learning-jax

Subscribe to Google for Developers → https://goo.gle/developers

Speaker: Robert Crowe
Category
Project
Tags
Google, developers, pr_pr: AI DevRel (fka Core ML);
Sign in or sign up to post comments.
Be the first to comment

Up Next