Google just announced a major speed upgrade for AI and machine learning. If you use PyTorch on Google Cloud, your projects are about to run much faster.
Takeaways 🚀
✅Boost AI with Rapid Storage
✅Minimize GPU idle time efficiently
✅Achieve 23% faster AI training
✅Leverage Colossus for seamless integration
✅Simplify workflows using fsspec interface
Google connected Rapid Storage directly to PyTorch. This setup uses Google’s powerful Colossus architecture and the standard fsspec interface. Now, developers can keep their GPUs working at top speed and achieve unmatched efficiency.
Here is what you will learn in this post:
- Why data loading slows down AI training.
- How Rapid Bucket fixes the GPU bottleneck.
- Why the
fsspecinterface matters. - How to get started with zero code rewrites.
Fixing the GPU Data Bottleneck
AI models keep growing. Because of this, developers face a major hurdle: moving massive amounts of data. Loading data and saving checkpoints take time.
Old storage systems use standard REST APIs. These systems struggle to handle the fast speeds and quick response times that modern AI training needs. When the storage system moves too slowly, expensive GPUs sit idle. They wait for data instead of crunching numbers.
Google built a new solution to fix this exact problem.
Enter Rapid Bucket
Rapid Bucket is a high-performance object storage solution. Google built it for dedicated zonal buckets.
Instead of relying on slow, legacy REST APIs, Rapid Bucket uses persistent gRPC bidirectional streams. This brings the power of Colossus—the exact same storage system that runs YouTube and Google Search—directly to PyTorch users.
Key Performance Upgrades:
- Massive Throughput: Reach over 15 TiB/s in total throughput.
- Ultra-Low Latency: Experience less than 1ms wait times for random reads and appends.
- High Query Rates: Handle over 20 million queries per second (QPS).
How Fsspec Connects Everything
You might wonder how you can connect your code to this fast storage. The answer lies in fsspec.
Fsspec is a popular Python interface for file systems. The PyTorch ecosystem relies heavily on it for data prep, checkpointing, and inference. It lets developers swap out storage systems without rewriting their entire codebase.
Google integrated Rapid Storage with gcsfs, which is Google Cloud Storage’s version of fsspec. You can access incredible speed gains using a simple fsspec.open() command.
How the Tech Works Under the Hood
Google optimized every part of the data path to give you these speed boosts. Here is how they did it:
1. Persistent gRPC Connections
Standard connections open and close constantly. That wastes time. Google uses stateful gRPC streaming to keep the connection open. This reduces overhead and speeds up the entire data exchange process.
2. Direct Network Paths
Clients connect directly to the underlying Colossus files. This direct link cuts out unnecessary network hops. It dramatically lowers the time it takes to read and write data.
3. Zonal Co-location
Distance creates lag. Google places your storage in the exact same zone as your compute resources. This eliminates cross-zone delays completely.
4. Automatic Upgrades
Google updated the backend to handle the traffic upgrade from HTTP to BiDi-gRPC automatically. You do not have to configure anything manually. Your existing fsspec code works perfectly.
Real-World Speed Results
Google tested this new system to prove its power. They loaded a 451GB dataset (134 million rows) onto 16 GKE nodes. Each node ran eight A4 GPUs.
The test ran for 100 training steps and saved checkpoints every 25 steps. The results were clear. The Rapid Bucket system delivered a 23% performance gain over standard regional buckets.
Smaller benchmark tests showed even more dramatic improvements:
- Read throughput jumped by 4.8x.
- Write throughput improved by 2.8x.
Start Using Rapid Bucket Today
You do not need to rewrite your apps to get these speeds. Your existing code stays exactly the same. You only need to switch your target bucket to a Rapid Bucket.
To unlock this performance boost, make sure you use version 2026.3.0 (or newer) of gcsfs. Upgrade your storage today, eliminate data bottlenecks, and keep your GPUs running at full speed.
