Crusoe is collaborating with VAST Data to offer Crusoe Cloud customers Shared Disks, a high performance storage product for AI. This collaboration between Crusoe and VAST delivers a petabyte scale file system capable of reads up to 200 MBps per TiB per node with hundreds of GPps in aggregate read and write bandwidth per cluster. With the VAST Data Platform, Crusoe customers have easy access to a NFS solution built for AI workloads at scale. For multi-GPU workloads like AI training, customers will be able to use Shared Disks to ensure they all have performant access to shared datasets.
Video
Overview
Background
“Crusoe chose the VAST Data Platform because of its exceptional ability to deliver the reliable file storage that our customers need without any depletion of performance as AI models are scaled,” said Patrick McGregor, Crusoe’s Chief Product Officer. “With client data housed full-time through VAST’s platform, customers will be empowered to build and innovate entirely through Crusoe Cloud. Because of their technology’s strong performance and outstanding team, VAST is a natural partner for Crusoe and our long-term data management goals. We’re excited to work with VAST as we advance Crusoe’s mission to align the future of computing with the future of the climate.”
“Powered by the VAST Data Platform, Crusoe’s Shared Disks offering delivers the modern AI cloud infrastructure today’s enterprises need to address the challenges of scaling data-intensive AI workloads,” said Chris Morgan, Vice President, Solutions at VAST Data. “Together, we’re delivering AI-driven solutions that provide the speed, security, and operational efficiency required for organizations looking to transform their data landscapes and accelerate their AI deployments to drive innovation and discovery.”
Outcome
Crusoe Shared Disks delivers the following features to Crusoe Cloud customers:
Disk operations: Customers have the ability to create, resize, mount, unmount and delete shared disks using the Crusoe Cloud API, CLI, UI interfaces or by using Crusoe’s Terraform provider.
Improved Performance: Crusoe Shared Disks can deliver up to 200 MBps of read throughput and 40MBps of write throughput per TiB of storage provisioned.
Secure Multi-Tenancy: Crusoe Cloud with Shared Disks are made available to a single project in an organization, with robust audit capabilities, and encryption at rest to deliver secure services to all customers at scale.
Fine-Grained Workload Isolation: Through granular quality-of-service policies that prevent multi-tenant I/O contention, Crusoe Cloud Shared Disks delivers the performance and data access customers need for AI workloads from a single cluster without being impacted by other tenants.
Reliability: Crusoe Cloud Shared Disks offer the same industry leading 99.5% SLA that is offered by Crusoe Cloud GPU VMs with data being protected both from hardware and component failures.
According to Moody’s Ratings, data center electricity consumption is forecast to grow by 23% annually between 2023 and 2028 with AI-specific energy usage expected to grow by 43% annually over the same period. In 2022, Crusoe launched Crusoe Cloud, a compute infrastructure platform for AI training, inference and HPC workloads. Crusoe’s cloud platform is powered by 100% clean, stranded or renewable energy, achieved by co-locating data centers with sources of clean energy to lower the cost and mitigate the environmental impact of computing. This empowers AI innovators to solve the world’s biggest challenges with climate-aligned computing solutions on Crusoe’s platform. Crusoe currently owns and operates a portfolio of more than 200 MW of data centers with plans to rapidly expand.
Crusoe chose the VAST Data Platform because of its exceptional ability to deliver the reliable file storage that our customers need without any depletion of performance as AI models are scaled. With client data housed full-time through VAST’s platform, customers will be empowered to build and innovate entirely through Crusoe Cloud. Because of their technology’s strong performance and outstanding team, VAST is a natural partner for Crusoe and our long-term data management goals. We’re excited to work with VAST as we advance Crusoe’s mission to align the future of computing with the future of the climate.
Patrick McGregor
Chief Product Officer, Crusoe