Setting New Standards in Environmental Research Infrastructure

Selection of the VAST Data Platform by NIWA will streamline management of critical climate data, enabling breakthrough scientific insights.

Industry Government

Use Case HPC & Data Protection

Overview

The National Institute of Water and Atmospheric Research (NIWA), New Zealand’s leading Crown Research Institute for environmental science, has selected the VAST Data Platform to update and future-proof its data infrastructure. The platform will enable NIWA to synchronize its production workloads and research across primary and secondary sites, optimize storage efficiency, and prepare for next-generation GPU-based scientific workloads.

NIWA’s groundbreaking research in atmospheric science, oceanography, and climate modeling relies on over 20 petabytes of historical weather data. With the advent of AI and machine learning technologies, NIWA required a solution to unify its operations, ensure seamless scalability, and support evolving demands for high-performance computing.

Background

The VAST Data Platform’s unique active-active clustering capabilities, enabled through a global namespace, allow seamless synchronization and utilization of data across multiple sites. This innovation reduces operational complexity compared to legacy systems, making it faster and easier to implement while offering advanced functionality like write-read access across both sites.

“VAST Data stood out in a crowded field for its ability to unify our sites and prepare us for the future of AI-driven research,” said Jeff Zais, HPC Senior Science Advisor and Platform Architect at NIWA. “The platform not only supports our current CPU-based workloads but also gives us the flexibility and confidence to integrate GPU capabilities as we transition to new weather and climate modeling codes. VAST’s synchronization features, density advantages, and vision for the future were critical in our decision to partner with them.”

In addition to addressing current CPU-based workloads, the VAST platform simplifies storage management with its integrated compression, deduplication, and multi-tenancy capabilities. These features ensure optimal storage space utilization while reducing daily management burdens, allowing NIWA to focus on advancing its research priorities.

VAST Data’s Disaggregated Shared-Everything (DASE) architecture played a pivotal role in NIWA’s selection, addressing challenges with legacy systems while reducing costs and operational complexity. The platform’s high-density design enabled NIWA to shrink its physical infrastructure footprint, cutting costs over a six-year period while improving data accessibility and reliability.

Outcome

“The VAST Data Platform will provide the backbone for NIWA’s advanced technology and intelligent data capture, empowering not only our storage and archival needs but also being able to support other research and agencies across New Zealand,” said Warrick Johnston, General Manager Technology and Innovation at NIWA.

“NIWA is a pioneer in climate and environmental research, and it’s critical that they have the tools to access and analyze their immense datasets in real time,” said Sunil Chavan, Senior Vice President, APAC at VAST Data. “The VAST Data Platform provides the scalability, performance, and integration needed to address the complexities of modern scientific workloads. We’re proud to support NIWA as they continue to innovate and lead on a global scale.”

With the ability to now fully utilize resources at its secondary site, NIWA can run more climate models simultaneously, improve disaster preparedness, and orchestrate more use cases with fewer resources. This ensures faster, more detailed insights into climate and environmental patterns, driving impactful science that benefits both New Zealand and global efforts.

The VAST Data Platform will form the backbone of NIWA’s data infrastructure as it embarks on projects involving GPU-ready applications, next-generation weather modeling, and collaborative global initiatives. The deployment process is underway, with full production readiness anticipated by May 2025.

This deployment positions NIWA as a leader in using AI-driven data platforms for environmental science. By adopting VAST’s cutting-edge infrastructure, NIWA not only sets a benchmark for other institutions but also highlights the transformative role of modern data solutions in tackling global challenges.

VAST Data stood out in a crowded field for its ability to unify our sites and prepare us for the future of AI-driven research. The platform not only supports our current CPU-based workloads but also gives us the flexibility and confidence to integrate GPU capabilities as we transition to new weather and climate modeling codes. VAST’s synchronization features, density advantages, and vision for the future were critical in our decision to partner with them.

Jeff Zais

HPC Senior Science Advisor and Platform Architect, NIWA