Building a Resilient and Adaptive Storage Environment
For its first decade of operation, DUG had been deploying and managing HDD-based storage to deliver the scale and cost economy that its seismic workloads required. During that time, DUG thoroughly optimized its applications to make use of the capabilities, and avoid the limits, of its Lustre HDD-based infrastructure. Here, DUG had to make many compromises. For example, when Lustre file system clients would hit peak throughput for a given workflow, other users sharing the same file system would suffer. From a resilience perspective, although DUG designed its software to protect against HDD failures, the need to swap out failed drives on a weekly basis was a constant thorn in DUG’s side.
Finally, while DUG’s applications were well optimized for Lustre and HDD storage, the new applications that DUG was evolving to support all handled storage input/output (I/O) differently. Storage versatility and multi-tenancy became vitally important to DUG; any new solution would need to support a broad set of requirements and to support them at exascale. DUG also needed storage that could handle the multiplicity of throughput requirements for different applications. DUG looked to solid state drive (SSD)-based storage to provide higher performance and reliability. However, moving to SSDs on Lustre would have been prohibitively expensive, and affordability was paramount for DUG. In order to build a resilient and adaptive storage environment that enabled expansion into new markets, DUG required a new approach to storage.
Universal Storage Unlocks Exascale NAS
DUG chose VAST Data Universal Storage to expand its business and support the needs of a wide diversity of new markets and customers. The Universal Storage offering combines the speed and scale of a parallel file system with a new level of flash affordability and multi-tenancy to deliver a complete technological leap forward for DUG. VAST Data’s disaggregated shared everything (DASE) architecture also provides consistent performance by isolating non-optimized I/O so as not to impact other tenants. With the DASE approach, VAST Data eliminates the concurrency challenges of parallel storage to deliver high performance for specific workloads that does not come at the expense of other workloads.
Beyond significantly improving the customer performance experience, VAST Data provides a combination of reliability, management, and support that is not otherwise found with legacy HPC storage technologies. VAST Data’s DASE architecture supplies exascale scalability, enabling DUG to grow to tens of petabytes of flash storage with no single points of failure in an architecture that can quickly recover from failure. The reliability of the DASE architecture comes “for free”: it is a direct result of VAST Data’s data-protection efficiency and the architecture’s statelessness. Beyond resilience, VAST Data Universal Storage also simplifies the deployment and management experience for DUG by providing an integrated scale-out appliance that consistently pushes out new features that are automatically applied while the system is online, so there’s no downtime for DUG.
Our complete HPCaaS ecosystem is a combination of innovative software and hardware solutions. After using the same storage solution for years, we required a new approach — one that was highly performant, resilient and adaptive. VAST Data’s promise of ‘no compromises’ has provided faster I/O, consistent performance and supreme data integrity for our DUG McCloud customers. It is changing our relationship with data, driving novel ways to mine, process and visualise big data. This allows us and our clients to gain new knowledge and insight in much shorter time frames.
Stuart MidgleyCIO, DUG
Storage as a strategic asset
DUG’s successful move into new markets was made possible by Universal Storage, powered by Intel technologies. Universal Storage provided DUG with the capacity, performance, and reliability to get rid of HDDs, move beyond complex HPC file-storage technology, and provide a leadership-class customer experience for customers within and beyond the oil and gas industry. An all-silicon storage offering provides the consistency and diversity of high performance that makes it possible for DUG to efficiently build out its multi-tenant cloud environment for its next wave of growth. The storage, reliability, and ease of management afforded by VAST Data has turned storage into a strategic asset for DUG, and has enabled it to better achieve its broader business goals.
With more than 17 years of experience and data centers in Perth, Houston, London, and Kuala Lumpur, DUG Technology is at the forefront of HPC. It combines innovative hardware and software solutions that enable clients to make use of large and complex datasets. DUG Technology’s industry experience and strong grounding in applied physics has equipped it to provide state-of-the-art HPCaaS delivered either direct-to-client or via its DUG McCloud platform.