Back in what we of a certain age call the “good old days”—but what our younger coworkers may call the “dark ages”—data was created, processed, analyzed and acted upon all in a single data or HPC center. Today, that’s no longer true: a TV epic might be filmed in Vancouver, while editing, special effects, rendering, sound design, and 20 other functions happen in Hollywood, Seoul, or remotely in the cloud, before executives in New York review the dailies.
VAST’s Global Access lets you build a global namespace across multiple VAST clusters, making folders/buckets available across multiple locations with local I/O performance. This way, the crew in Hollywood can load the camera output into their local VAST cluster, automatically making that footage accessible in NY seconds later. Even better, users can write to those global folders as if they were local too. Unlike Google Drive, Dropbox, and most previous global namespace solutions, VAST’s global namespace is strictly consistent.
Why Strict Consistency Matters
Strict consistency ensures that multiple users, and/or servers, in multiple locations can read and write to the same folder—or even the same file—and every user always sees the latest version. When a user in London updates a file, any user accessing that file from any location will see those updates immediately.
Sync-and-share solutions (like Dropbox), most object stores, and most legacy global namespace solutions rely on an eventual consistency model, which is far less rigorous. For instance, a user in Chicago accessing a file that’s currently being updated in London may still read see a locally cached version because the local copy has not yet synchronized with the changes.
When users in two different locations write to the same file faster than the system can make that file consistent, the last writer wins. That means the version of the file with the later modification time (MTIME) becomes the current version. While the system may store both conflicting versions and generate an alert indicating a “collision” on a given file, users must manually merge the updates. Only then will both Joe in Chicago’s and Nigel in London’s updates be reflected in the final document.
Satellites Provide Local Cache
Before we dig into the details of how VAST Global Access delivers strict consistency, let’s take a step back to understand how the global namespace operates from the user’s and, of course, the system operator’s point of view. Global Access, like most VAST Data Platform services, is managed at the folder level, giving VAST customers full control over where their data is available.
For each global folder, one VAST cluster serves as that global folder’s Origin, holding a full copy of the folder and its contents. Other clusters in a VAST DataSpace can hold Satellites of the global folder, caching its contents and making them available at the Satellite’s locations with VAST’s customary all-flash performance.
It’s important to remember that Origin and Satellite are roles that apply to the relationship between VAST clusters for a specific global folder they share. These roles do not define the relationship between the VAST clusters overall. A single cluster can hold the Origin for some global folders and be a Satellite for others.
Figure 1: Global Access makes folders available across multiple locations.
Figure 1 shows how a production studio could use Global Access to simplify the process of creating a new show:
In LA, the cluster is the Origin of the /media global folder, where raw footage is saved directly from the cameras, and a Satellite of the /renders folder.
A VAST cluster in the cloud is a Satellite for the /media folder and the Origin for the /renders folder, where the cloud servers save processed work and intermediate outputs.
The VAST cluster in New York holds Satellites for both the /media and /renders folders, enabling the NY executives to view the cloud server’s nightly output and the raw footage.
With limited GPU resources on-premises, the studio’s job manager (in this hypothetical case, the VAST DataEngine) sends some overnight rendering and color correction jobs to run on a cloud provider. This ensures that completed shots are ready for executives in New York by 8 AM Eastern Time. The VAST cluster in the cloud will fetch the required footage from the /media folder on the LA cluster as the cloud severs run their assigned jobs. The cloud cluster uses its local SSDs to cache the data, so those servers have all-flash performance during processing, and write their results to the /renders folder.
Leases Maintain Strict Consistency
This brings us to how VAST Global Access maintains strict consistency across any number of VAST clusters that provide Global Access to a folder. Strict consistency ensures that any user or application accessing a VAST element as a file or object always sees the latest version, including all changes made to that element, even if updates occur from a user who’s connected to a different cluster while accessing the file.
The VAST global namespace maintains consistency through read and write leases. At any given time, only one cluster in the global namespace holds the write lease for an element. While clients can initiate writes from any cluster in the namespace, all writes are processed at the cluster holding the write lease. As of VAST 5.2, all write leases are held by the Origin cluster for the global folder. Satellites hold read leases on data in their local cache; a read lease on an element, or a byte-range within it, indicates the cached data is up to date.
When a client writes to an element at a Satellite cluster, the Satellite cluster synchronously proxies the write to the Origin cluster, which directly writes the update to its local, authoritative copy of the folder. The write also invalidates any read leases at Satellite clusters, informing the Satellites that any cached copy of the newly written data is invalid. When a client tries to read the new data from a Satellite, that Satellite will fetch the latest version of the data from the Origin.
While synchronously proxying writes ensures strict consistency, it does, like synchronous replication, add the roundtrip network latency between a Satellite and a folder’s Origin for remote writes. To mitigate this, VAST customers can deploy multiple global folders and, as in our example above, place the Origin of each folder near the clients that that will write in that folder.
In the future, write-leases will become portable, the write-lease for a newly created file on the cluster where the file is created. Once the new file is asynchronously replicated to the Origin, the write lease can migrate there.
Controlling Satellite Caches
When configuring a VAST cluster as a Satellite for a global folder, you can control how that Satellite caches and prefetches data from the Origin. As part of this configuration, you specify the amount of the Satellite’s local flash to allocate as a cache for the global folder. The Satellite cluster manages the specified capacity by evicting stale data and replacing it with new data when the cache is full.
By default, Satellites fetch data and metadata from the Origin only when clients request access. Satellite clusters will do some fetch-ahead on sequential reads, but this is limited to files, objects, and folders accessed by a client at the Satellite.
VAST 5.2 gives admins control over how Satellites prefetch data. Admins can configure each Satellite to prefetch data from a global folder as it changes, enabling users to access updated data at the Satellite without the delay of fetching it across the WAN. In our movie studio scenario, the New York cluster could be configured to prefetch all data from the /media folder. Since executives aren’t known for their patience, prefetching ensures that everything they might want is readily available, justifying the expense of additional network bandwidth.
Admins also have the option to configure a Satellite to prefetch only metadata changes to the global folder. This allows clients attached to the Satellite to see new files/objects appear in the global folder immediately, without the overhead of sending data that may never be accessed. Going back to our example, the cloud cluster could be set to prefetch metadata for the /media folder. This would accelerate the metadata lookups required for the cloud cluster to run its assigned jobs while avoiding the unnecessary bandwidth consumption and storage overhead of prefetching data most jobs will not access.
As you can see, Global Access, VAST’s global namespace, enables customers to make their data available across multiple VAST clusters, both on-premises and in the cloud, without read/write limitations. For more on the VAST global namespace, check out:
