There are currently more than 400 federal agencies engaged in digital initiatives that generate and collect data. That represents a huge amount of data to store, protect, access, and analyze, and it will only increase as time goes on.
Every federal agency I've worked with has similar challenges to those in the private sector: 1) data is growing exponentially; 2) there's demand for real-time analytics; and, 3) data protection is key.
But unlike the private sector, federal agencies often lack the agility, budget, and professional resources to do things better, faster, and cheaper.
The federal industry is ripe for new ways of storing data securely — a problem that's been around for 30+ years. That’s why my team and I have been laser-focused on enabling customers to ditch legacy-tiered data storage and upgrade their infrastructure to solve mission-critical and data-intensive challenges.
No one tiers data from one storage medium to another for any reason other than cost savings. Public cloud vendors offer dozens of tiers for data storage. Users make performance and accessibility tradeoffs every time they move data to a lower, less expensive tier.
But if you can solve the cost problem of single all-flash tier for all data - which VAST has - you can solve bigger problems and deliver more value to the organization.
Deep Learning at Air Force
The Air Force Research Lab’s (AFRL) work with packet capture (PCAP) data and sophisticated machine learning algorithms offers an intuitive example of how a single tier of all-flash infrastructure makes data more actionable and valuable.
As with other deep learning applications, PCAP is a random read-intensive workload. PCAP files are small, and because you can’t prefetch a random read, it would take months and even years to analyze PBs of PCAP on spinning disks. (And that’s after IT admins would have had to arrange the files chronologically to minimize the random read nature.)
The only practical solution for AFRL is to store massive amounts of PCAP data on flash. This allows the team to form a comprehensive view of organizational data and run trained algorithms that find anomalous security events out of the billions of captured log messages. This means that if a new zero-day attack comes out, AFRL can retroactively see if they've been hit by or have a vulnerability that needs to be fixed.
Backup Versus Restore
Data protection is important for any organization—but it's especially critical for government agencies: government data often contains sensitive information about citizens, national security, and more. If this type of data were to fall into the wrong hands, it could have disastrous consequences.
For the longest time, our industry focused on answering the question, "How fast can you back up your data?" Then, ransomware came along. Today my customers are worried about how fast they can restore the data; being shut down for weeks or months at a time is not an option.
In fact, the backup industry needs to be renamed the restore industry.
SLAs for low-cost storage cloud storage tiers can be a day or longer. If you have to restore something from the cloud it's going to take a while, and that’s not tenable for most federal agencies.
Here again we talk with customers about upgrading infrastructure to solve problems. The leading backup appliance was brought to market more than 20 years ago. Modern challenges like ransomware recovery require modern solutions. A flash-based backup appliance enables the ultimate priority for agencies: fast recovery.
The Path Forward
As the amount of data generated by the federal government continues to grow at an unprecedented rate, it's more important than ever that we find ways to improve the infrastructure that supports all of it.
VAST Federal is at the forefront of processing data assets in real-time by eliminating storage tiering and taking customers to a cost-effective all-flash end state that they’ve never been able to afford. All data is equally critical and our mission is to ensure that data age never defines the time to data access.
If you’d like to learn more visit vastfederal.com or drop me a line: randy at vastdata dot com.
