product
Nov 29, 2023

Illuminating the Data Landscape with Uplink Anomaly Detection

images

Posted by

Peter Kogan, Senior Software Engineer

Uplink is VAST’s cloud-based service for telemetry and centralized fleet management, offering customers a single control plane to monitor their VAST clusters. This centralized view also provides the opportunity to gain deeper insight into IO patterns, user behavior, and the lifecycle of data. VAST Uplink now brings the power of AI to help customers optimize and secure their data with real-time anomaly detection. 

Understanding the rhythm of your data

A core benefit of the VAST Data Platform is the ability to consolidate all of your data into a single AI-ready platform. Increased performance and radically simple data management are the obvious benefits, but this centralization also delivers a holistic view of data and IO usage that was nearly impossible with legacy approaches. VAST Uplink uses this centralized vantage point combined with machine learning to discern the patterns that emerge, even from organizations with complex and varied applications. 

Detecting and responding to anomalies

By tracking the most important storage usage telemetry on a granular level and learning the behavior of users and data assets, VAST Uplink begins to establish a baseline. Knowing the usual behavior of different actors in the system, we continually monitor the new incoming telemetry to spot unusual patterns. VAST customers manage exabytes of data in dynamic environments, in many cases, anomalies may be caused by legitimate activity so it's vital to present events in a way that lets customers categorize for normalization or for further investigation.

Once Uplink notices a deviation for a data asset or user, for example, a database with a large number of read operations in a time window that would normally have low activity, an event is recorded and further information is collected to help administrators or application owners understand if additional investigation is needed.

images

Uplink dashboard with Anomaly Detection

From the technical point of view, we have created an automated machine learning operations (MLOps) infrastructure. Uplink ingests telemetry data from VAST clusters and allows the rapid deployment of custom models tailored to specific customer needs. By leveraging state-of-the-art gradient boosting algorithms along with additional high-end machine learning techniques, Uplink learns normal behavior patterns in customer data, forming the basis for detecting anomalies.

Catching bad actors in stealth mode

Anomaly detection is a critical tool for identifying and mitigating security events such as ransomware attacks or data breaches. Perpetrators of these crimes understand how security surveillance and threat detection systems work and seek to evade detection. Uplink provides an additional layer of observation that can help catch threats that would have eluded detection.

images

Uplink is able to identify potential security threats, here write patterns and changes in data reduction rates point to a potential ransomware attack.

The human element: critical context

While Uplink monitors infrastructure and I/O patterns, we recognize teams outside of IT often own the affected data and applications. They provide the context to categorize and respond to events. With this in mind, Uplink includes a user interface with Slack and email alerts configurable per data owner. Our goal is to supply actionable information requiring minimal manual effort where users receive the most essential data for efficient investigation.

images

Uplink sends notifications via Slack or email providing key information to users so they can act on the anomaly alert.

More from this topic

Learn what VAST can do for you
Sign up for our newsletter and learn more about VAST or request a demo and see for yourself.

By proceeding you agree to the VAST Data Privacy Policy, and you consent to receive marketing communications. *Required field.