That’s where the first challenge comes into play: because the datasets involved are so gargantuan, they can be time-consuming simply to load up and access. This is where VAST Data helps, because it enables these large datasets to be loaded and accessed in a reasonable amount of time. The time involved in getting to work obviously impacts the amount of work that gets done, so the VAST Data Platform helps improve productivity and capability. Indeed, speed is key to keeping up with fast-changing, always-moving markets.

The second challenge is related to how the datasets are handled once they’re loaded and are ready to process: as only then can they be ingested for analysis and used to drive models or simulations. Man Group’s previous database solution limited the size of datasets that it could ingest. Given that many important datasets are expected to break the 10PB barrier soon, this limitation was problematic. So flexible capacity that can accommodate datasets of 5PB or even 10PB is increasingly essential. 

The VAST Data Platform easily supports massive datasets, and Man Group has also developed ArcticDB, a high-performance Python-native database built in order to respond to the ever-increasing amount of data and complexity of front-office research at the firm. This is a challenge faced by many large buy-side and sell-side institutions. Using ArcticDB, Man Group’s investment professionals and technologists can better power robust, near-real-time automated trading. It also enables point-in-time analysis of research datasets and provides functionality for signal backtesting.

VAST and ArcticDB work in a complementary fashion. What the combination brings gets at the third challenge involved: while ArcticDB provides a set of data abstraction and definition tools that exercises capabilities across multiple file systems and object stores, VAST provides the underlying scale, speed and efficiency necessary to support the levels of performance for the huge datasets involved.