Scality: Using Software-Defined Storage to Manage Petabyte-Scale Data

Scality develops software that enables enterprises to manage storage more efficiently and effectively at large scale (i.e., petabytes) than traditional storage solutions typically achieve. This high storage volume portion of the enterprise storage market is called the capacity-driven segment and reflects a major rethinking of the enterprise storage market.

In the past, a traditional (controller-based) enterprise storage system could typically address the majority — if not all — of what were well defined enterprise storage needs. Although still appropriate in many cases, this one-size-fits-all approach has been smashed to smithereens in a large number of other use cases. One example of this is the more or less bifurcation into latency-optimized and capacity-driven segments. Flash storage is the hot topic in the latency-optimized segment that seeks, among other things, to dissolve the I/O bottleneck that throttles overall system performance. Software-defined storage (SDS) has great importance in the capacity-driven segment where the scale of storage requirements can now easily range into the petabytes.

The data types at petabyte scale also tend to be different from traditional business process applications (such as online transaction processing) in areas like Web and cloud services, sensory data as part of the Internet of Things, mobile data, video and active archiving, such as for high performance computing. The management of this data through SDS is also different. Scality’s SDS approach illustrates how petabyte scale data is managed in a capacity-driven scenario.

Scality RING manages data at the petabyte scale

The Scality RING is a purely hardware vendor-agnostic software that runs on any X-86 based industry standard server. IT organizations can choose their own hardware or solutions from Scality partners, which offer a bundle of servers, storage, and services tailored to specific IT requirements.

But what does the RING do? The answer is that it provides for the data and storage management of humungous quantities of data leveraging an object storage architecture on the back end. Object storage is a hot topic in storage (due to the success of Amazon Web Services’ S3) as it is a very cost- and management-efficient way of storing large amounts of data.

But the Scality RING offers much more than that. Its unified storage management approach combines file applications with a scale-out file system that works along with object-based applications and OpenStack applications. This “front end” connects to the object storage backend via a completely distributed, completely parallel routing mechanism, with no bottlenecks or single points of failure. As a result, customers reap substantial benefits by using large numbers of servers to provide parallel, high performance access to large pools of storage, for legacy, as well as web, cloud, and mobile types of applications.

That is not a simple process. For example, data protection requirements are different in the petabyte world where standard RAID is not adequate to accommodate the multiple disk failures that inevitably occur. A loss of performance (or even the loss of data) cannot be tolerated. That is why the RING employs a technique called erasure coding. Not only that, but petabyte data cannot be stored in an all-storage-eggs-in-one-site data basket approach. Data protection has to be managed in a distributed manner and replication and geographical redundancy have to be included along with the erasure coding.

SDS and object-based storage are not only about the ability to provide robust and powerful data and storage management capabilities, but also about economics. Bottom line, this type of solution is less costly than traditional approaches. Scality asserts that the RING can deliver 50% to 70% savings over traditional solutions.

Scality expands its partnership with Hewlett Packard Enterprise

Two key code phrases to keep in mind here are software-defined storage and object storage, both areas where Scality is not the only game in town. Among the larger storage vendors, EMC has long offered Isilon for large capacity scale-out storage environments, as well as its Atmos object-based solutions. IBM has added Cleversafe to boost its object storage capabilities, along with its emphasis on software-defined storage in its Spectrum Storage Family. Among the smaller companies, Caringo offers SDS and object-based storage through the integration of its FileFly and Swarm software while Spectra Logic’s BlackPearl delivers a gateway to object-based deep storage, such as on its ArcticBlue disk storage.

Scality also plays with partners. Among the larger vendors with whom it has a strategic alliance, are Dell, Cisco, and Seagate. In addition, the company recently announced an expansion of its relationship with Hewlett Packard Enterprise (HPE), including tighter integration of marketing, engineering and sales resources. Hewlett Packard Ventures has also put equity money into Scality. Money talks, and this means that HPE has a serious relationship with Scality.

One result of this partnership is that HPE is emphasizing the integration of capacity-optimized computing with its Apollo 4000 series servers (which are designed for high performance computing) with the Scality RING to manage the accompanying high capacity storage.

Mesabi Musings

Organizations that have petabyte data storage requirements — and that includes more each day – are unlikely to turn to traditional enterprise storage solutions to manage these environments. Rather, they are more and more likely to turn to software-defined storage that includes the use of object-based storage. And that is where products such as the Scality RING come in. Managing petabyte-scale amounts of data requires a different mindset and way of doing the data and storage management functions, such as data protection, and that is the role of software-defined storage.

Vendors large and small are targeting the capacity-driven segment of the revamped enterprise storage space. The fact that Hewlett Packard Enterprise is throwing its weight (i.e., resources, including money) behind Scality qualifies not only as an endorsement of the company’s solutions, but the capacity-driven market itself.