Business storage continues to be driven by two revolutions: one is storage systems–based and the other software-based. The former is focused on NVMe (nonvolatile memory express) networking technology that is accelerating the adoption of all-flash storage systems. In the latter case, software-driven innovation has become a driving force among virtually all major storage vendors.
One vendor that is making notable progress in both areas is IBM. On the systems/network side, i.e. NVMe-oF (NVM over Fabrics), IBM now supports Fibre Channel in addition to InfiniBand. Additionally, the company’s new Storwize V7000 Gen 3 has been architected for NVMe at the storage array level as well, joining the FlashSystem 9100 family (announced in July) with NVMe inside of the storage array. On the storage software side, IBM has just introduced Spectrum Discover as a new product in its IBM Spectrum Storage portfolio. Let’s examine these additions in a little more detail.
IBM continues to push the NVMe revolution
NVMe has two basic functions. NVMe-oF is the network side of the house and improves the performance of moving data between a host and a storage array. IBM initially enabled NVMe-oF for storage networks that use InfiniBand interconnects but now supports NVMe-oF with storage networks that use Fibre Channel (FC) to improve application performance and data access. This functionality runs in conjunction with the company’s Spectrum Virtualize through a straightforward, non-disruptive software upgrade. FC NVMe uses existing 16 Gb FC adapters and supports SVC (Model SV1), FlashSystem 9100, FlashSystem V9000 (Model AC3), and Storwize V7000F/V7000 Gen 2+ and Gen 3, and VersaStack that uses those storage arrays. This is likely to be important for users of those systems, as many of them likely have a FC SAN (storage area network).
IBM also continues to push NVMe at the storage device-level. Recall that the FlashSystem 9100, IBM’s enterprise-class entree in the virtual storage infrastructure space managed by Spectrum Virtualize was the first IBM storage system to offer NVMe at the device level. (See https://mesabigroup.com/ibm-flashsystem-9100-the-importance-of-nvme-based-storage-in-a-data-driven-multi-cloud-world/ for more detail.) Now, the new Storwize V7000 Gen 3–also managed by Spectrum Virtualize–offers the same NVMe end-to-end capability. That includes the use of the same version of IBM’s well-accepted FlashCore Modules that the FlashSystem 9100 pioneered.
Although the Storwize V7000 Gen 3 is technically not an all-flash solution (as users have the option to have some HDDs, such as for supporting non-performance-sensitive data), it can be configured as an all-flash system, and with the notable growth of all-flash arrays over the past few years, Mesabi Group expects a high percentage of them to be all-flash configurations. Since only flash (not hard disks) can benefit from NVMe technology at the device level, IT can maximize its use of a Storwize V7000 Gen 3 by having as much of its storage as feasible reside on flash storage modules (the new Storwize V7000 supports both IBM’s FlashCore technology as well as industry standard NVMe SSDs) instead of HDDs. If they do, Gen 3 offers up to a 2.7x throughput performance improvement over Gen 2+ as a key benefit.
IBM Spectrum Discover drives additional value from oceans of unstructured data
IT must get the most out of its investment in its physical architecture. For storage management purposes, that includes how storage arrays work in conjunction with the servers that demand services through a storage network. IBM’s storage management software, Storage Insights, is an AI-based tool that is offered through IBM Cloud to help users better manage their storage environments. For example, the latest version diagnoses storage network “gridlock” issues often referred to as “slow drain”. That gridlock occurs when a storage system attempts to send data to a server faster than the server can accept it; this is not a good thing! IBM storage technicians (who can monitor systems on behalf of clients who authorize it) are notified by Storage Insights of the problem as it is identified by AI technology. The technicians then review the situation and work with the client to resolve it.
Now, while Storage Insights deals with the physical side of storage as a storage management tool, recently announced IBM Spectrum Discover is an in-house data management software tool that targets the voluminous, and ever-rapidly growing amount of data, such as that created for Internet of Things (IoT), AI, and Big data analytic applications. Spectrum Discover works with file data managed by IBM Spectrum Scale or object data managed by IBM Cloud Object Storage, and enables users to get more out of their data for analytical, governance and storage investment purposes (IBM will also support Dell/EMC’s Isilon offerings in 2019).
How does it accomplish this? On the analytical side, getting to useful and actionable insights that would not be discovered otherwise within a data ocean of unstructured data rapidly is facilitated by such things as its ability to orchestrate machine learning and MapReduce processes. On the governance side, mitigating business risks by ensuring that data is compliant with governance policies and speeding up investigation into potentially fraudulent activities obviously may be of great value. On the investment side, the ability to facilitate the movement of “colder” (i.e., less frequently-accessed data suitable, say, for archiving) data to cheaper storage and to weed out and destroy unnecessary redundant data is financially advantageous.
The heart of Spectrum Discover’s power revolves around its metadata management and related processes. Any search and discover tool needs good data about data (i.e. metadata) to succeed. Spectrum Discover uses both automatically-generated system metadata at the time of data creation and custom metadata tagging that adds extra intelligence that is needed at analysis time. All that leads to automatic cataloging with the creation of an index where large quantities of data can be searched extremely rapidly for discovery purposes, thus reducing data scientist and storage admin preparation time and costs associated with that.
Although for different purposes (and not totally similar technologies) as an analogy, think of the search and discover capabilities of a public Internet browser for speed and flexibility for publicly-available data in contrast to the private data that Spectrum Discover deals with. Accompanying search and discover functions are a number of features and capabilities that greatly facilitate the use of the tool, including policy-driven workflows, a drill down dashboard, and an Action Agent that manages data movement and facilitates content inspection.
In essence, IBM Spectrum Discover is designed to significantly simplify and speed the data and storage processes required for analytics and AI processes. That should provide notable benefits for enterprises that aim to maximize the effectiveness and value of their advanced analytics investments.
Mesabi musings
You would think that storage innovations would show signs of slowing down after all these years, but the opposite seems to be true. In fact, IBM continues to be at the forefront of storage progress.
As illustrations of its continuing leadership, IBM has introduced the new NVMe-enabled Storwize V7000 Gen 3 on the systems side of storage, Spectrum Discover on the software side as a data management tool, and enhanced Spectrum Insights as a storage management tool.
Overall, IBM customers should be pleased with the progress IBM is making with NVMe technology, a fundamental storage systems underpinning technology on the hardware side, while Storage Discover, on the software side, continues the push toward extracting additional value from up to oceans of unstructured data.