By Kevin Davies
January 22, 2010 | With one eye firmly on the soaring data demands in life sciences and next-generation sequencing, San Jose-based Quantum Corp. has released a version 4.0 of its scalable StorNext data management software, which it believes offers a solution for the storage needs imposed by the remarkable growth in unstructured data and rich file formats.
Quantum provides data storage solutions for a variety of industries, including digital media, oil and gas, financial and the sciences. The major focus is on data protection, archiving, backup and recovery, using virtual tape libraries and other products to share high-speed content for files on disk, with petabyte volumes par for the course.
The StorNext product line, which has been around for about a dozen years, started as a high-speed, shared SAN file system and as the product evolved, transparent data movement for tiered storage and archiving was added with the Storage Manager option.
According to Shawn Klein, director of software partner development, Quantum views the life sciences is an “adjacent market” but Quantum hasn’t needed to make many changes to the product. The move beyond media and entertainment is relatively new, says Klein.
Quantum’s strength is in managing data storage, from disc-based and tape libraries used for data backup to tape libraries used for archiving. Klein says the “secret sauce” is that the software works across multiple platforms, compatible with most types of disk drive, tape libraries, and operating system.
What Happens Next
The StorNext file system is a heterogeneous, high-performance file system for large volumes of data. It makes the SAN storage look like a local hard drive. “You just point the application in Windows to the D drive and it looks like a local hard drive to that box,” says Klein. “We don’t care whether it’s a satellite image or MRI. We know how to handle images real well.”
The storage engine moves data automatically. Users can set whatever data policy they like: for example, move any file in a certain directory into a second tier, or move to tape if unopened in 10 days or 30 days. “After you render an image for a couple of weeks, you can move it to a lower tier of storage. Because that file isn’t being accessed with the same frequency, you can save cost and better manage your environment by moving to a lower tier of storage,” says StorNext product marketing manager Chris Duffy. “And yet, the applications and users don’t know the files have been moved. It looks like the file is still in your D drive.”
This provides flexibility for long-term archiving, providing a large number of distributed targets. Klein says that whereas competitors have to introduce a third-party data mover, StorNext has it embedded. “There’s no third-party application to plug in to lift the information and place it somewhere else,” says Klein.
Among the new features of the version 4.0 is support for file-based replication from one environment to another. The data deduplication feature is fully integrated resides in the file system. As Klein explains, “you can drop a file into a “dedupe” directory, and once it is there, the engine takes over and deduplicates the data.”
Klein calls the data dedupe feature nothing short of revolutionary, like “compression on steroids.” Says Klein: “Dedupe has been hot and putting it into a primary file system is a really big deal. Replication has been around for a long time, and we’ve been using third parties for that functionality.” Now, the policy engine for replication drives the dedupe feature. There’s a lot of cool use cases for that combined functionality.”
The new version of StorNext also has a web services GUI. “Managing petabytes of data isn’t easy, so the GUI is simplified, providing monitoring and web services XML available for application partners,” says Duffy.
So far, the most visible life science case study is at Baylor College of Medicine (BCM). Perhaps not coincidentally, Baylor is based in Houston, where Quantum has a strong presence given its relationship with the oil and gas industry. Quantum won the Baylor deal last summer after a relatively short sales cycle. Quantum partnered with storage company DASDI, which originally identified the opportunity for StorNext. Quantum has also done deals in Singapore and Austria.
“With such high volumes of data generated daily from DNA sequencing, and the need to access hundreds of terabytes of data at any given time, StorNext offers the scalability and support we need,” says Geraint Morgan, director of Information Systems at the BCM Human Genome Sequencing Center. Morgan expects the BCM genome center’s storage requirements to push past two PB.
“We can scale performance and capacity independently. If somebody needs to add more sequencers but their capacity remains unchanged, we can help them increase performance without adding mandatory capacity,” says Klein. “Similarly, if you just need another 1 PB data, you don’t have to buy additional appliances, you can just grow capacity. We’re vendor agnostic when it comes to disk. If the customer loves HP today but wants to put in EMC or IBM, we can put those all together and make them look like one big hard drive and virtualize that. So we really help the customers keep costs down.”