Subscribe to Active Archive Alliance updates - blog posts, newsletters, and more!
Rich Gadomski's blog
Active archiving is getting a lot of attention these days as end users seek to relieve the tremendous pressure on primary storage from relentless growth of unstructured data. Like the pain from a toothache, they just want it to go away. The solution lies in extending the existing file system with active archive software across all storage pools and moving seldom accessed or totally dormant files from primary storage to a more cost-effective economy tier such as tape.
As summer approaches with temperatures already reaching into the 90s here in the Northeast, I wonder what the forecast holds for the summer of 2013. According to the Farmer’s Almanac, the summer of 2012 was exceptionally hot in many regions and 2013 is expected to be just as bad with hotter than expected temperatures. I guess we will have to deal with higher electric bills to run the AC in our homes; that will hurt! Not to mention our data centers, we will no doubt be straining the electrical grid and our IT budgets keeping all that spinning disk from overheating.
The recently announced availability of innovative LTO-6 Ultrium drives and media comes just in time, not only for the holiday wish lists of IT managers everywhere, but for the New Year when we can expect the continuing relentless growth of data in need of protection and preservation. While estimates of data growth among organizations vary from 30% to more than 50% annually, the fact that it is growing while IT budgets are not, makes for a timely introduction of this new generation of LTO.
Having attended HD World last year in New York and the more recent NAB show this past April in Las Vegas, I could sense a genuine concern among many attendees that I met regarding the challenge of storing their ever-increasing volumes of content. The M&E industry is in fact creating enormous amounts of new digital content thanks to HD and 3D movie production, new applications and techniques for special effects, etc. But in addition to all the new content being born digital, there are decades worth of a vastly greater amount of existing content, much of it sitting on shelves in the form of video tape. How to convert and digitize this content is becoming a priority for the industry in the interest of preserving these assets for future use and profit. While this undertaking has presented budgetary challenges, the good news is I have been hearing that several companies offer solutions for video tape to digital data file conversion, typically to LTO tape, and costs have been coming down over time.
It seems that the subject of archiving is getting a lot of press these days as enterprises of all shapes and sizes are facing a deluge of data and content which is being generated by new technologies and applications. This is not a bad thing per se as there may be value to be had from this data and content in the form of business analytics or monetization of the content at some point in the future. The problem comes in the cost management of this data/content effectively over time. With government regulations and legal compliance dictating long-term retention periods, and with the future value potential of certain data, much of the data has to be archived indefinitely.
In my previous blog posts, I talked about the advantages of data tape in terms of reliability and capacity and how tape plays a crucial role in supporting active archive systems. While these factors are key considerations in the merits of leveraging tape for tier 3 storage, cost effectiveness is a major factor that favors tape as well.
Leading analysts project that organizations will need to grow their data storage capacity dramatically in the coming years as a result of the explosion of unstructured file data, regulatory compliance and the need to keep data for longer periods in active archive mode. The numbers vary, but the consensus is around 50% data growth annually (there’s no recession in data creation!) Yet, IT budgets are barely increasing, so close attention is being paid to storage-related investments that can consume significant portions of CAPEX budgets. With OPEX budgets multiplying acquisition costs by several factors, it becomes clear that storing all data on disk drives is cost prohibitive from both an acquisition and an operations point of view.
In addition, the price of electricity to power and cool disk storage continues to climb, and some areas of the power grid are already over taxed, creating the problem of simply supplying the needed power to data centers.
TCO studies from leading analysts show tape systems cost less than disk. The Enterprise Strategy Group reported a 2-4X cost advantage in backup applications using LTO-5 compared to disk with de-duplication.
For long term archiving, The Clipper Group did a detailed TCO study and reported a 15X cost advantage for LTO-5 tape vs. disk in an archiving application over a 12-year period.
In yet another recent TCO study done by the Information Storage Industry Consortium, disk system acquisition prices turn out to be 9X more than the equivalent tape system for 500TB of storage over a five-year period.
When it comes to power consumption, tape is far greener than disk and this is where the real cost savings are to be found. The TCO study from The Clipper Group shows that disk consumes at least 238X more power than tape as data on tape consumes little or no energy, and tape does not require the significant energy associated with cooling spinning disks. In fact, Clipper showed that the cost of powering the disk solution over the 12-year period is the same as the cost of an entire tape solution including hardware, media and power!
Undoubtedly flash and disk technologies play a critical role in active archive systems for certain data applications, for example where rapid access time is important. But once again, studies show that anywhere from 60 to 90% of data is rarely accessed after 30 days. So it makes sense to move data from more expensive tiers of storage to the more cost-effective tape tier. And with active archive systems, the data remains accessible. Data storage that is efficient and always available – it’s the best of both worlds!
I often get asked the question, “what does the future hold for data tape in terms of capacity?” Usually followed up with “Are we on the threshold of the super paramagnetic limit? Will we need some new technology or breakthroughs to keep up with the explosion of data? Are currently published roadmaps really achievable? Will tape continue to play a critical role in data protection and emerging applications such as active archiving?”
To get started answering these questions, let’s look at the midrange tape market where LTO clearly dominates all other technologies with close to a 90% share. LTO Generation 4 is currently the most popular format with a native capacity of 800 GB. But the latest generation introduced in 2010, LTO-5, has been growing rapidly in popularity with a native capacity of 1.5 TB. LTO-5 also features a major breakthrough known as LTFS, or Linear Tape File System, which allows for dual partitioning of the tape where a portion of the tape is dedicated to a file index to enhance file management and facilitate data exchange and long-term data retention.
Next in line for LTO is Generation 6, with a native capacity slated for 3.2 TB, followed by LTO-7 at 6.4 TB and LTO-8 with a pretty impressive native capacity of 12.8 TB. We expect to see LTO-6 in 2012 or early 2013 with the next generations every two to three years thereafter. But back to the questions, is this truly achievable?
The answer is a resounding “yes!” and we can look to the enterprise tape market for a good indication of tape’s future direction.
In 2006, Fujifiim and IBM demonstrated a world record in data density on linear magnetic tape of 6.67 billion bits per square inch. This meant the ability to achieve multi-terabyte capacities of up to 8.0 TB on a single tape cartridge. The tape sample used was based on Fujifilm’s NANOCUBIC technology incorporating a new Barium Ferrite (BaFe) magnetic particle with the ability to resist outer magnetic interference and to maintain a strong magnetic signal even at greatly reduced dimensions compared to commonly used metal particles.
Once again in 2010, Fujifilm and IBM announced a new world record in data density on linear magnetic tape, achieving 29.5 billion bits per square inch using the new Barium Ferrite magnetic particle, this time with a perpendicular orientation, an even smaller particle size and a more advanced coating and dispersion technology. This translates into the possibility of developing a single tape cartridge capable of holding a massive 35 TB of native data. That’s 23 times the capacity of today’s LTO-5 and far exceeds the LTO roadmap requirements for LTO-8!
Earlier this year, the first product based on this technology came to market in the form of Oracle’s enterprise T10000-C drive and cartridge with a native capacity of 5.0 TB. This product represents a 3X capacity increase compared to the latest LTO-5 product and clearly speaks to the reality of increasing capacity on tape.
With the explosion of tier 3 unstructured file data, regulatory compliance and the need to keep data for longer periods in active archive mode, expect to see further advances in tape capacities as tape continues to play a vital role in cost effective, reliable, long-term mass storage.
I was recently having dinner with an IT executive for an internet service provider, when the conversation turned to the so-called “Tape Revival.” “Tape?” he said, “c’mon!” He had heard about tape’s killer archive application but he held onto the common misconception that tape is not as reliable as disk for archiving purposes. Because tape is playing an increasingly important role in both archive and backup, and especially in active archiving applications, I decided to present the case for tape.
To begin with, I explained that the archival shelf life of today’s modern tape formats such as LTO Ultrium is up to 30 years based on accelerated life testing. (Not really a stretch by any means, I still have some party mix audio tapes from school that still work 27 years later!) Disk on the other hand, is typically rated at about 3 to 5 years. And I admitted that even if tape drives aren’t around after 30 years, best practices call for a migration at least every 7 to 10 years to keep up with hardware/software changes and take advantage of faster and higher capacity generations of media.
“Yeah, okay, but how about error rates?” inquired my dining companion. I explained that the numbers here also look better for today’s tape than they do for disk. The generally published numbers for hard error rates are as follows:
LTO Tape: 1 bit in 1 x 10E17 bits
EnterpriseTape: 1 bit in 1 x 10E19 bits
Desktop SATA: 1 sector in: 1 x 10E14 bits
EnterpriseSATA: 1 sector in: 1 x 10E15 bits
EnterpriseFC/SAS: 1 sector in 1 x10E16 bits
I had to remind him that the error rates for disk are the number of bits read before the failure of a sector, whereas the error rates for tape are the number of bits read before the failure of a bit. As someone who deals with massive amounts of data, he wanted to know what this meant when archiving terabytes or petabytes of data. Simply put, I said, it means you have less risk associated with tape archives, especially if the applications involve encryption or data compression, as the loss of a single bit can render the data unrecoverable. Finally, to drive the point home, I told him that expert analysis based on these error rates in large archives, does show that LTO and Enterprise tape can achieve higher reliability ratings than disk.
I think my internet friend was satisfied with the explanation. He, like many other IT professionals, just wasn’t up to speed on the latest developments in tape today compared to the legacy formats of the 80’s and 90s, but these facts have been the case for more than five years now.
“Reliability is just part of the tape story however,” I continued. There are numerous other benefits associated with today’s tape including improved capacity and lower cost per/GB, improved transfer speeds, energy efficiency, and applications like active archiving that improve accessibility. So stay tuned to the active archive blog for more….. I promised to save that part of the conversation for the next meal as long as he was buying!