Preserve Data for Decades with an Active Archive

QStar Technologies, along with many of the Active Archive Alliance members were at NAB (National Association of Broadcasters) Show in Las Vegas a few weeks ago. 

I had the pleasure of speaking with a number of users in the Media and Entertainment world, some small and some very large. Almost universally they expressed the need to preserve their digital content for years and decades, yet all are looking for a “migration free” archive, which does not exist. All technology has a useful life, after which it becomes prohibitively expensive to continue operating. The objective is to migrate to a new technology before this occurs, so designing in migration is imperative when creating an archive.

Increasingly, object-based disk systems are being considered within the Media and Entertainment world, mainly because of the “self-managing, self-healing” nature of this form of storage technology. Object-based storage monitors the health of the archived data and if the data is corrupted or is lost due to media or system failure, it can recover the data from other good copies. Using this feature, disk storage can be removed and replaced with new disk storage when it becomes out-dated, unreliable and/or expensive to run. The object-based storage realizes it has lost data but gained fresh storage and begins a background task of copying or replicating data to this new store to replace what it has lost. Typically this is done a few storage nodes at a time.

When using object-based storage in the Cloud, the same applies. However, this is done by the Cloud provider and the user should be completely unaware that this operation is taking place. This type of migration is often referred to as “Incremental Migration.”

The point that many users did not understand at NAB was that active archive solutions that use tape can also employ ‘Incremental Migration.’ Active archives can support more than one tape technology, either in the same library or by supporting multiple libraries. Of course, with LTO, this is simplified, as LTO has the ability to read and write previous generation media and can also read the generation before that.

When upgrading from LTO4 to LTO5, for example, the density of the library typically doubles. From LTO3 to LTO5 it is a fourfold increase. This gain in capacity does not need to happen overnight. By incrementally migrating data, capacity is freed up within the library as it is needed. Older media can be stored as additional disaster recovery (DR) copies, if necessary. This whole operation can be handled as a background task without any disruption to the user workflow.  Active archive databases are updated in the background with the new file locations. The user is unaware if data is coming from new media or old media.

In the future, with tape libraries using LTFS, this process will be even easier. Not only could the storage be updated but also the application controlling the archive can be replaced. The LTFS industry standard file system allows archive applications to read existing data on LTFS tape media. With object-based storage, the application writes in a proprietary format, so migration to a different object-based system is impossible without a complete data migration.

With a good maintenance scheme and periodic replacement of old drives for new, tape libraries have a useful life over 15 years, probably three times that of a disk system. Today, tape is the lowest cost per TB technology for acquisition and running costs and, due to its longevity, it will reduce migration costs in the future as well.