De-Duping Data

Yesterday I wrote a little bit about how you can stack DVDs. Well, the next logical challenge here is to reduce the size of data, especially redundant data. According to the same IDC report 75% of the world's data is a copy -- meaning only 25% is unique! Think of all the plastic we could save.

Well, this calls for the ability to get rid of all the replicated data in a process of de-duplication, or what I like to call Single Instance Storaging (SIS). Now, the intracies of SISing over an entire database can be quite cumbersome. Imagine trying to access each database, fileshare, or hard drive over a network. SISing over an archive can also be challenging as well, if solutions do not deploy the right architecture. Because of poor source code, many solutions cannot seamlessly scale one archive to fit an entire Fortune 1000-level corporation.

However, it is necessary to ensure that all the files are stored in one virtual vault, otherwise it is impossible to achieve true SIS. And without true SIS, how do enterprises cut down on that 75%?

True SIS also lends itself to legal concerns; when a legal hold needs to be put in place, in-house counsel can guarantee that there is only one document, and that it is being preserved.