Different Types Of Data Deduplication

801 views This article is copyright free and is published in Data Recovery » Computers
Joined: May 27, 2013
216 articles
There are several different ways that data deduplication works. The type that is used will depend on the data being fixed as well as the storage space needed. While some data deduplication techniques might be used alone, others may be used in conjunction with another. Some might be take away too much data, while others leave too much. This is why some companies will choose to use a combination of all three methods. The three main types of data deduplication include hash based, chunking and primary and secondary storage deduplication.

Hash Based

In hash based data deduplication, the data is processed in chunks that end with hash numbers. These hash numbers are a symbol to the system that the data has already been processed. When data is being processed, the system will read these hash numbers and look for duplicates of these numbers. Any time a duplicate hash number is found, it is not saved in the system. However, sometimes this can result in false positives and data might be missed because the system believes it is a duplicate when it is not. That is why hash based data deduplication is usually used in conjunction with another type of deduplication. In this even, all positives might be added to a secondary storage source to be rechecked using a different process.


Chunking data is just what it sounds like. Data will be given chunking boundaries and then compeered to other chunks of data. When duplicates in these chunks are found, they will be eliminated. There are two different types of chunking. In the first case, the streams of data are reviews in static chunks that are only visited once. In more advanced chunking, the chunks are reviews in a sliding window and changed to be compared multiple times. In this type of data deduplication, duplicates might be missed. This is why it is sometimes used in conjunction with hash based data deduplication. That way, the data will be as compressed as possible and at the same time, key pieces of data won’t be lost.

Primary and Secondary Storage Data Deduplication

This process is actually more of a back to the original backup in this instance, there is a primary storage source that holds all the originally data, while the secondary storage systems will only hold secondary or duplicate data. The goal in this case is to optimize the performance of the primary storage system and maintain cost effectiveness. The problems from these systems occur when the system needs to be recovered, as the data must be sifted through to find the best data to reload into the system.

About author: Robert Fogarty

A very good data deduplication system will often use a combination of all three of these types of data deduplication processes. It might use the chunking method first, to find the moist obvious duplicates. The hash based might be used to more thoroughly check for duplicates. To avoid data loss, a secondary storage system may be used after both of these processes are followed. This ensures that space is optimized without the loss of important data.


Other article from Computers

Effective Strategies For Application Server
It is software framework that handles operations of almost all applications connecting users and organizations. It is also called apps server. It is usually used for multifaceted operation based applications. An application server enclose all-inclusive ...
View by 587 people
Posted by Robert Fogarty in ,
An Unbiased View Of Internetwork Packet Exchange
Internetwork Pocket Exchange is networking protocol supported by Novel’s operating systems. Internetwork Pocket Exchange become most popular and well liked protocol in the late 1980 and mind of 1990 because of Netware’s popularity. ...
View by 475 people
Posted by Robert Fogarty in ,
Top Reasons To Buy A Tablet
Considering investing in purchasing your very first tablet computer? Check out these top reasons to go ahead and make the plunge. Makes Travel Easier It used to be that carrying a heavy laptop was the only way to keep your files with you when ...
View by 483 people
Posted by Robert Fogarty in ,
The new CAD soft ware for smart phone has been published by Apple
Recently, the U.S. Patent and Trademark Office have approved an application for Apple's patent virtual drafting tool. This patent of Apple has already described the detail information about the interface of very innovative CAD ...
View by 305 people
Posted by Irfanuddin in ,
The precaution about selection of the mouse for the CAD user
Compared to the game users¡¯ requirements for mouse which should be good at product¡¯s hardware mouse and appearance, the opinion of CAD software user for the selection of the mouse would be focus on the actual using ...
View by 334 people
Posted by Irfanuddin in ,