Deduplication technologies do not directly compare file contents. Instead, the encryption signatures, also known as the hash values, of files are compared. If the hash values for two different items are identical, the content of the two files is assumed to also be identical. Key generation then, refers to the process of creating an encryption signature for a file so that files can be easily compared. File hashing and metadata hashing are the two primary methods used by the ED Loader and Turbo Import for generating keys.

Currently, the ED Loader and Turbo Import generate two output hashes in parallel: MD5 and SHA-1. In the ED Loader and Turbo Import options you can choose which of these is used in the deduplication process.

•E-docs. The key value is generated using the entire file as the input.

•E-mail. The key value is generated using an input value of certain metadata after processing of the metadata fields has been executed. The purpose of using the post-processed metadata is to match the metadata that is stored in CloudNine™ LAW; therefore, if the key is regenerated in the future, the value would match the original. E-mail includes both e-mail messages contained in mail stores and loose e-mail messages. The term "loose e-mail" refers to a file that is identified as a mail item and successfully converted to a mail item by Outlook. These include .msg files, .eml files, and other RFC822-format e-mails.

For Microsoft Outlook PST files, e-mail also includes non-email items — calendar items, contacts, journal entries, notes, and tasks.

Electronic Discovery in LAW uses the fields shown in the following table to generate the hash values for e-mail, loose e-mail items, and Outlook PST non-email items in CloudNine™ LAW: