Running Near-Duplicate & Email Thread Analysis

The Near-Duplicate/Email Thread license is required to run the Near-Duplicate & Email Thread Analysis utility.

1.On the Tools menu, click Near-Duplicate & Email Thread Analysis.

Clicking Near-Duplicate & Email Thread Analysis opens the Near-Duplicate & Email Thread Analysis dialog box, and the Near-Duplicate & Email Thread Analysis utility automatically runs, examining the case and displaying the case's current near-duplication and email thread status. The Refreshing status is displayed while the utility examines the case. After the utility finishes examining the case, the Up to date or Out of date status is displayed for Near Duplicate and Email Threading.

The following near-duplicate information is displayed for the current case:

•Near-Duplicate. The near-duplicate analysis status for the current case.

•Pending Work. Total number of documents to be analyzed for near-duplicates in the current case.

•Email Threading. The email thread analysis status for the current case.

•Pending Work. Total number of documents to be analyzed for email threads in the current case.

•Documents with Tag. Total number of documents in the current case assigned the tag currently selected in the Scope analysis to specified tag list. If the Scope analysis to specified tag check box is not selected, the Documents with Tag field is set to 0.

2.To run the near-duplicate analysis, select the Near-Duplicate check box.

To disable the near-duplicate analysis, clear the Near-Duplicate check box.

If the near-duplicate analysis status is Out of date, the Near-Duplicate check box will automatically be selected. If the near-duplicate analysis status is Up to date, the Near-Duplicate check box is not selected.

3.To run the email thread analysis, select the Email Threading check box.

To disable the email thread analysis, clear the Email Threading check box.

If the email thread analysis status is Out of date, the Email Threading check box will automatically be selected. If the email thread analysis status is Up to date, the Email Threading check box is not selected.

Each time the Near-Duplicate & Email Thread Analysis dialog box is opened, it will retain the last settings for the Near-Duplicate and Email Threading check boxes.

4.If you want to modify the near-duplicate analysis settings, click the Settings link next to Near Duplicate.

Clicking the Settings link opens the Near-Duplicate Settings dialog box.

5.The Cluster Threshold setting determines the minimum similarity to be considered when assigning clusters of related documents. The Family Threshold setting determines the minimum similarity to be considered when assigning documents to near-duplicate families. By default, the Cluster Threshold is set to 70 and the Family Threshold is set to 85. Once you run the Near-Duplicate & Email Thread Analysis utility, the utility retains the last cluster and family threshold settings. If you want to change the cluster and/or family threshold for analyzing the new case documents, adjust the Cluster Threshold and/or Family Threshold settings.

To adjust the Cluster Threshold and Family Threshold settings, click and move the slider to the left or right to change the threshold. Moving the slider to the right increases the threshold, and moving the slider to the left decreases the threshold.

The Ignore email headers and conversation history check box determines whether the Near-Duplicate & Email Thread Analysis utility performs the near-duplicate analysis on all text within an email message or only the top-level message body of an email message. By default, the check box is not selected. When the check box is not selected, the near-duplicate analysis is performed on all text within an email message.

6.To limit the near-duplicate analysis to only the top-level message body for email messages, select the Ignore email headers and conversation history check box. To have the near-duplicate analysis review all text within an email message, make sure the Ignore email headers and conversation history check box is not selected.

7.Click OK.

Clicking OK saves the near-duplicate settings and closes the Near Duplicate Settings dialog box.

8.If you want to modify the email threading analysis settings, click the Settings link next to Email Threading.

Clicking the Settings link opens the Email Threading Settings dialog box.

The Populate email metadata from analyzed text check box determines whether the Near-Duplicate & Email Thread Analysis utility will populate the following fields from the text analyzed in the email messages:

EMail_Subject

DateSent

TimeSent

From

To

BCC

CC

Attach

The Near-Duplicate & Email Thread Analysis utility also populates the ET_MetaUpdate field. If the check box is selected, then ET_MetaUpdate field is populated with a Y for yes. If the check box is not selected, the ET_MetaUpdate field is populated with an N for no.

For more information about these fields see, Field Descriptions.

By default, the Populate email metadata from analyzed text check box is not selected. Once you run the Near-Duplicate & Email Thread Analysis utility, the utility retains the last Populate email metadata from analyzed text check box setting.

9.If you want the utility to populate the fields if they do not contain any values, select the Populate email metadata from analyzed text check box.

Clear the check box if you do not want the utility to populate the fields.

10.Click OK.

Clicking OK saves the email threading settings and closes the Email Threading Settings dialog box.

By default, the Scope analysis to specified tag check box is not selected and the Preserve existing near-duplicate families and master documents check box is selected. Once you run the utility the check boxes retain the last settings for the check boxes. So if a check box was selected, the next time you open the Near-Duplicate & Email Thread Analysis utility, the check box will be selected.

11.If you want the Near-Duplicate & Email Thread Analysis utility to only analyze documents containing a specific tag, select the Scope analysis to specified tag check box and in the tag list, click the applicable tag.

When the Scope analysis to specified tag check box is selected, the Document with Tag field displays the number of documents in the case the selected tag is currently applied to.

When the Preserve existing near-duplicate families and master documents check box is selected, the Near-Duplicate & Email Thread Analysis utility will retain the existing near-duplicate family and master document IDs already assigned to the documents in the ND_IsMaster and ND_FamilyhID fields when the utility previously ran. Any new documents identified in an existing near-duplicate family will be added into the existing near-duplicate family.

When the Preserve existing near-duplicate families and master documents check box is not selected any existing near-duplicate family and master document data, such as IDs, is deleted and new near-duplicate data is assigned to all master documents and documents within a near-duplicate family.

12.If you want to retain existing near-duplicate family and master document data, select the Preserve existing near-duplicate families and master documents check box.

Clear the check box if you want to delete existing near-duplicate family and master document data and have the utility reassign new near-duplicate data.

13.Click the Start button.

Clicking the Start button saves all the near-duplicate and email threading settings and starts the Near-Duplicate & Email Thread Analysis utility. The Near Duplicate & Email Thread Analysis status dialog box is displayed while the utility runs.

If the Near-Duplicate & Email Thread Analysis utility encounters any errors during the process, an error log is generated and saved as a .log file in the LAW case directory. By default, the near-duplicate and email thread analysis error logs are stored in the ...Program Files (x86)\Law50\Cases\[case name folder]\$Logs\NearDup directory.

During the near-duplicate and email thread analysis process, the utility generates a near-duplicate and email threading index of every case file, unless the Scope analysis to specified tag check box is selected. In that scenario, the utility generates a near-duplicate and email threading index of the case files tagged with the selected tag. Once the Near-Duplicate & Email Thread Analysis utility successfully finishes running, the Analytics Processing Complete message is displayed.

14.Click Close.

If you ran the near-duplicate analysis, you can now view and compare the case's near-duplicate documents in the Duplicate Viewer dialog box. If you ran the email thread analysis, you can now view the email threading information for the case's documents in the grid view.

For more information, see Viewing Near Duplicates and Viewing Email Threads.