Deduplication

Last updated: 2026-05-16

The Deduplicate dialog runs in two modes from the case page menu. Files mode finds files with identical content (even when filenames differ) and removes the extra copies. Emails mode collapses duplicate email threads using inclusive-email rules so you review the full conversation once instead of reading the same exchange five times. Both modes preview before they change anything. The live deduplication action is permanent from the review side. Cases on legal hold can preview but can't apply.

Deduplicate dialog: Files tab

Files mode finds duplicate files by content and removes the extra copies.

Deduplicate

Found 3 duplicate groups (7 files) across 482 scanned files.

Group 1: Contract_MSA_2024.pdf
2 copies
Identical content, different filenames. Earliest copy kept; 1 file removed.
Group 2: Email_Thread_RE_Settlement.eml
3 copies
Identical content, collected from three custodians. Earliest copy kept; 2 files removed.
Group 3: Financial_Records_Q3.xlsx
2 copies
Identical content. Earliest copy kept; 1 file removed.

Deduplicate dialog: Emails tab

Emails mode collapses duplicate threads to inclusive copies.

Deduplicate
Policy
Scope
BCC handling

Two modes, one dialog

Files mode is content-based. Two files with different names but byte-identical content are duplicates. Earliest copy stays, the rest go. Useful after big imports, after multiple custodians contributed overlapping documents, and before production.

Emails mode is thread-based. When a five-reply chain lands in five custodians' mailboxes you end up with 25 message copies. The inclusive copy is the one that contains the full conversation as quoted text. Hintyr identifies it, keeps it visible, and hides the redundant earlier replies from review. Hidden copies stay in the case record, so the audit trail is complete. This pattern aligns with EDRM Processing Standards guidance.

Files mode vs. Emails mode

WhatFiles modeEmails mode
DetectionByte-identical content matchThread reconstruction plus inclusive-email rules
What happens to extrasRemoved from the caseHidden from review; records stay in the case
ReversibilityNot reversible without re-uploadNot reversible from the review interface
Use whenSame document collected from multiple custodiansLong reply chains with the same conversation copied across mailboxes
Legal holdBlocked while a hold is activePreview allowed; Apply blocked while a hold is active

Opening the dialog

Open the case menu from the top navigation, pick Deduplicate, and choose either the Files or Emails tab. The dialog opens with Files selected. Switch tabs at any time before you apply. Each tab runs its own preview before any change lands.

About permanence

Files mode removes the extra copies from the case. There's no trash bin. Emails mode hides the duplicate copies from review but retains the underlying records, including custodian information, original paths, and dates received. Both actions can't be rolled back from the review surface. Run a preview first and confirm the results match expectations before you commit.

When the case is on legal hold

Preview always runs. It's read-only and useful for measuring the reduction you'd get if you ran the same action after the hold releases. The Apply button stays disabled while the hold is active. Release the hold first, then re-open the dialog.

Deduplication subpages

Frequently asked questions

Should I use file dedup or email dedup, or both?
Run both, in order: file dedup first to collapse byte-identical copies of any file (including .eml attachments and PDFs), then email dedup to collapse thread-level redundancy across mailboxes. The two modes don't conflict; they target different overlap patterns.
Can I run dedup multiple times on the same case?
Yes. Both modes are idempotent. After the first run completes, subsequent runs only find new duplicates introduced by later uploads. The preview tells you whether there's anything to act on.
What's the difference between hiding and deleting?
File dedup deletes the extra copies. Email dedup hides duplicate thread copies from the default review surface but keeps the underlying records, including custodian info, original paths, and dates received. That distinction matters when you need to defend the practice to opposing counsel.
Does deduplication run on cases under legal hold?
Preview runs. It's read-only and produces the same numbers you'd get if you applied the change. Apply is blocked until the hold is released. This protects the preservation duty while still letting you measure proportionality.