Remove duplicate pages
What Remove duplicate pages actually does
A repeated page in a PDF is almost never a repeated object. A stack that went through the scanner twice, a packet where a cover sheet was appended to every section, a merged bundle where two contributors both included the same exhibit: in each case the pages look identical to a reader and are different objects inside the file, with their own images, their own names for those images and their own byte layout. So a tool that compares the internals answers the wrong question. It answers it in both directions, and one of them destroys data. It misses the real duplicates, because two scans of the same sheet are never byte-identical. And it can report duplicates that are not, because a great many writers emit the same drawing instructions on every page of an image-per-page document and keep the differing picture in each page's own resource table, so pages that share nothing but a template are declared the same. That is not hypothetical; it is what the most common library for writing scanned PDFs does. This page asks the reader's question instead. Every page is drawn at a stated resolution, reduced to gray, and compared as a picture. The resolution is a control, because it is the definition of how different two pages have to be to count as different. Groups are named before anything is removed, the first page of each group is kept, and the pages that survive are listed.
How to use it
- Choose the PDF. Every page is drawn in this tab; nothing is sent anywhere.
- Set the comparison resolution if the default of fifty needs changing. Higher notices smaller differences and takes longer.
- Run it. Each page is rendered and compared as an image.
- Read the groups the checks panel names, and the list of pages kept.
- Download the trimmed document, or change the resolution and run it again.
Useful for
- Clean a stack that went through a sheet feeder twice without noticing which sheets doubled.
- Strip a repeated cover or separator sheet out of a merged packet.
- Reconcile a bundle where two contributors both attached the same exhibit.
- Check whether a document has duplicates at all, without removing anything, by reading the groups and downloading nothing.
Limits worth knowing
- Two pages differing only below the comparison resolution are treated as the same page. Raising the resolution is the control for that, and it costs time.
- The first page of each group is the one kept. There is no per-group choice of which copy survives.
- A near-duplicate is not found. This groups pages that render identically, not pages that are similar.
- A page that fails to render is compared as whatever came back, so a document the reader cannot draw is not a document this can judge.
- Rendering every page is the slowest operation on this site. A long document takes a while and reports its progress rather than freezing.
Questions people ask
undefined
undefined
undefined
undefined
undefined
undefined
undefined
undefined