How to redact a PDF so the text is really gone

Drawing a black rectangle over text in a PDF hides it from the eye and from nobody else. The words are still in the file, under the box, and anyone can copy them out. It is the most common redaction mistake, and it has a long history. In 2005, a few hours after the Pentagon published a censored report on the death of the Italian intelligence officer Nicola Calipari, readers made the blacked-out paragraphs reappear by pasting them into a Word document.

Real redaction removes the content from the file. This guide shows the difference on a test document, lists the other places a PDF keeps text you cannot see, and ends with a checklist for before you send.

The test: a box versus a redaction

We made a one-page PDF with one sentence: "Payment to account DE89 3704 0044 0532 0130 00 is due on 1 March." The account number is a widely used example IBAN, not a real account. We gave the file a title and an author, then drew a black rectangle over the account number, the way many people redact.

On screen the number was gone. Then we ran the file through pdftotext, a common command-line text extractor, and got this back:

Payment to account DE89 3704 0044 0532 0130 00 is due on 1 March.

The whole sentence, number included. The document properties still showed the title and the author. You do not even need a tool: select the text in a viewer, copy it, and paste it into an email.

Then we redacted the same area with Redact PDF, which redraws every page as an image with the marked areas painted black. This time pdftotext returned nothing, because there was no text left in the file. The title and author were gone too, because the redacted PDF is a new file built only from the page images.

The test caught a second mistake. Our box started a few points too far to the right, and the rendered page still showed the "D" of "DE89". A redaction can be permanent and still leave part of a number visible, so check the picture as well as the text.

Why the box does not work

A PDF page is a list of drawing instructions: put this text here in this font, fill this rectangle, place this image. A black box is one more instruction, drawn on top. The text instruction under it is untouched. Text extraction, search, copy and paste, and screen readers all work from the instructions, not from what you see.

A 2005 guide from the US National Security Agency on publishing sanitised documents names three common mistakes: covering text with black, covering parts of images, and leaving metadata in place. Its main advice still holds: delete sensitive content, do not just hide it.

Where else a PDF keeps text

Covering the visible page is not the whole job. The PDF standard, ISO 32000, defines several places that hold content you will not see on the page:

• Metadata. The document information dictionary (title, author, subject, keywords, the software that made the file) and an XMP metadata stream (section 14.3). A title like "Settlement draft for J. Smith" can say more than the page.
• Annotations. Comments, highlights, sticky notes, and the boxes some editors draw as redaction marks before they are applied, are annotations (section 12.5). They sit on top of the page and can be deleted to reveal what is underneath.
• Form fields. Field values are stored apart from the page drawing (section 12.7) and survive when the page is covered.
• Bookmarks. Outline entries (section 12.3.3) often repeat headings and names.
• Attachments. A PDF can embed other files (section 7.11.4), such as the Word document it was made from. The NSA guide warns about exactly this conversion setting.
• Earlier versions. When a PDF is saved incrementally, changes are appended, and the standard says that "deleted objects shall be left unchanged in the file" (section 7.5.6). A redaction saved this way can leave the old page inside the file.
• Hidden OCR text. A scan that has been through OCR has an invisible text layer under the picture. Black out the picture and the recognised text is still there.
• Tags and alternative text. Accessible PDFs carry a structure tree and alternative text (section 14.8) that can describe what a figure shows.

Two ways to redact properly

Remove the content and keep the rest as text. Full PDF editors have redaction features that delete the text, image pixels and shapes under each marked area, and can then clear metadata, comments and hidden layers. Done right, the rest of the document stays searchable and small. With one option left unticked, some of the hidden content above can survive.

Rebuild every page as an image. This is how Redact PDF works. Each page is rendered at 150 dpi, the marked areas are painted black, and a new PDF is made from those pictures. Nothing that was not visible on the rendered page can survive: no text layer, metadata, attachments, bookmarks, form fields or earlier versions.

It has costs:

• No selectable text. The document becomes pictures, so it cannot be searched or copied, and a screen reader has nothing to read. You can run OCR on the result to make it searchable again. The redacted areas are black pixels, so there is nothing for OCR to recover from them.
• Larger files for text documents. Our one-line test page went from 1.2 KB to 12.5 KB. A scanned document changes much less, because it was already a picture.
• What is visible stays visible. Rendering removes what is hidden, not what is on the page. Anything you did not mark is in the picture.

For a court filing, a records request or a document going to the other side of a negotiation, rebuilding the pages is the approach that is hardest to get wrong.

A checklist before you send

1. Work on a copy. Keep the unredacted original apart, and name the redacted file so nobody sends the wrong one.
2. Mark generously. Cover the whole word with a margin, including descenders and any nearby text that gives it away.
3. Search for what you removed. Open the result, press Ctrl+F or Cmd+F, and search for a name or number you redacted. It should not be found.
4. Copy everything. Select all, copy, and paste into a plain text editor, or extract the text with PDF to text. The redacted words must not appear.
5. Check the properties. Look at the title, author and subject in the document properties, or clear them with Remove PDF metadata.
6. Look at every redacted area at high zoom. Make sure each edge is covered and no partial letters show.
7. Check what the rules require. In US federal courts, for example, Rule 5.2 of the Federal Rules of Civil Procedure allows only the last four digits of social security, taxpayer and financial account numbers, the year of birth, and a minor's initials in a filing.

Redact PDF runs in your browser, so the unredacted file is never uploaded while you work on it.

Sources

• "Readers 'declassify' US document", BBC News, May 2005. news.bbc.co.uk
• National Security Agency, "Redacting with Confidence: How to Safely Publish Sanitized Reports Converted From Word to PDF", report I333-015R-2005, 13 December 2005, archived copy. web.archive.org
• ISO 32000-1:2008, PDF 1.7, Adobe's public copy: sections 7.5.6, 7.11.4, 12.3.3, 12.5, 12.7, 14.3 and 14.8. PDF32000_2008.pdf
• Federal Rules of Civil Procedure, Rule 5.2, Legal Information Institute, Cornell Law School. law.cornell.edu