What to ask before you build or buy automatic redaction

Across legal tech, government work, and procurement, one of the most commonly requested features I get is for automatic redaction of sensitive information from documents. And I'm happy to oblige (we built it into SquarePact and published a downloadable skill). But usually significant clarification is needed first on what kind of redaction they want the system to automatically perform, and why. So here are some notes on what to consider before developing (or purchasing) an automatic redaction system.

Most of that clarification comes down to six questions, and they apply whether you're building the system or buying one.

Which rule is it redacting to?

There isn't one list of what counts as sensitive, and the lists that exist contradict each other. The federal court rule keeps part of each value: the last four digits of a Social Security or account number, the year of birth, and a minor's initials. HIPAA's Safe Harbor method removes 18 kinds of identifier. It also allows no partial values, so those same initials and last four digits have to go.

Florida's own courts are stricter than the federal rule, because their rule allows no part of a Social Security or bank account number at all:

So a system told to "redact the PII" has to guess which rule you meant. If it can't tell you which one it applied, you can't check its work.

What has to stay visible?

You might then think, "Well, I'll just be extremely aggressive and black out everything that seems remotely personal." But some rules actually forbid this. In Florida, the records custodian can redact only the part that's exempt and has to release the rest. Federal FOIA also requires any reasonably segregable portion of a record to be released once the exempt parts are deleted. And some things you'd instinctively hide are public anyway. Under Florida law, an email address someone sends to an agency is a public record.

Labeling a document doesn't decide what stays hidden, either. If you give a Florida agency your trade secrets, you have to mark them before you hand them over, or you lose the protection. But the label alone doesn't make something a trade secret. When Yellow Cab asked Broward County for Uber's airport reports, a Florida appeals court held that what Uber paid the county under its airport license wasn't a trade secret, even though Uber had marked its reports as trade secrets.

Does it catch what identifies someone without a name?

Finding names and account numbers is the easy part. When Victoria's privacy regulator in Australia investigated a child protection worker who'd used ChatGPT to draft a court report, it pointed out that a document with the names removed can still be re-identified from everything else in it. HIPAA reaches past the patient, too, because the Safe Harbor list covers the identifiers of the patient's relatives, employer and household members.

Is the text gone from the file?

If the text is still underneath, a black box over it hides nothing. In 2004, CNET opened the Microsoft Word file of SCO's lawsuit against DaimlerChrysler and found, in the tracked changes, that it had originally named Bank of America as the defendant. In 2019, Paul Manafort's lawyers filed a response whose sealed sections anyone could read by copying the blacked-out text and pasting it somewhere else.

AI agents add a new way to get this wrong. When an agent marks redactions in a Word file as tracked changes, the original text stays in the file until someone accepts them, so a half-finished file shows every passage that was marked for redaction. And the usual advice to run Word's Document Inspector doesn't help everyone, because Word for Mac doesn't have a Document Inspector at all.

Where does the document go while it's being redacted?

If the redaction runs on a cloud AI service, that service reads the original, which is the exposure the redaction was supposed to prevent. Employees use AI in unsanctioned ways. In 2023, employees in Samsung's semiconductor division pasted source code and a transcribed meeting into ChatGPT, within three weeks of the company letting them use it. Samsung responded by capping uploads at 1,024 bytes per question, and it warned staff that once something's entered, the company can't get it back. Likewise, in a University of Melbourne and KPMG survey, about half of the employees who use AI at work admitted uploading sensitive company information to public AI tools.

For lawyers it's also an ethics question. The ABA says you need your client's informed consent before putting their information into a self-learning AI tool, and a boilerplate line in your engagement letter doesn't count. And this February, a federal judge in New York ruled that a defendant's exchanges with Claude weren't privileged. Even if some of what he typed had been, the judge wrote, sharing it with Claude waived the privilege.

What record does it leave?

Someone will eventually ask why each piece came out. Florida's records law makes the custodian cite the statute behind each redaction. Federal FOIA goes further, because each deletion has to show how much was removed and which exemption applies, unless showing that would give away what the exemption protects. A system that only blacks things out leaves you rebuilding those reasons by hand.

Using a Skill File

If you want to use an AI agent to perform redaction, I built a skill around these questions that you can use. It asks which standard you're working under before it marks anything. Each standard is one plain file with the statutes bundled in, so when the skill cites a law, it quotes the text in that file, and you can edit the file to match your own policy. It keeps a redaction log that never repeats a redacted value, and it tells you which parts of the file it couldn't check. Then it walks you through finishing the job in Word, including Word for Mac. It runs inside your agent, though, so your agent still reads the original, and it can't take back what an AI tool has already seen.

You can install it in Claude Code with /plugin install redaction@licato, or get it for other agents at actualization.ai/skills. It's also built into SquarePact for Word if you prefer zero data retention for the redaction step.

I share more findings like this on my mailing list, Dr. Licato's AI Field Notes.