Research question and scope
This protocol measures the share of documents that fail a redaction check on first review before external sharing. The unit is one document prepared for a recipient outside the team. A failure means at least one required field remains visible or recoverable, or the file metadata exposes it. The measure describes the document and the check, not the intent of the preparer.
Set the scope to one business function, one field list, and a fixed period. Record the document types, the sharing channels, and the redaction tools used. If the field list changes during the period, split the sample at the effective date. A new required field changes the result.
Keep the measure separate from a legal compliance judgment. A failed check may still be within policy if an exception was approved. The purpose is to find where the process allows data to leave. State that purpose and the limits of the review.
Population and unit of analysis
Define the population as every document prepared for external sharing in the period, including documents that passed on the first review. The unit is the document, not the field or the page. One document may fail three fields and should count once as a failed document while the fields are reported separately. Use the file identifier to group them.
Include documents shared with a client, a vendor, and a regulator. Report the recipient group separately because the field list may differ. A payroll summary and a public brochure do not carry the same risk. Excluding one group would narrow the result without saying so.
Exclude internal drafts that never left the team and documents that were withdrawn before sending. Report the withdrawn count separately. A draft cannot fail an external sharing check. Keep the two groups distinct in the analysis.
Definitions that make the measure reproducible
Define a required field as any item on the published list for that document type. Define a visible failure as a field a reader can see in the final file. Define a hidden-layer failure as a field recoverable from metadata, comments, tracked changes, or an embedded object. Publish one example of each.
Define a pass as a document where every required field is absent and no hidden layer exposes one. Define an approved exception as a field a named owner allowed to remain with a reason. A pass with an exception should be reported as an exception, not as a clean pass. The distinction keeps the result honest.
Decide how to treat an illegible image or a partial name. A field is present if a reasonable recipient could identify the person or account. Record the judgment with a short note. Two reviewers should apply the same standard before the full sample.
Sampling and data collection
Draw the sample from the sharing log rather than from documents that were already caught. Record the export date, the filters, and the row count. Include every document type and recipient group in the period. A sample of failed documents alone cannot produce a failure rate.
Capture the file identifier, the document type, the recipient group, the required fields, the failure layer, the reviewer, and the outcome. Add the final file that was actually shared. Store the export with the report. The final file is the core evidence.
Check a small overlap sample with a second reviewer. If the two reviewers disagree about whether a hidden layer exposes a field, the definition is too vague. Rewrite the rule and retest before the full sample. Agreement matters more than sample size for a check that will be reused.
Record the time from first review to release. A check that finds a failure after the file has been sent is a recovery step, not a control. Report the timing with the result. The rate alone does not show whether the check prevented disclosure.
Measuring redaction failures
Report the share of documents that passed, the share with a visible failure, and the share with a hidden-layer failure. Show the counts behind each share. The groups describe different problems and should not be collapsed into one rate. A single rate hides whether the template or the reviewer is the source.
Break the result down by field, document type, and recipient group. A field that fails often may need a template change rather than more training. A document type with many failures may need a stronger check. Compare only groups with similar volume and risk.
Report the documents that failed more than once and the oldest unresolved failure. A repeated document may point to a shared template that no one owns. The oldest failure may show a decision that was never made. Both examples make the aggregate result actionable.
Limitations and next action
The measure cannot prove that a document contains no personal data; it can only test the required fields on the list. It also cannot show a disclosure that happened through another channel. Treat the result as a measure of the check. Note the document types and tools that were not included.
Give the result to the document owner and the privacy lead with one change to test. The change might be a new template field, a true redaction tool, or a second reader for one document type. Record the effective date and repeat the measure with the same definitions. A controlled change is easier to judge than a broad instruction.
Review the hidden-layer failures first because they are the least visible to the preparer. A single export setting may remove many of them. Fix the tool or the template before asking for more care. The goal is a process that removes the data, not a longer warning.
