Research question and scope
This protocol measures the share of shipped orders that lack a matching delivery proof when the desk closes the order. The unit is one shipped order closed during the period. A missing proof means no acceptable carrier or recipient record was attached at closing. The measure describes the record, not whether the parcel arrived.
Set the scope to one order desk, one carrier set, and a fixed period. Record the proof forms accepted, the match fields, and the point at which an order is closed. If the acceptance rule changes during the period, split the sample at the effective date. A new proof type changes the result.
Keep the measure separate from a customer satisfaction score. A missing proof may still be followed by a satisfied customer, and a valid proof may sit on a disputed delivery. The purpose is to find where the record is incomplete. State that purpose in the report.
Population and unit of analysis
Define the population as every order that shipped and was closed in the period. The unit is the order, not the parcel or the tracking event. One order may ship in several boxes, and each box should be checked while the order counts once. Use the order identifier to group the parcels.
Include partial deliveries and orders closed after a carrier inquiry. Report them by status rather than dropping them. A partial delivery has proof for some items and none for others. Excluding it would hide the most complex cases.
Exclude orders cancelled before shipment and orders still open at the end of the period. Report the open count separately. An open order cannot fail a closing check. Keep the boundary visible in the report.
Definitions that make the measure reproducible
Define an acceptable proof as a carrier or recipient record that matches the tracking number, the delivery address, and the delivery time. Define a mismatch as a proof that matches the tracking number but not the address or the time. Define a missing proof as no record at closing. Publish one example of each.
Define the closing point as the timestamp when the order status becomes complete. Define the proof window as the period after delivery in which the record may still arrive. A proof that appears after closing is a late proof, not a match. Record the delay separately.
Decide how to treat a signature that is illegible or a photo that shows only a door. A record can be acceptable if it matches the address and the carrier scan. Record the limitation rather than calling it a failure. The rule should be the same for every reviewer.
Sampling and data collection
Draw the sample from the order system and join it to the carrier records by tracking number. Record the export date, the filters, and the row count. Include every carrier and every order value band. A sample of disputed orders alone will overstate the gap.
Capture the order identifier, the tracking number, the carrier, the delivery time, the address match, the proof type, the closing time, and the status. Add the carrier record that supports the match. Store the export with the report. The join is the core evidence.
Check a small overlap sample with a second reviewer. If the two reviewers disagree about whether a photo is an acceptable proof, the definition is too vague. Rewrite the rule and retest before the full sample. Agreement matters more than sample size for a rule that will be reused.
Record the time between delivery and the proof record. A carrier feed may arrive hours late, which can make a valid order look missing at closing. Keep the delay beside the status. The detail separates a feed problem from a desk error.
Measuring proof gaps
Report the share of orders matched, the share missing a proof, the share with a mismatch, and the share with a partial proof. Show the counts behind each share. The groups describe different failures and should not be collapsed into one rate. A single rate hides whether the desk or the carrier is the source.
Break the result down by carrier, route, order value, and proof type. A carrier with many missing scans may need a different feed or a manual step. A high-value band with many mismatches may need a stronger acceptance rule. Compare only groups with similar volume.
Report the median and longest delay for late proofs. A proof that arrives one hour after closing is different from one that never arrives. The delay distribution shows whether the problem is a feed timing issue or a real gap. Include the closing rule in the report.
Limitations and next action
The measure cannot show a parcel that was delivered without any carrier record or a customer who received the order but disputed the proof. It also cannot show the condition of the goods. Treat the result as a measure of the closing record. Note the carriers and order types that were not included.
Give the result to the order desk and the carrier contact with one change to test. The change might be a delayed closing window, a required photo for high-value orders, or a manual proof step for one carrier. Record the effective date and repeat the measure with the same definitions. A controlled change is easier to judge than a broad push.
Review the mismatch cases before changing the whole rule. A few addresses with apartment numbers may explain a large share of the gaps. Fix the address check first if that is the pattern. The goal is a closing record the team can defend, not a lower gap by definition.
