Research question and scope
This protocol measures how often a customer promise changes after the first reply. The unit is one promise made to one customer about one outcome, such as a delivery date, a refund amount, or a callback time. The review counts revisions, not messages, so one revised promise counts once even if it is discussed several times. The study measures promise stability, not whether the final outcome was good.
Set the scope to one queue and one promise type, such as delivery dates or refund decisions. Record the promise definition and the channel used for the first reply. If the queue handles several promise types, run a separate sample for each because they may behave differently. Mixing types hides the pattern that matters to the customer.
Keep the promise types separate because a delivery date and a refund amount fail for different reasons. A delivery date may move because of a carrier, while a refund amount may change because of a policy check. Combining them produces an average that no owner can act on. Report each type with its own denominator.
Record the customer's original request time and the first reply time as separate facts. A late first reply may force a more conservative promise, which then looks unstable. The delay before the first reply is part of the story. Report it so the revision rate is not blamed on the promise alone.
Population and promise unit
Define the population as every first reply during the window that contains a promise or an estimate. Include replies that were made by phone if the promise was recorded in the system. Exclude replies that only acknowledged the request without a date or amount. Count the exclusion so the denominator is clear.
A promise that is restated without a change is not a revision. A revision exists only when the date, amount, condition, or owner changes in a way the customer would notice. Record the exact before and after values. If the customer requested the change, note that separately from a change the team initiated.
Record whether the customer accepted the revision or objected to it. A revision that the customer accepted is a different event from one that triggered a complaint. The response is often visible in the next message or the call note. Include it when the record allows, and mark it as unknown when it does not.
Count a promise revision only when the customer would experience a different outcome. An internal wording change with the same date and amount is not a revision. Applying this test keeps the measure focused on what matters. It also prevents a large count of cosmetic edits.
Definitions and data dictionary
Define the first promise as the earliest recorded commitment after the customer request. Define a revision as any later recorded change to that commitment, with a timestamp and the reason. Define customer notification as the moment the customer was told about the change, not the moment the internal note was written. These three timestamps make the measure reproducible.
Capture the record identifier, the promise type, the original value, each revised value, the reason, and the notifying owner. Add whether the change was caused by new facts, a missing dependency, or an unrealistic first estimate. A reason code is more useful than a free-text note, but allow a short note for cases that do not fit. Test the codes on ten real promises before collection.
Define the notification time from the customer's perspective, not from the internal note. A revision that was written in the system but never sent is not a completed update. The gap between the internal change and the customer message is often where trust is lost. Measure that gap explicitly.
Define who is allowed to revise a promise and whether the revision needs approval. A team that revises freely may look responsive but unstable, while a team that cannot revise may keep a bad promise. Record the approval rule in the report. The rule is part of the promise system, not an afterthought.
Sampling and collection
Draw the sample from the customer communication system rather than from a list of complaints. Record the export date, the filters, the row count, and the selection rule. Include promises from every weekday and every owner so the result is not dominated by one shift. If the population is small, review every promise in the window.
Read the thread in order and record each promise as it appeared at the time, not as it looks after later edits. If a message was deleted or edited, note that and use the remaining record. Do not reconstruct a missing promise from the final outcome. The sample should reflect what the customer could actually see.
Read at least one full thread per owner so the sample is not limited to the tidiest records. Owners with messy threads may be underrepresented in an export that filters by status. Include a few records with missing timestamps and note how they were handled. The sample should reflect the real state of the system.
Include promises made through self-service tools if the customer sees them as commitments. A portal date and a spoken date carry the same weight to the customer. Add the channel to the data dictionary. A measure that ignores self-service will undercount the revisions customers actually experience.
Measuring revision frequency
Report the share of promises revised at least once, the share revised more than once, and the median time between the first promise and the first revision. A high single-revision share may be normal in a complex service, while a high repeat-revision share usually signals an unstable process. Show both numbers rather than one combined rate. Add the count of promises that were never revised so the reader can see the full distribution.
Break the result down by promise type and by cause. If most revisions come from missing information at intake, the fix belongs in the intake form. If most come from a supplier or a system the team does not control, the fix may be a more conservative first promise. Compare like with like and state the sample size.
Report the share of promises revised within the first day separately from revisions that arrived later. A same-day correction may reflect a fast catch, while a late correction may reflect a missed dependency. The timing changes the interpretation. Show both so the owner does not treat every revision as a failure.
Report the time from the first promise to the first revision in business hours and state the calendar used. A revision after two business days means something different from a revision after two calendar weeks. The calendar should match the customer's expectation. Publish it with the result so the numbers can be compared.
Limitations and review cadence
The record cannot show a promise that was made verbally and never written down. It also cannot show how the customer felt about the change if they did not reply. Treat the revision rate as a measure of documented promises, not of all customer experience. Note any verbal promises the team reports.
Review the result with the queue owner and the customer-facing team. Ask for one change to intake, one change to the promise language, or one change to the escalation path. Record the effective date and repeat the measurement on the same promise type. A stable promise is easier to keep and easier to explain.
Ask the customer-facing team which promise is hardest to keep and why. The answer often points to a dependency that the data cannot show, such as a supplier who never confirms in writing. Record the explanation beside the measure. A number with a named cause is easier to fix than a number alone.
Ask the owner to name the one revision they would most want to prevent and why. The answer often reveals a missing check at intake or a dependency that was never confirmed. Record the answer and the proposed change. A single named case is easier to act on than a general goal of fewer revisions.
