Research question and scope
This protocol measures the share of vendor follow-ups that arrive after the date the vendor promised. The unit is one promised item, such as a quote, a document, a delivery date, or a replacement confirmation. A follow-up is late when the first usable response arrives after the promised date. The study measures promise reliability, not whether the vendor was polite or responsive.
Set the scope to one client, one vendor list, and one follow-up channel for a fixed window. Record the request types included and the cadence version in use at the start. If the vendor list or cadence changes during the window, keep the earlier definition for the full sample. A new reminder rule can change the measured rate without changing vendor behavior.
Population and unit of analysis
Define the population as every vendor promise recorded during the window. The unit is the promise, not the thread, because one thread can contain several promised items with different dates. A promise that is revised remains one unit with a revision event. Record each revision so the analysis can separate a new commitment from a missed one.
Exclude promises that were withdrawn before the date and record the exclusion count. Include promises that were met by a different contact or channel, because the customer only sees the result. Include items with no promised date as a separate category rather than forcing them into the late group. Test the unit definition on five threads before collection begins.
Definitions and date rules
Define the promised date as the date the vendor stated in writing or the date recorded by the team after a call. Define arrival as the first response that contains the usable item, not the first message that acknowledges the request. Define late as arrival after the end of the promised business day in the client's time zone. Write these rules before looking at the results.
Do not treat a promise to try as a promised date. Do not count an automated acknowledgment as a usable response. If the vendor promised a range, record the end of the range and note the range in the packet. Each definition needs a positive and a boundary case so a second reviewer can apply it consistently.
Evidence standard and data dictionary
For each promise, capture the thread identifier, vendor, request type, promised date, promised item, actual arrival, contact, and channel. Add the message that contains the promise and the message that contains the arrival. Record any revision and the reason given. Store the packet with the vendor record so the evidence survives a later dispute.
Also capture whether the team followed the cadence on time. A late response caused by a late reminder is a team problem, not a vendor problem. Record the reminder dates and the person who sent them. If the reason code is other, require a short note. A reviewer should be able to verify the result from the packet without reading the entire thread.
Sampling and collection
Draw the sample from the shared follow-up log rather than from a list of complaints. Record the export date, filters, row count, and selection rule. Include every vendor and request type that appears in the log during the window. If one vendor dominates the population, report its share separately so the overall rate is not mistaken for the whole vendor base.
Collect the thread as it existed at the promised date, not as it looks after later edits. If the vendor changed the date in a later message, record both dates and the time of the change. Do not infer a promise from the team's expectation. Note the collection date separately so the reader can judge the age of the record.
Measuring promise versus arrival
Report the number of promises, the share that arrived late, and the median days late. Break the result down by vendor, request type, and contact. A vendor that is late on quotes but reliable on delivery needs a different response from one that misses everything. The breakdown is more useful than a single vendor score.
Show on-time promises separately from promises that were revised before the date. A revision is not the same as a miss, but it can still delay the customer. Report the revision rate beside the late rate. If revisions are common, the first promise may be unrealistic rather than the vendor unreliable.
Analysis and reporting
Report counts and rates with the same denominator for every comparison. Show the distribution of lateness rather than only the average, because a few very late items can distort the mean. Use a small table with the request types and the number of promises in each. State the exclusions and the missing records.
Compare the late rate with the cadence version and the reminder dates. If lateness falls when the team sends the second reminder earlier, the cadence is doing the work. If lateness stays the same across cadence versions, the vendor's behavior is the stronger explanation. Do not claim causation from an uncontrolled comparison.
Limitations and sources of error
The log cannot show promises made in a call that nobody recorded. It also cannot show work that arrived early but was unusable, unless the reviewer recorded the quality problem. A vendor may be late because the request was incomplete, and the packet may not make that visible. The study measures recorded promises and recorded arrivals.
Threads can be split across channels, and a vendor may reply to one person while another waits. An automated message may look like a response until someone checks the content. Record those cases and their treatment rather than dropping them. If the person who sent the reminder also recorded the arrival, note the independence limitation.
Review cadence and next action
Give the result to the vendor owner and the operations lead and ask for one observable change. The change might be a required promised date, a named backup contact, or a second reminder that fires sooner. Record the owner, the effective date, and the measure that will show whether the change worked. Avoid changing the cadence and the vendor contact at the same time.
Repeat the protocol with the same definitions after the change and compare the same request types. If lateness fell but the number of promises also fell, the team may be recording fewer commitments rather than getting better service. Ask the vendor owner to confirm that the same work is still being tracked. Close the review when the owner accepts the finding and the next sample is scheduled.
Vendor-level patterns
Rank vendors by the number of late promises and by the median days late. Do not rank by percentage alone when the volume is small. A vendor with two promises and one late item is not the same risk as a vendor with forty promises and twelve late items. Show both the count and the rate beside each name.
Look for a pattern in the request type and the contact. A vendor may be reliable with the account manager and unreliable with the finance desk. The fix may be a different route rather than a tougher reminder. Record what changed after the team switched contacts so the next review can test it.
Reporting to the client
Report the result in a form the client can use in a vendor conversation: which request types are late, which promises were missed, and what the team did before escalating. Avoid framing the report as a complaint about the vendor. Use examples with identifying details removed. The client should leave with one change it can request.
Show the denominator and the exclusions so the client can judge the sample. If the sample is too small to support a conclusion, say so. A cautious report is more useful than a confident one built on a handful of threads. Offer to repeat the measurement after the vendor conversation and name the date.
Replication and version control
Keep the definition, the cadence version, and the export query with the report. A second analyst should be able to repeat the measurement on a later window and compare the same request types. If the definition changes, publish a new baseline rather than mixing periods. Replication is what turns a vendor complaint into an operating measure.
Schedule the next measurement before the first report is filed. The date should be far enough away for the vendor change to take effect but close enough to keep attention. Assign the owner who will run it. A protocol that is never repeated becomes a story about the past. Put the date in the same calendar as the vendor review so it is not forgotten.
