Research question and scope
This protocol asks how complete the onboarding evidence is for a new outsourced role before the person handles live work. The unit is one onboarding file for one person and one role. The study measures whether the file contains the task walkthrough, the access approval, the supervised case, and the first review. It does not evaluate the person's overall performance or the quality of the training content.
Set the scope to one site, one role family, and one onboarding template. Record the access system, the review process, and the start date definitions at the beginning. If the template changes during the window, keep the earlier definition for the full sample or analyze the change as a separate period. A new checklist can look like better evidence when it is only a new form.
Population and unit of analysis
Define the population as every person who started the selected role during the window, including people who left before the first review. The unit is the onboarding file, identified by the person and the start date. A person who changes roles within the window is a new unit with a new role. A person who is rehired after a break is a new unit with a reference to the earlier file.
Record the role, the start date, the reviewer, and whether the person handled live work during the window. These strata matter because a file for a role that never touched customer data carries a different risk. Include files that are incomplete because the person left early, and record the reason. Test the unit definition on two ordinary files before collection begins.
Definitions that make the measure reproducible
Define a complete onboarding file as one that contains the signed role brief, the task walkthrough record, the access approval with a named approver, the supervised first case, and the 30-day review. Define a supervised case as one where a named reviewer inspected the work before it reached a customer or a live record. Define an access approval as a record that names the permission, the system, and the expiry. Write these definitions into the onboarding template before the next person starts.
Do not count a welcome email as a walkthrough record. Do not count a shared password as an access approval. A review that exists only in a chat message counts only when the message names the file and the reviewer. Each definition should include one positive example and one boundary example so a second reviewer can apply it consistently. Write the definitions into the onboarding template before the next hire starts.
Evidence standard and data dictionary
For each file, capture the person identifier, the role, the start date, the reviewer, the walkthrough date, the access request date, the approval date, the supervised case date, and the review date. Add whether the file contains the role brief and whether the access was removed at the end of the engagement. Record the source system for each field and whether the value was observed or inferred. Keep the file in one approved repository so the evidence can be found later.
Also capture whether the person handled live work before the supervised case and whether any access was broader than the role required. These fields help explain a gap that a simple checklist count would miss. If a document was signed after the start date, record both dates. A second reviewer should be able to rebuild the timeline from the file without asking the manager. Keep the file in one approved repository so the evidence can be found later.
Sampling and collection
Draw the sample from the onboarding tracker or the human resources system rather than from a list of recent hires the manager remembers. Record the export date, the filters, the row count, and the selection rule. Include people from every role family and every start month so the result is not dominated by one cohort. If the population is ordered by start date, a fixed interval spreads the sample across the window.
Collect the file as it existed at the end of the observation window, not as it was reconstructed later. If a missing document was added after the person started live work, record the addition date and analyze the original gap separately. Do not accept a verbal confirmation in place of a missing record. Note the collection date separately from the start date.
Analysis and reporting
Report the number of files, the share complete, the share missing each required element, and the share where live work began before the supervised case. Show the counts by role family and by reviewer. Present the median time from start date to walkthrough and from start date to review. Use the same denominator for every rate and show the number of files with incomplete evidence.
Inspect the files with the largest gaps and describe the pattern without naming the person. A cluster under one reviewer may point to a workload problem, while a cluster in one role may point to a missing template. Compare the result with the access log, because a file can look complete while the permissions were broader than recorded. Report the early departures separately.
A useful additional view is the order in which the evidence appeared. A file where access was approved before the walkthrough tells a different story from one where the review happened before the first live case. Show the sequence for the longest gaps and describe the consequence. That view helps the owner fix the process rather than asking for more documents.
Another useful measure is the time between the supervised case and the first unsupervised batch. A short gap may be appropriate for a simple queue, while a complex role may need a longer practice period. Record the batch size and the reviewer's sample rate beside the gap. The owner can then judge whether the pace matched the risk rather than the calendar.
Limitations and sources of error
The file cannot show whether the person actually understood the training or whether the reviewer was effective. It can show only whether the required evidence exists and when it was recorded. A complete file is not proof of competence, and an incomplete file is not proof that the person was untrained. The protocol also cannot capture informal coaching that left no record.
Trackers can omit people who started through a different process, and files can be stored in more than one system. A document that was completed in a meeting may not appear in the repository. Record those cases and their treatment instead of dropping them silently. If the reviewer also approved the access, note the independence limitation and analyze that subgroup separately.
Review cadence and next action
Give the result to the operations owner and the onboarding reviewer and ask them to choose one observable change. The change might be a required walkthrough date, an access expiry, or a rule that live work cannot begin without a supervised case. Record the owner, the effective date, and the measure that will show whether the change worked. Avoid changing the checklist and the access process at the same time.
Repeat the protocol with the same definitions after the change and compare the same fields. If completeness improved but the time to first live case rose, the new step may be creating a bottleneck. If access gaps fell but review gaps did not, the problem may be reviewer capacity rather than the checklist. Close the review when the owner accepts the finding and the next sample is scheduled.
