Research question and scope
Do overtime records across supervisors include a reason, an approval, and a payroll match at the same rate? The purpose is a repeatable description of the record, not a performance score. The scope is the operational record held by the team, not the legal, financial, or clinical conclusion that may follow from it.
The study is deliberately narrow. It asks what the record can show about overtime approvals in a defined population, during a defined window, using a stated evidence standard. It does not estimate an industry rate or claim that one staffing model performs better than another.
Population and unit of analysis
Overtime records in a defined quarter across sites and supervisors.
The unit is one overtime entry. Keeping the unit explicit prevents a team from counting linked events as independent cases. Record the opening count, additions, withdrawals, resolutions, and unresolved items at the end of the window so the denominator can be reconstructed.
Definitions that make the measure reproducible
Write down the start event, the stop event, the pause rule, and the treatment of missing evidence before looking at results. Define the measures as entries with a recorded reason, approvals before the shift, and payroll matches. A measure without a stated denominator or observation window is not yet a measure.
Use the same definitions for every case. If a definition changes during the review, keep the earlier classification and apply the new rule only to later cases. A protocol that quietly rewrites its own definitions cannot support a comparison.
Evidence standard and data dictionary
Each case needs the time entry, the shift need, the approval, the client authorization, and the payroll record. Record the source, the observation time, the field it supports, and the reviewer. Mark missing, unavailable, and confirmed negative values separately; they mean different things and should not be collapsed into one blank.
Use a pseudonymous case key. Keep the crosswalk between the key and the customer or worker identity in an approved system with restricted access. The analysis file should carry only the fields needed for the question.
Sampling and collection
Select cases from the frozen population using a written rule. Include ordinary cases, exceptions, and cases that were reopened or withdrawn. Add a small random sample so the review does not depend only on the items someone remembered.
Collect on a fixed cadence and preserve the source version observed at that time. A later export can overwrite a value that was present during the review. If the team must use a later export, record the difference rather than replacing the original observation.
Reviewer independence and agreement
Have two reviewers classify the same cases independently before discussing them. Compare the first classifications and group the differences by cause: missing evidence, ambiguous definition, authority question, or calculation error.
Agreement is not accuracy. Two reviewers can agree on the same incomplete file. Follow the agreement check with a completeness check on the items both reviewers marked as supported. A disagreement is useful evidence about the record, not a personal failure.
What the evidence can support
The evidence can show whether the required fields are present, whether the source is current, whether an owner is named, and where the record is thin. It can show how a queue behaves in the reviewed window. Those findings can support a supervisor checklist, a client billing review, and a payroll audit sample.
It cannot show that a customer will receive a particular outcome, that a regulator will agree, or that a change in staffing caused a result. The protocol does not decide whether the overtime was legally required or billable. Keep those limits in the finding rather than in a closing footnote.
Limitations and sources of error
An approval may be given verbally and recorded later.
A client may authorize overtime under a different rule. A supervisor may approve overtime for work that was not actually performed. A single window may not represent another month, and a convenience sample may not represent the queue. Report the window, the volume, the mix, and the exclusions beside the result.
Sensitivity and alternative explanations
Repeat the main count under at least two reasonable alternatives: a different pause rule and a different treatment of unresolved cases. If the direction changes, the finding is fragile and should be reported as such.
Name the plausible alternatives instead of choosing the most convenient one. A busier period, a system migration, a changed approval rule, or a different client mix can all explain a shift without any change in the quality of the work.
Privacy, access, and retention
Collect the minimum fields required. Use named accounts, role-based access, masked identifiers, and an approved working location. Log who accessed the file and set a disposal date before the review begins.
DOL and IRS guidance frames the time and payroll record question. The cited public sources frame records management, internal control, privacy, and security practice. They do not provide a result for this sample and do not replace legal or compliance review.
Operational interpretation and reporting
Publish the case flow, the definitions, the counts, the unresolved items, the disagreement summary, and the limits together. A reader should be able to tell which statements are observations, which are calculations, and which are owner judgments.
Use the result to test one change, such as a required source field or a named owner at handoff. Recheck the same measure on a later window. If the change does not move the measure, the likely cause is upstream in the source or the rule, not in the reporting.
Effort and practical constraints
A review of this kind fits in a short cycle when the files are accessible: roughly one day to freeze the population and definitions, one to pull and mask the sample, and two to review and reconcile. The expensive step is usually locating evidence across systems, not the analysis.
If the review takes much longer, treat the record structure as the first finding. A process that cannot be sampled without reconstructing private email is a control problem in its own right. Keep the scope small enough to finish, and write down what was excluded so the next reviewer can extend it deliberately.
A calibration case
Take one ordinary case, one exception, one withdrawn case, and one unresolved end-of-window case. Two reviewers independently reconstruct the time entry, the shift need, and the approval and record their classifications before discussion. The protocol owner compares the first judgments and records why any difference arose.
The calibration is not a search for consensus. It tests whether the evidence standard can be applied the same way twice. If the reviewers need different documents to reach the same code, the record is ambiguous and the data dictionary should be revised before the main review begins.
Reporting package and reopening conditions
Publish the case flow, the definitions, entries with a recorded reason, approvals before the shift, and payroll matches, the unresolved count, the disagreement summary, and the limitations together. A reader should be able to see which numbers are observations, which are calculations, and which are owner judgments.
Record what would reopen the conclusion: a system change, a new policy, a different client or patient mix, or a material shift in volume. Name the next review date and the owner who will decide whether the finding still applies. A protocol without a reopening condition becomes an old opinion dressed as data.
Interpreting a small sample
A small review is useful for finding where the record breaks, not for ranking teams or estimating a population rate. Report the number of cases, the number of fields checked, and the number of unresolved items. A count of five missing fields in twenty cases is easier to act on than a percentage with no denominator.
If the sample is too small to support a comparison, say so and describe what a larger review would need. The next step can still be a targeted fix. It should not be a general claim about the quality of the team, the vendor, or the client relationship.
Governance and stopping conditions
Name the protocol owner, the data custodian, the operational owner, and the reviewer who can pause the study. Pause on unauthorised disclosure, a source that cannot be verified, or a request to change the sample after results are known.
Record what would reopen the conclusion: a system change, a new policy, a different client mix, or a material shift in volume. The brief is complete when the next reviewer can repeat the method without asking the original author what was meant.