Research question and scope
This protocol measures how often the same reviewer handles consecutive peer reviews in a small quality team. The unit is one review assignment, not one article or one reviewer. A repeat means the same reviewer reviewed two assignments in a row for the same queue, regardless of the writer. The measure describes rotation discipline, not reviewer competence.
Set the scope to one queue, one review type, and a fixed period. Record the rotation rule and the declared conflict list in force. If the rule changes during the period, split the sample at the effective date. A different rule creates a different denominator.
Keep the measure separate from a correction rate. A reviewer can repeat assignments and still catch every issue, or rotate perfectly and miss the same error in each case. The purpose is to test independence and coverage. Report correction quality separately.
Name the decision the result will support before collection starts. The client may use it to add a backup reviewer, change the rotation order, or accept a smaller team. A measure that cannot change the roster or the rule is only a curiosity. Write the intended action beside the scope.
Record the review type and the risk level for each queue before sampling. A queue of routine acknowledgments may not need the same rotation as a queue of money-related replies. Record the risk level beside the queue name. The same repeat rate means different things at different risk levels.
Population and unit of analysis
Define the population as every peer review assignment completed during the period. Include reviews of customer replies, internal records, and sample audits. The unit is the assignment, not the reviewer or the ticket. A ticket reviewed twice by two people is two assignments.
Include assignments where the reviewer declared a conflict and was replaced. Mark the replacement and the original assignment separately. Exclude a review that was cancelled before any reading began. Report the excluded count and the reason. A cancelled assignment is not evidence of rotation.
Exclude a second review of the same item caused by an appeal. Record the appeal separately and link it to the original. An appeal is a quality escalation, not a routine rotation event. State the exclusion rule before sampling so a second reviewer applies it the same way.
Decide how to treat a reviewer who is the only person trained for a task type. Count the assignment and mark it as a sole-reviewer case. Report those cases separately rather than hiding them inside the repeat rate. The sole-reviewer group shows where the team has no real rotation.
Include reviews done by a team lead or a quality manager in the assignment count. A manager who reviews every hard case is part of the rotation pattern. Record the reviewer role beside the name. Leaving leaders out hides the concentration at the top of the team.
Definitions that make the measure reproducible
Define a review assignment as a named reviewer, a named writer, a review type, a start time, and a decision. Define a consecutive repeat as two assignments for the same queue with no different reviewer between them. Define a declared conflict as a written note that names the relationship or prior involvement. Publish one example of each.
Define the review queue order by the assignment time, not by the ticket creation time. A reviewer can pick up an old ticket today and still be repeating the last reviewer. Record the assignment time and the ticket age separately. The two orderings answer different questions.
Define independence as no self-review and no review of work the reviewer helped write. A reviewer may still be independent when they share a shift with the writer. Record the specific conflict rather than a general concern. Independence should be judged on evidence, not on discomfort.
Decide how to classify a reviewer who reviews two assignments in a row for different queues. Count the repeat only within the same queue and review type. A small team may legitimately rotate across unrelated work. Report the cross-queue pattern separately if the client wants it.
Agree whether a reviewer who declines an assignment counts in the order. A decline can break a run without any review taking place. Record the decline and the reason in the log. Treating a decline as a review would understate the repeat pattern.
Sampling and data collection
Draw the sample from the review log and the assignment queue for the full period. Record the export date, the filters, and the row count. Do not build the sample from the reviewers' own notes. A self-reported log tends to omit the hurried assignments.
Capture the assignment identifier, the reviewer, the writer, the queue, the review type, the assignment time, the decision, and any conflict note. Add the ticket age and the reviewer's other assignments that day. Store the export with the report. The reviewer and assignment time are the core evidence.
Check a small overlap sample with a second reviewer. If the reviewers disagree about whether two assignments are consecutive, the ordering rule is ambiguous. Rewrite the rule and retest before the full sample. Agreement matters more than sample size.
Record the time spent matching the review log to the assignment queue. A report that requires reading every review note will not be repeated. Note the fields the review tool does not capture. The collection cost belongs beside the result.
Collect the reviewer's own assignment list as well as the queue log. A reviewer may pick up work that was never formally assigned through the queue. Record the source of each assignment and the overlap. The two sources together give a more complete picture of concentration.
Measuring reviewer concentration
Report the share of assignments that follow the same reviewer, a different reviewer, and a sole-reviewer case. Show the counts behind each share. The three groups describe different levels of rotation and should not be collapsed. A single repeat rate hides the queue where no backup exists.
Break the result down by queue, review type, and shift. A night shift may show more repeats because fewer reviewers are online. A complex queue may show repeats because only one person is trained. Compare only groups with a similar volume and skill base.
Report the longest run of consecutive assignments for one reviewer. A single long run is different from scattered repeats across the team. Include the run length and the queue. The distribution shows whether the issue is a roster gap or a scheduling habit.
Report the share of assignments with a declared conflict and the share replaced after the declaration. A low declaration rate may mean the conflict rule is not understood. A high replacement rate may mean the roster is too small. Both numbers belong beside the rotation result.
Report the share of assignments that were reviewed by someone outside the queue's trained group. An outsider may be a manager helping out or a specialist borrowed from another team. Record the role and the reason. The pattern shows whether the rotation depends on informal borrowing.
Limitations and next action
The measure cannot show whether a reviewer was biased or whether a different reviewer would have caught more. It also cannot show a review performed informally in chat. Treat the result as a measure of recorded assignments. Note the queues and review types that were not included.
Give the result to the quality lead and the operations owner with one change to test. The change might be a named backup, a rotation order, or a conflict prompt in the review form. Record the effective date and repeat the measure with the same definitions. A controlled change is easier to judge.
Review the boundary cases before forcing a strict rotation. A few specialist queues may explain a large share of sole-reviewer cases. Train a second reviewer first if that is the pattern. The goal is independent coverage, not a lower repeat number by definition.
Set the next review date and the person who will run it. Include the same queues and review types so the comparison is fair. Record any change to the team or the tool in the note. A follow-up with a different scope cannot show whether the rotation improved.
