Research question and scope
This protocol asks whether a focused knowledge base audit reduces repeat questions in a delegated support team. The unit is one question event, identified by the asker, the topic, the channel, and the time it was asked. A repeat question is a question on a topic that the knowledge base already covers. The study measures repeat questions and search time, not the number of pages edited.
Set the scope to one client, one team, and one knowledge base for a fixed baseline and follow-up window. Record the search tool, the page inventory, and the guide version at the start. If the tool or the page structure changes during the study, keep the earlier definition for the full baseline or analyze the change separately. A new search tool can change the result without changing the content.
Unit and population
Define the population as every question asked by a specialist during the window, including questions sent to a teammate, a manager, or a chat channel. Exclude purely social messages and record the exclusion count. Include questions that were answered by a saved reply if the person still had to ask where the reply lived. Test the unit definition on ten question events before collection begins.
A single conversation can contain several questions on different topics. Split the conversation into question events by topic rather than by message. Record the topic using a short controlled list so the baseline and follow-up can be compared. If a topic does not fit the list, add it with a written definition rather than forcing it into another category.
Defining a repeat question
Define a repeat question as one whose answer exists in a live knowledge base page at the time it is asked. Define a new question as one with no matching page or with a page that the asker could not find. The distinction matters because a new question may need content, while a repeat question needs findability or trust. Publish one positive and one boundary example for each.
Do not count a question as a repeat just because the topic is familiar to an experienced person. Check whether the page existed, whether it was live, and whether it answered the specific question. If the page was wrong, the question is a content defect rather than a repeat. A second reviewer should be able to classify the question from the page snapshot.
Evidence standard
For each question event, capture the asker, topic, channel, time, the page that should have answered it, and how the answer was found. Add the search terms used, if any, and whether the page appeared in the first three results. Store the page snapshot with the event so a later edit does not change the evidence. A question without the page snapshot cannot be classified reliably.
Record whether the asker had used the knowledge base before and whether they had received training on the topic. Record the answer source, such as a page, a teammate, or a manager. If the answer came from a teammate, note whether the teammate pointed to the page. A reviewer should be able to follow the path from question to answer without interviewing the asker.
Baseline measurement
Collect two to four weeks of question events before the audit. Use the same channels and the same classification rules throughout. Report the number of questions, the repeat share, the median search time, and the share answered by a page. Keep the raw events with the summary so the follow-up can use the same definitions.
Do not announce a target during the baseline. A target can change how people report questions, especially if they think a repeat question reflects badly on them. Ask for the raw event, not a judgment. The baseline should describe the current work, not a performance review. Explain why the baseline is being collected so the team reports events rather than impressions.
The audit intervention
Limit the audit to a defined set of actions: fix the top conflicting pages, add missing search aliases, assign owners, and retire dead pages. Do not rewrite the entire knowledge base during the study. Record each action with the page, the change, the owner, and the date. A focused intervention is easier to connect to a change in the question rate.
Keep the audit separate from any change to the queue rules or the product. If the product changes during the window, note the date and the affected topics. A product change can create new questions that look like a failed audit. The analysis should separate questions about changed topics from questions about stable ones.
Sampling and comparison
Use the same sampling rule for the baseline and the follow-up. If the baseline uses every question in a two-week window, the follow-up should use every question in a comparable window. If the volume is too high, sample by topic and day rather than by convenience. Record the export date and the filters for both periods.
Compare the same topics and the same channels. A drop in repeat questions on one channel may mean the team moved the conversation elsewhere. A rise in new questions may mean the audit surfaced gaps that were previously hidden. Report both repeat and new questions so the reader can see the whole change.
Analysis and reporting
Report the repeat share, the median search time, and the share answered by a page for both periods. Show counts beside every percentage and state the exclusions. Break the result down by topic so the owner can see where the audit helped and where it did not. A single overall number can hide a worsening topic.
Compare the change with the audit actions and the product changes. If repeat questions fell only on the pages that were edited, the audit is a plausible cause. If they fell everywhere, the team may have changed its reporting behavior. Do not present a difference as causation without a controlled comparison.
Limitations and sources of error
The record cannot show questions that were never asked because the person gave up or guessed. It also cannot show the time spent reading a page that was correct but hard to understand. A team may ask fewer questions because it is busier, not because the knowledge base improved. The study measures recorded questions and recorded search time.
Pages can be edited, moved, or deleted during the window, and a snapshot may not capture the version the asker saw. Chat retention may be shorter than the study period. Record those cases and their treatment rather than dropping them. If the person who ran the audit also classified the questions, note the independence limitation.
Ownership and review cadence
Give every audited page an owner and a review date. The owner confirms the content, not the formatting. Group pages by owner so the review load is visible. A page without an owner will drift again, and the next audit will repeat the same work. The ownership record is part of the intervention, not an administrative extra.
Schedule a light review each quarter and a focused review after any policy or product change. Review the question log at the same time to see which topics are returning. If the same page keeps generating questions, the problem may be findability rather than content. Change the title, the aliases, or the entry point before rewriting the page again.
Reporting to the client
Report the result in a form the client can act on: which topics improved, which pages still cause questions, and which searches still fail. Avoid presenting the numbers as a grade for the team. Use examples with identifying details removed. The client should leave with one content change and one ownership change it can approve.
Show the denominator and the exclusions so the client can judge the sample. If the sample is too small to support a conclusion, say so. A cautious report is more useful than a confident one built on a few days. Offer to repeat the measurement after the next review and name the date.
Replication and version control
Keep the definitions, the page inventory, and the export query with the report. A second analyst should be able to repeat the measurement on a later window and compare the same topics. If the definition changes, publish a new baseline rather than mixing periods. Replication is what turns a one-time audit into a learning loop.
Schedule the next measurement before the first report is filed. The date should be far enough away for the content to settle but close enough to keep attention. Assign the owner who will run it. A protocol that is never repeated becomes a story about the past. Put the date in the same calendar as the content review so it is not forgotten.
