Two interview rounds can sound similar while answering different questions. In the first round, participants may be describing an unfamiliar process. In the second, they may be reacting after trying a revised version. If every mention is merged into one theme count, the comparison loses the context that makes it useful.
A good comparison keeps each observation connected to its source and asks a narrower question: what repeated, what changed, what appeared for the first time, and what still cannot be concluded?
Start with one comparison question
Do not begin by comparing every sentence in both note sets. Write one question that defines the decision the comparison should support.
Comparison question:
After the revised sign-up instructions were introduced, which points of confusion repeated, disappeared, or appeared for the first time?This question identifies the intervention, the evidence to compare, and the outcomes of interest. It does not assume that the revised instructions caused any change.
Confirm that the two rounds are comparable
Before building a matrix, record the main differences between the rounds:
- who participated and how they were recruited;
- which questions or tasks were used;
- when and where the interviews happened;
- what changed between rounds;
- which notes, transcripts, or recordings are available;
- which material the team is permitted to analyze.
If the participant mix or interview method changed, the rounds can still be compared, but the difference belongs in the interpretation. A new pattern may reflect the new sample rather than a change in the experience.
Keep a source key
Assign every permitted excerpt a stable source key. A simple key can show the round, participant label, and note location without exposing personal details in the synthesis.
R1-P03-12: round 1, participant 03, note line 12
R2-P02-08: round 2, participant 02, note line 8The key lets another authorized reviewer return to the original context. It also prevents a paraphrase from becoming detached from the words or observation that supported it.
Build a comparison matrix
Use one row per research question or clearly defined observation. Keep evidence and interpretation in separate columns.
| Observation | Round 1 evidence | Round 2 evidence | Comparison | Limit or open question |
|---|---|---|---|---|
| Where to begin | R1-P01-05, R1-P04-09 | R2-P03-06 | Repeated, but fewer observed examples | Samples are small and not matched |
| Meaning of the confirmation message | No supporting excerpt found | R2-P02-08, R2-P05-11 | New in round 2 | Could be prompted by the revised wording |
| Need for a printable copy | R1-P02-14 | R2-P04-10 contradicts the earlier request | Mixed evidence | Different work settings may explain the difference |
The matrix shows where evidence exists. It does not convert a few mentions into a population estimate.
Use four comparison labels
Four labels are usually enough for a first pass:
- Repeated: evidence supporting the same observation appears in both rounds.
- New: supporting evidence appears only in the later round.
- Not observed again: evidence appears in the earlier round but not in the later material reviewed.
- Mixed or contradictory: excerpts point in different directions within or across rounds.
“Not observed again” is deliberately narrower than “resolved.” Absence from a small set of notes is not proof that the issue disappeared.
A fictional two-round example
Imagine a fictional team testing instructions for a community workshop sign-up. Round 1 contains four interviews conducted before the wording was revised. Round 2 contains five interviews conducted after the revision. The participants are different people, so the team cannot treat the rounds as a matched before-and-after test.
In round 1, two participants ask where to begin. In round 2, one participant pauses at the same point. That supports a repeated observation with fewer examples in the later notes. It does not prove the revision reduced the problem.
Two round-2 participants interpret the confirmation message as final approval, an issue that was not recorded in round 1. The team labels this as new evidence and checks whether the revised wording introduced ambiguity.
One round-1 participant asks for a printable copy. A round-2 participant says a printed version would become outdated too quickly. The team keeps both excerpts and labels the evidence mixed. Combining them into “participants want print support” would erase a useful disagreement.
Do not treat mention counts as prevalence
Counts can describe the notes reviewed: two supporting excerpts in round 1 and one in round 2. They do not establish how common an experience is in the wider audience, especially when the sample, questions, or interview conditions differ.
Use language such as “appeared in two of the four round-1 interviews reviewed.” Avoid “half of users” unless the study design supports that inference.
Preserve counterexamples
A counterexample can show where a proposed explanation fails. If several participants ask for more guidance but one completes the task easily, inspect the difference. The counterexample may reveal prior experience, a different device, a clearer entry point, or a note-quality issue.
Do not hide the counterexample in a footnote. Put it beside the main observation so readers can judge the strength and boundary of the interpretation.
Separate evidence from interpretation
Evidence is what the permitted source records. Interpretation is the team's explanation of what that evidence may mean. Write them in different fields.
Evidence: Two round-2 participants asked whether the confirmation message meant they were fully registered.
Interpretation: The revised confirmation wording may not distinguish receipt from approval.
Next check: Test a message that names both states explicitly.The word “may” matters. It keeps the explanation provisional until the next check.
Turn the comparison into follow-up work
Each important row should lead to one of four actions: keep monitoring, revise and test, collect missing evidence, or make no change because the evidence is too weak. If the comparison reveals uncertainty rather than a conclusion, turn it into source-linked follow-up questions.
If several excerpts point to the same underlying difficulty, a carefully bounded problem statement from interview notes can help the team decide what to investigate next.
Copy this comparison template
Comparison question:
Round 1 context and source set:
Round 2 context and source set:
Important method or sample differences:
Observation:
Round 1 source keys:
Round 2 source keys:
Label: repeated / new / not observed again / mixed
Interpretation:
Counterexample:
Limit or open question:
Next check and owner:Run a final evidence check
Before sharing the comparison, reopen every cited excerpt. Check that the wording, context, round, and label are accurate. Confirm that a missing mention has not been rewritten as resolution, a count has not been presented as prevalence, and disagreement has not been merged into a convenient average.
Choose one narrow question from two rounds of notes you are authorized to analyze. Add one source-linked excerpt from each round, label the comparison, preserve one counterexample or gap, and write the next check. If the rounds differ in a way that could explain the result, place that limitation beside the conclusion.
