Resolving screening conflicts is the process of settling the records where two independent reviewers disagreed on inclusion, using either discussion to consensus or adjudication by a third reviewer. Every conflict must be resolved on its merits and never dropped, because a disagreement is a signal that one reviewer caught something the other missed, or that the eligibility criteria are not yet sharp enough.
Why conflicts are useful, not a problem
Disagreement between two careful reviewers is exactly what double screening is designed to surface. A conflict means the record sat on a boundary that a single reviewer would have crossed in one direction without anyone noticing. Treating conflicts as information rather than nuisance changes how you handle them: a cluster of disagreements around one criterion tells you that criterion is ambiguous, and fixing the rule prevents dozens of future conflicts. This is why screening teams track their disagreement rate alongside their inter-rater reliability in screening.
The two resolution methods
Discussion to consensus
The first method is for the two original reviewers to discuss each conflicted record and reach agreement. This works well when the disagreement is a simple oversight, one reviewer skimmed past a detail in the abstract, and it has the benefit of recalibrating both reviewers for the records still ahead. The risk is that the more senior or more confident reviewer dominates, so consensus should be reached on the evidence in the record, not on seniority.
Adjudication by a third reviewer
The second method sends each conflict to a third reviewer who decides. This is faster on large reviews and removes the power imbalance of a two-person discussion, which is part of why the standard answer to how many reviewers screening needs includes a third for exactly this role. Keeping adjudication with one consistent person stops the resolution itself from drifting over the course of the review. The third reviewer should adjudicate blind to who voted which way wherever the tool allows it, so the decision turns on the record against the criteria rather than on which colleague to side with. Most teams reserve this role for the member with the firmest grasp of the eligibility rules, often the methodologist who drafted them.
Choosing between the two methods
A simple decision rule keeps the choice consistent. Use discussion to consensus early in a review, when the disagreement count is small and recalibrating both reviewers pays off for the records still ahead. Switch to third-reviewer adjudication once volume climbs, when sitting two people down to debate hundreds of conflicts is slower than routing each to a single decider, and whenever a hierarchy between the reviewers risks the more junior one deferring. Many large reviews run both: consensus during the pilot and early batches to tighten the rules, then adjudication for the bulk of the screen. Whichever you pick, name the method in your pre-specified review plan so the approach is not chosen retrospectively to suit a particular study.
Resolving conflicts at each stage
The stakes differ by stage. At title and abstract screening a conflict usually resolves toward inclusion, because the safe move at a lenient stage is to pass a borderline record on to full text. At full-text screening the decision is final, so conflicts there demand more care and the resolution must also fix the single recorded reason for exclusion when a record is finally excluded. Conflicts are not unique to screening either; the same process applies when two reviewers disagree during data extraction.
Recording and reporting resolutions
A conflict is only resolved once the decision and how it was reached are recorded. Working inside a dedicated systematic review screening tool captures this automatically: who voted what, which method settled it, and the final outcome. That trail matters because the resolved decisions feed the counts in your PRISMA flow diagram, and a reviewer who asks how a particular study was handled should find a clear answer rather than an undocumented override.
A repeatable conflict-resolution workflow
Resolving disagreements is faster and more defensible when it follows a fixed sequence rather than ad hoc judgement:
- Let both reviewers finish the batch independently, so no conflict is settled while screening is still anchoring one reviewer to the other.
- Generate the conflict list and sort it, because a pattern, for example a run of disagreements on one population definition, points to a rule to fix rather than records to argue over.
- Resolve each conflict by the pre-agreed method, recording who decided and on what basis, never by simply deleting the disputed record.
- At full text, attach the single reason for exclusion to any record finally excluded, using the same fixed hierarchy both reviewers applied.
- If one criterion drove a cluster of conflicts, amend its wording and note the change, so the same ambiguity does not regenerate disagreements in later batches.
Following the same five steps every batch is what lets a team report a clean disagreement rate and demonstrate that no conflicted record was ever quietly lost, which is precisely what an auditor of the selection process wants to confirm.
When conflicts stay high
A persistently high conflict rate is a warning, not something to push through. It almost always traces back to eligibility criteria that are vague or that drifted from the original protocol. The right response is to pause, re-pilot the criteria on a fresh sample, and only then continue. Resolving conflicts one by one without fixing the underlying rule treats the symptom while the cause keeps generating new disagreements. As a rough working signal, a pilot disagreement rate that leaves chance-corrected agreement below the substantial band, broadly a kappa under 0.60, is the moment to rewrite before screening the full set.
Mistakes that make conflicts harder to resolve
Several habits turn a manageable disagreement list into a stalled review. Discussing conflicts mid-screen, before both reviewers have finished a batch, contaminates the remaining independent votes. Letting seniority decide rather than the evidence in the record introduces exactly the bias double screening exists to remove. Resolving silently, by overwriting one vote without a record of why, breaks the audit trail. And changing criteria informally to settle a single awkward case, instead of amending the rule for everyone, creates inconsistency that surfaces later as irreconcilable counts. If reviewing already screened records means an eligibility rule must change, treat it as a formal change to the registered protocol and rescreen anything the change touches.