JBI critical appraisal is the system of design-specific checklists published by the Joanna Briggs Institute for judging the trustworthiness, relevance, and results of published studies. There is a separate checklist for each study design, from randomised trials to case reports and prevalence studies, each item answered as yes, no, unclear, or not applicable, feeding an explicit overall decision to include, exclude, or seek further information about a study.

Why JBI built a checklist for every design

The Joanna Briggs Institute, based at the University of Adelaide, has supported evidence synthesis for nearly three decades, and its appraisal suite reflects a founding conviction: evidence for practice comes in many forms, so appraisal must too. Where Cochrane instruments concentrate on intervention effects, JBI reviews routinely synthesise prevalence estimates, qualitative findings, case reports, and expert opinion, designs that domain-based bias tools simply do not cover. The result is the broadest single family among the major critical appraisal tools: whatever design your review admits, there is almost certainly a JBI checklist written for it, using the same response format and the same overall decision logic throughout. That consistency is the practical selling point. A mixed-design review can appraise every included study inside one coherent system instead of stitching together instruments with incompatible outputs.

There is a second, less obvious advantage. Because every checklist ends in the same three-way decision, include, exclude, or seek further information, the appraisal stage produces a uniform output that can be tabulated across the review regardless of design. A reader can scan one table and see how the qualitative studies fared next to the cohort studies, something impossible when a star-based scale sits beside a domain-based algorithm. For reviews in nursing, allied health, and public health, where evidence bases are rarely tidy, that uniformity is why the JBI suite has become the default rather than the fallback. The checklists are also freely available with published guidance for every design, and they are built into JBI’s own synthesis software, which lowers the cost of doing the appraisal properly rather than approximately.

The checklists by study design

The suite covers the full spread of quantitative, qualitative, and text-based evidence:

  • Randomised controlled trials: thirteen items on randomisation, allocation concealment, blinding, follow-up, and analysis.
  • Quasi-experimental studies: non-randomised intervention studies, probing comparability and temporal precedence.
  • Cohort studies: recruitment from the same population, exposure measurement, confounding, and follow-up.
  • Case-control studies: matching, exposure ascertainment, and identical measurement across groups.
  • Analytical cross-sectional studies: inclusion criteria, valid exposure and outcome measurement, and confounder handling.
  • Prevalence studies: sampling frame, sample size, coverage, and valid condition measurement.
  • Case reports and case series: clear patient characteristics, condition measurement, and consecutive, complete inclusion for series.
  • Qualitative research: congruity between philosophy, methodology, methods, and interpretation, plus researcher reflexivity.
  • Text and opinion papers: the source of the opinion, its standing in the field, and the logic of the argument.

There are also checklists for systematic reviews, economic evaluations, and diagnostic accuracy, though for those designs many teams prefer AMSTAR 2, ROBIS, or QUADAS-2 respectively. The practical difficulty is not the number of checklists but classifying each study correctly before appraisal. A single-arm study reporting outcomes over time might be a case series or a quasi-experimental study depending on whether an intervention effect is claimed; a survey might be a prevalence study or an analytical cross-sectional study depending on whether it tests associations. Because each classification pulls a different checklist, the review protocol should define the classification rules in advance, and ambiguous studies should be classified by two reviewers before anyone opens a checklist. When a study genuinely straddles two designs, appraise it with the checklist matching the analysis your synthesis will actually use, and note the alternative classification in the appraisal record so the decision is visible rather than silent.

The 2023-2024 revisions

JBI has been revising the suite since 2023, and the updated checklists matter for anyone citing the tools today. The revised instruments for randomised trials, quasi-experimental studies, cohort, case-control, and cross-sectional designs reorganise items to align more closely with modern risk of bias thinking, separating internal validity from reporting, clarifying items that conflated two questions, and tightening guidance on how each item should be judged. The quasi-experimental checklist, for instance, grew clearer about what counts as a control and about temporal ordering between intervention and outcome, while the analytical cross-sectional revision sharpened the distinction between how variables were measured and how confounding was handled in the analysis.

The practical consequences are two. First, always state which version of a checklist you used, because item numbering and wording differ between the pre-revision and revised forms, and a reader comparing your table against the wrong version will conclude you answered questions that do not exist. Second, do not mix versions across studies within one review; pick the current form at protocol stage and hold it fixed, even if a new revision lands mid-project. A version switch halfway through an appraisal is a methods change, and methods changes belong in protocols, not in the middle of data collection.

Scoring: item responses and the overall appraisal decision

Every item is answered yes, no, unclear (the paper does not report enough to judge), or not applicable. What JBI does not prescribe is a cut-off. Some teams compute the proportion of “yes” answers, but JBI’s own guidance warns against fixed thresholds because the items are not equally important: a prevalence study with a biased sampling frame is compromised no matter how many other items it passes. Instead, appraisers record an overall appraisal decision for each study: include, exclude, or seek further information, and the review protocol should state in advance which weaknesses will drive exclusion. Two reviewers appraise independently and reconcile, exactly as they would during full-text screening decisions, and disagreement rates are worth reporting alongside the reliability statistics used for screening. Where a team lacks a second appraiser, our second reviewer service supplies the independent judgement the method requires.

Running a JBI appraisal step by step

A defensible JBI appraisal in a systematic review follows a fixed sequence:

  1. Classify every included study by design using pre-agreed rules, and assign the matching checklist and version.
  2. Pre-specify in the protocol how the appraisal will be used: purely descriptive, feeding exclusion above a stated set of failed items, or driving a planned sensitivity analysis.
  3. Have two appraisers work independently, answering every item with a citation to the page, table, or supplementary file that supports the answer, reserving unclear for genuine reporting gaps.
  4. Record the overall appraisal decision per study, reconcile disagreements by discussion or a third reviewer, and keep both original answer sets.
  5. Tabulate per-item results for the manuscript and carry the judgements into the synthesis and discussion.

The step most often skipped is the second. Deciding after the appraisal what the appraisal will mean invites exactly the results-driven reasoning the tools exist to prevent.

JBI versus the Cochrane tools

The two families answer different questions. Cochrane’s instruments, such as RoB 2 for randomised trials and ROBINS-I for non-randomised intervention studies, are strict risk of bias tools: they route signalling questions through an algorithm to a domain-level judgement, per outcome, built for GRADE. JBI checklists are broader methodological quality instruments: they cover bias but also conduct and reporting, they judge at study level, and they end in an inclusion decision rather than a bias rating, a contrast unpacked in risk of bias versus quality assessment. A workable rule: for an intervention review aimed at a Cochrane-style journal, use RoB 2 and ROBINS-I; for reviews of prevalence, aetiology, case series, qualitative evidence, or mixed designs, the JBI suite is often the only instrument that fits, and for observational aetiology reviews it competes directly with the star-based Newcastle-Ottawa Scale, usually winning on transparency because every item is reported rather than compressed into stars.

Prevalence studies and proportion meta-analyses

The prevalence checklist deserves its own mention because nothing else like it exists in the other families. Reviews that pool proportions, the prevalence of a condition, the frequency of a complication, the uptake of an intervention, live and die by sampling. The nine items interrogate exactly that: was the sampling frame appropriate to the target population, were participants recruited in a way that avoids selection bias, was the sample size adequate, were the subjects and setting described, was coverage of the identified sample sufficient, were valid methods used to identify the condition, was it measured consistently, was the analysis appropriate, and was the response rate adequate or its shortfall managed. A study failing the frame or coverage items can distort a pooled proportion far more than random error will, which is why these judgements should be carried straight into the synthesis rather than parked in a table. Reviews that force prevalence studies through a generic checklist leave readers no way to tell a population-based estimate from a convenience-sample artefact.

Feeding appraisal into inclusion and sensitivity analysis

JBI appraisal is designed to have consequences. The first is the inclusion decision itself, taken against thresholds set in the protocol. The second is analytic: appraisal results should drive a planned sensitivity analysis in which the pooled estimate is recomputed without the studies that failed key items, to test whether the conclusions rest on weak evidence, the same logic explored in sensitivity analysis in meta-analysis. Reporting follows the familiar pattern: a methods statement naming the checklist, version, number of independent appraisers, and reconciliation process; a per-item results table for every study; and a discussion that says plainly how methodological weakness shaped the findings. Do that, and the appraisal stops being a formality and becomes part of the argument your review makes.

Common mistakes with the JBI checklists

Four errors recur across submitted reviews. The first is the wrong-checklist problem: appraising a prevalence survey with the cross-sectional checklist, or a case series with the cohort checklist, which produces answers to questions the study never set out to address. The second is converting responses into a percentage and imposing an unpublished cut-off such as “seventy per cent equals good quality”; reviewers who do this are inventing a threshold the tool’s developers explicitly declined to give, and peer reviewers increasingly say so. The third is treating unclear as a diplomatic middle ground rather than what it is, a reporting failure that should trigger an attempt to contact authors or a documented note that the information was unavailable. The fourth is single-reviewer appraisal, which no appraisal instrument survives: item wording always leaves interpretive room, and independence plus reconciliation is what turns individual readings into a review-level judgement. Avoid those four, report per item, and the JBI suite will do what it was designed for: make the quality of a messy, multi-design evidence base legible in one consistent frame.