Qualitative studies

More than 100 critical appraisal tools for qualitative studies have been developed. However, still few have been validated.

Some reviews on critical appraisal tools for qualitative studies:

Checklist from Boejie et al (2011)

Study designsQualitative studies
Number of items10
Rating0 (item is absent), 1 (item is dealt with but weak), and 2 (item is satisfactorily dealt with).
ValidityThis tool was developed from existing critical appraisal tools for qualitative studies. Two reviewers rated 17 articles using expert judgement (1 to 10) and the tool. The correlation between the overall expert judgement and the checklist appraisal was 0.867.
ReliabilityTwo reviewers appraised 17 articles. The ICC was 0.94 .
Other information
Main referencesBoeije, H. R., van Wesel, F., & Alisic, E. (2011). Making a difference: towards a method for weighing the evidence in a qualitative synthesis. Journal of evaluation in clinical practice, 17(4), 657-663.

Checklist from Vermeire et al (2002)

Study designsFocus group
Number of items13
Ratingyes, no, can’t tell
ValidityTool developed based upon a review of the literature and expert panel discussions among primary care researchers experienced in focus group research (the six authors). To test the feasibility of using the checklist, primary care researchers not experienced in focus group research were asked to use the draft checklist to assess five focus group research articles randomly chosen among a sample of 70 papers. The median time to appraise a study using the pilot version was 30 minutes (with mean of 68 minutes). The applicability varied among the studies. The rating scale was considered too rigid.
ReliabilityThree papers were appraised by 9 primary care researchers and there was 68% agreement, 16% contradiction and 16% of discrepency because of doubt.
Other information
Main referencesVermeire, E., Van Royen, P., Griffiths, F., Coenen, S., Peremans, L., & Hendrickx, K. (2002). The critical appraisal of focus group research articles. European Journal of General Practice, 8(3), 104-108.

Checklist from Reis et al (2007)

Study designsQualitative studies
Number of items15
Rating0 (unable to rate); 1 (low/barely); 2 (moderate/moderately); 3 (high/clearly)
ValidityTool developed by research group base on existing literature. Pilot tested by analysing a set of qualitative papers and refined the tool until content, clarity and wording was satisfactory.
ReliabilityThe interrater reliability was tested for the two pairs of investigators(ranged from 0.22 to 0.77) and for one investigator compared to the other three (ranged from 0.38 to 0.60). Agreement on the item of global rating ranged from 0.36 to 1.00.
Other information
Main referencesReis, S., Hermoni, D., Van-Raalte, R., Dahan, R., & Borkan, J. M. (2007). Aggregation of qualitative studies—From theory to practice: Patient priorities and family medicine/general practice evaluations. Patient Education and Counseling, 65(2), 214-222.