Ensure your ML-assisted screening meets publication standards
Essential steps for PRISMA compliance and research integrity
These validation guidelines are provided for informational purposes only and are not a substitute for professional research methodology consultation. Users are responsible for ensuring their research meets all necessary institutional, ethical, and publication requirements. Always consult with experienced systematic review methodologists and follow your institution's research guidelines.
Dual screen 10-20% sample
Two independent reviewers
Calculate kappa (κ ≥ 0.61)
Inter-rater agreement
Review all inclusions
Manual verification required
Document in methods
PRISMA compliance
If κ < 0.61: Expand validation sample to 30-50%, refine screening criteria, provide additional reviewer training, consider full human screening
Title/Abstract Screening: Initial screening was performed using machine learning assistance (Litry ML Screening Tool), followed by validation screening of [X]% (n=[Y]) randomly selected abstracts by two independent reviewers (Author1, Author2).
Agreement: Inter-rater agreement was assessed using Cohen's kappa (κ = [value], [interpretation]). Disagreements were resolved through discussion [or third reviewer].
Quality Assurance: All ML-included studies (n=[Z]) were manually reviewed for accuracy. Additionally, [X]% of excluded studies were spot-checked for false negatives.
Import to Screening Platform
Upload RIS file to Covidence, Rayyan, or similar
Create Validation Subset
Tag random sample for dual screening
Assign Reviewers
Set up blinded review for validation sample
Calculate Agreement
Use platform tools or external calculator
Complete QA Checks
Review all inclusions and spot-check exclusions
Document & Report
Update PRISMA diagram and methods section