The Screening Model uses AI to learn from screening decisions within a specific nest, generating inclusion probabilities based on configuration.

You may use the screening model in two ways:

• Generate inclusion probabilities to be displayed on records and assist in your own manual screening.
• Dual modes only: Turn on Robot Screener to replace an expert reviewer, which makes decisions based on these probabilities.

Both methods require the model to be trained but the first only displays probabilities, allowing you to order studies in the screening queue by likelihood of inclusion or bulk screen studies by inclusion probability. Whereas Robot Screener actively takes the place of a second reviewer, and hides the probabilities for the remaining reviewers by default.

The below guidance is specifically for using Robot Screener, for information on training the model for probability generation only and general information on how the model works see Using and Interpreting the Screening Model.

The Screening Model can be used to power AI-assisted screening, replacing one expert in Dual Screening processes.

Robot Screener may only be turned on in Dual Screening modes and it's important to note at what stage they are generated and the language used:

• Standard Mode: Robot Screener replaces a reviewer in the singular round of Screening based on Inclusion Probabilities.
• Two Pass Mode: Robot Screener replaces a reviewer in the Title/Abstract round of Screening only, based on Advancement Probabilities.

To learn about configuration settings, which enable you to toggle Manual updating vs. Automatic and Displayed vs. Hidden, see the Settings page.

In its default setting, the Screening Model must be run manually. To do so, click “Train Screening Model” on the Screening panel:

Once the modal opens, click “Train New Model.”

It may take a minute to train, after which it will populate a histogram on the left. From then on, each record will show a probability of inclusion or advancement:

Robot Screener has been validated in several published studies assessing its decisions in comparison to human decisions across multiple reviews and review types.

• Internal validation: Nested Knowledge assessed the Recall and Precision of Robot Screener in 19 projects with over 100,000 cumulative decisions, finding significantly lower Precision than humans (that is, humans correctly exclude studies more often) but significantly higher Recall– meaning that the Robot Screener misses fewer includable records. In this analysis, Robot Screener was found to have 97.1% Recall.
• External validation: Cichewicz et al. assessed diagnostic accuracy across many metrics in 15 projects, finding Robot Screener had significantly lower Precision than humans and no statistical differences in Recall between Robot Screener and humans. Robot Screener had fewer overall False Negatives, but no significant differences were found.
• Estimates of time savings using different modes of Robot Screener have been previously published online.

You can see a deeper summary of the Validation Studies and their implications here.