Nested Knowledge

Bringing Systematic Review to Life

User Tools

Site Tools


wiki:autolit:screening:inclusionpredictionmodel

This is an old revision of the document!


Screening Model

The Screening Model uses AI to learn from screening decisions within a specific nest, generating inclusion probabilities based on configuration.

You may use the screening model in two ways:

  • Generate inclusion probabilities to be displayed on records and assist in your own manual screening.
  • Dual modes only: Turn on Robot Screener to replace an expert reviewer, which makes decisions based on these probabilities.

Both methods require the model to be trained but the first only displays probabilities, allowing you to order studies in the screening queue by likelihood of inclusion or bulk screen studies by inclusion probability. Whereas Robot Screener actively takes the place of a second reviewer, and hides the probabilities for the remaining reviewers by default.

The below guidance is specifically for using Robot Screener, for information on training the model for probability generation only and general information on how the model works see Using and Interpreting the Screening Model.

Robot Screener

The Screening Model can be used to power AI-assisted screening, replacing one expert in Dual Screening processes.

Robot Screener may only be turned on in Dual Screening modes and it's important to note at what stage they are generated and the language used:

  • Standard Mode: Robot Screener replaces a reviewer in the singular round of Screening based on Inclusion Probabilities.
  • Two Pass Mode: Robot Screener replaces a reviewer in the Title/Abstract round of Screening only, based on Advancement Probabilities.

User Guide

Running the Screening Model

To learn about configuration settings, which enable you to toggle Manual updating vs. Automatic and Displayed vs. Hidden, see the Settings page.

In its default setting, the Screening Model must be run manually. To do so, click “Train Screening Model” on the Screening panel:

Once the modal opens, click “Train New Model.”

To provide the model with sufficient information to begin understanding your review, we require 50 total adjudicated screening decisions with 10 advancements or inclusions before the model can be trained. If there is insufficient evidence to train the model, complete more adjudicated screening (2 reviewers and 1 adjudicator) until the “Train New Model” button becomes available.

It may take a minute to train, after which it will populate a histogram on the left. From then on, each record will show a probability of inclusion or advancement:

wiki/autolit/screening/inclusionpredictionmodel.1721839802.txt.gz · Last modified: 2024/07/24 16:50 by jthurnham