This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
wiki:autolit:screening:robot [2023/05/17 23:12] jthurnham [Guidance on Robot Screener in Dual Two Pass Mode] |
wiki:autolit:screening:robot [2024/06/10 19:38] kevinkallmes |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Robot Screener | + | ===== Guidance on Robot Screener |
- | Robot Screener | + | The recommended practice |
- | + | ||
- | ===== Enabling Robot Screener ===== | + | |
- | + | ||
- | Robot Screener may be enabled from Screening Settings. When Dual Screening mode (standard or two pass) is switched on, Robot Screener can be switched on under " | + | |
- | + | ||
- | | + | |
- | | + | |
- | * And contain **10** | + | |
- | + | ||
- | The latter two requirements help ensure a minimum model accuracy; however, the model will usually be suboptimal with this volume of training data, and typically improves as more records are screened. | + | |
- | + | ||
- | By enabling | + | |
- | + | ||
- | When switching on Robot Screener, you will see the below modal. In red are the actual numbers of adjudicated and included or advanced records compared to the numbers you need to meet in order to Enable Robot Screener: | + | |
- | + | ||
- | {{ : | + | |
- | + | ||
- | ===== In Action ===== | + | |
- | + | ||
- | ==== Which Records ==== | + | |
- | + | ||
- | Robot Screener automatically adds a single | + | |
- | + | ||
- | * Fewer than 2 reviewer-level screening decisions | + | |
- | * No adjudicated screening decision | + | |
- | + | ||
- | Records it excludes will assigned the exclusion reason '' | + | |
- | + | ||
- | ==== When ==== | + | |
- | + | ||
- | Robot Screener adds/ | + | |
- | + | ||
- | * Robot Screener is enabled | + | |
- | * A new Screening | + | |
- | | + | |
- | | + | |
- | + | ||
- | As pointed out above, Robot Screener will not modify its decision on a record after that record has been adjudicated. Prior to adjudication, | + | |
- | + | ||
- | ===== Usage Guidance ===== | + | |
- | + | ||
- | Robot Screener, by accuracy, is not a replacement for a human reviewer. It should typically be used for reviews where budget, team size, or time available are restricted. Some additional usage tips: | + | |
- | + | ||
- | * While Robot Screener may be enabled at 50 adjudicated / 10 included records, we typically advise not enabling it until a cross validation recall of 0.7 or greater is achieved. | + | |
- | * Only use the " | + | |
- | + | ||
- | Learn more about [[wiki: | + | |
- | + | ||
- | ===== Why can't I turn on Robot Screener yet? ===== | + | |
- | + | ||
- | It may be due to not having enough screening decisions in your nest. [[wiki: | + | |
- | + | ||
- | If you are in Dual Two Pass Screening | + | |
* Abstract Screening requires **50** | * Abstract Screening requires **50** | ||
* These actions must be performed before Robot Screener can be turned on for Abstract Screening | * These actions must be performed before Robot Screener can be turned on for Abstract Screening | ||
- | * Full Text Screening requires **50** | + | * Because fewer records are available, more intensive assessment of the full text is needed, and final decisions on inclusion are made at the Full Text stage, AutoLit does not currently offer Robot Screener at the Full Text Screening |
- | * These actions must be performed before | + | * You can keep Robot Screener |