This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
wiki:autolit:screening:robot [2023/05/17 22:43] jthurnham |
wiki:autolit:screening:robot [2024/03/06 02:40] (current) kevinkallmes |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== Robot Screener | + | ===== Guidance on Robot Screener |
- | Robot Screener | + | To clarify, the recommended practice is to **use Robot Screener |
- | ===== Enabling | + | For projects with large numbers of Full Text records, |
- | Robot Screener may be enabled from Screening | + | * Abstract |
+ | * These actions must be performed before | ||
- | * Be configured for [[wiki: | + | * Full Text Screening requires |
- | * And contain | + | * These actions must be performed before |
- | * And contain **10** | + | |
- | + | ||
- | The latter two requirements help ensure a minimum model accuracy; however, the model will usually be suboptimal with this volume of training data, and typically improves as more records are screened. | + | |
- | + | ||
- | By enabling Robot Screener, automatic updating to the screening model is made non-optional, | + | |
- | + | ||
- | When switching on Robot Screener, you will see the below modal. In red are the actual numbers of adjudicated and included or advanced records compared to the numbers you need to meet in order to Enable Robot Screener: | + | |
- | + | ||
- | {{ : | + | |
- | + | ||
- | Doing Robot Screener in Two Pass Mode and need more guidance? [[wiki: | + | |
- | + | ||
- | ===== In Action ===== | + | |
- | + | ||
- | ==== Which Records ==== | + | |
- | + | ||
- | Robot Screener automatically adds a single reviewer-level screening decision to records with: | + | |
- | + | ||
- | | + | |
- | | + | |
- | + | ||
- | Records it excludes will assigned the exclusion reason '' | + | |
- | + | ||
- | ==== When ==== | + | |
- | + | ||
- | Robot Screener adds/ | + | |
- | + | ||
- | | + | |
- | | + | |
- | * New records | + | |
- | * 10 additional records have been adjudicated, | + | |
- | + | ||
- | As pointed out above, Robot Screener will not modify its decision on a record after that record has been adjudicated. Prior to adjudication, | + | |
- | + | ||
- | ===== Usage Guidance ===== | + | |
- | + | ||
- | Robot Screener, by accuracy, is not a replacement for a human reviewer. It should typically | + | |
- | + | ||
- | * While Robot Screener may be enabled at 50 adjudicated / 10 included records, we typically advise not enabling it until a cross validation recall of 0.7 or greater is achieved. | + | |
- | * Avoid use of the " | + | |
- | + | ||
- | Learn more about [[wiki: | + | |