This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
wiki:autolit:screening:dual [2022/10/07 18:17] nicole_hardy_alumni.brown.edu |
wiki:autolit:screening:dual [2024/04/01 11:43] jthurnham [2. [OPTIONAL] Auto-Adjudicate] |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== Dual Screening and Adjudication ====== | ====== Dual Screening and Adjudication ====== | ||
- | **Dual Screening** is a quality-controlled screening process, where two users independently screen each article, and then all screening decisions are adjudicated by an Administrator. Note, this is different than two-pass screening where a user first reviews abstracts and then full-texts of advanced articles. You can, however, perform [[: | + | **Dual Screening** is a quality-controlled screening process, where two users independently screen each article, and then all screening decisions are adjudicated by an Administrator. Note, this is different than [[: |
The Admin adjudicates any disagreement between the original screeners and sets the final determination for each study. For example, if Screener 1 includes a given study but Screener 2 excludes it for Reason 1, the Adjudicator will then need to choose between Inclusion, Excluding for Reason 1, or choosing to Exclude for Reason 2. | The Admin adjudicates any disagreement between the original screeners and sets the final determination for each study. For example, if Screener 1 includes a given study but Screener 2 excludes it for Reason 1, the Adjudicator will then need to choose between Inclusion, Excluding for Reason 1, or choosing to Exclude for Reason 2. | ||
Line 14: | Line 14: | ||
===== Configure Dual Screening ===== | ===== Configure Dual Screening ===== | ||
- | To configure dual screening in a nest, click on the "Admin" link in the Nest menu. Once here, scroll down to the Screening section. Then, click on the " | + | To configure dual screening in a nest, click on the "Settings" link under Nest Home. Once there, scroll down to the Screening section. Then, click on the " |
- | {{:wiki:autolit: | + | {{ :undefined:screenshot_2023-12-05_at_16.33.45.png?nolink |}} |
Once this is complete, a new " | Once this is complete, a new " | ||
- | {{:wiki:autolit: | + | {{ :undefined:screenshot_2023-12-05_at_16.40.54.png?nolink |}} |
<WRAP center round important 60%> **Note:** Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, | <WRAP center round important 60%> **Note:** Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, | ||
Line 27: | Line 27: | ||
==== 1. Screen each study twice ==== | ==== 1. Screen each study twice ==== | ||
- | Before Adjudication can take place, two independent users will need to [[: | + | Before Adjudication can take place, two independent users will need to [[: |
- | ==== 2. [OPTIONAL] Auto-Adjudicate ==== | + | In Dual Screening, it can be useful to view the number of prior reviewers for the current record. This is displayed to the right of the include button (see below). 0 means no decisions have been made about the current record, 1 means 1 reviewer has made a decision, and so on. |
- | All studies that have undergone two screening decisions are sent forward for adjudication, | + | {{ : |
- | To Auto-Adjudicate all eligible studies, navigate to Adjudicate Screening, and in the upper right, select " | + | However, in Dual modes the status of whether the full text has been uploaded or not by the other reviewer is hidden. This is to avoid bias as the knowledge |
- | If Screener 1 and Screener 2 selected | + | {{ : |
+ | |||
+ | ==== 3. Adjudicate Disagreements ==== | ||
+ | |||
+ | For any study that is not Auto-Adjudicated, | ||
+ | |||
+ | Note: by default, | ||
+ | |||
+ | ==== Kappa Statistics for Interrater Reliability ==== | ||
+ | |||
+ | After you finish Dual Screening, you can view the [[: | ||
+ | |||
+ | ==== Guidance on Dual Screening Best Practices ==== | ||
+ | |||
+ | For guidance on best practices in Dual Screening, click [[: | ||
- | {{: | ||
==== 3. Adjudicate Disagreements ==== | ==== 3. Adjudicate Disagreements ==== | ||
- | For any study that is not Auto-Adjudicated, | + | For any study that is not Auto-Adjudicated, |
+ | |||
+ | Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can [[wiki: | ||
+ | ==== Kappa Statistics for Interrater Reliability ==== | ||
+ | |||
+ | After you finish Dual Screening, you can view the [[wiki: | ||
==== Guidance on Dual Screening Best Practices ==== | ==== Guidance on Dual Screening Best Practices ==== |