Nested Knowledge

Bringing Systematic Review to Life

User Tools

Site Tools

Trace: • Mobile Device Policy • Information Security Policy
wiki:autolit:screening:dualtwopass

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
wiki:autolit:screening:dualtwopass [2022/11/20 00:27]
nicole_hardy_alumni.brown.edu [3. Scroll to Screening settings. Select Two Pass under Mode and Dual under Number of Reviewers.] 		wiki:autolit:screening:dualtwopass [2023/12/06 15:38] (current)
jthurnham
Line 10: Line 10:
 ===== Configure Dual Two-Pass Screening ===== ===== Configure Dual Two-Pass Screening =====
  
-==== 1. Click on Settings ====+==== Navigate to Settings and select "Dual" and "Two-Pass" ====
  
-{{:wiki:autolit:screening:img69.png?nolink&  }} +{{ :undefined:screenshot_2023-12-06_at_15.28.24.png?nolink |}}
- +
- +
-==== 3. Scroll to Screening settings. Select Two Pass under Mode and Dual under Number of Reviewers. ==== +
- +
-{{:wiki:autolit:screening:img70.png?nolink&  }}+
  
 <WRAP center round important 60%> **Note:** Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, so all records without an adjudicated Include or Exclude decision will be reverted to Unscreened and **all data associated with individual users' decisions will be lost**! </WRAP> <WRAP center round important 60%> **Note:** Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, so all records without an adjudicated Include or Exclude decision will be reverted to Unscreened and **all data associated with individual users' decisions will be lost**! </WRAP>
Line 26: Line 21:
 ==== 1. Screen each study twice at the abstract level. ==== ==== 1. Screen each study twice at the abstract level. ====
  
-Two independent reviewers will need to review the abstract of every study and [[:wiki:autolit:screening:exclude|screen]] the abstracts using the same approach as Standard Screening Mode with the exception that studies are only advanced to full-text screening at this stage instead of included. AutoLit automatically queues the abstracts to all users until two screening decisions are made; then, the abtracts are sent forward for adjudication.+Two independent reviewers will need to review the abstract of every study and [[:wiki:autolit:screening:exclude|screen]] the abstracts using the same approach as Standard Screening Mode with the exception that studies are only advanced to full-text screening at this stage instead of included. AutoLit automatically queues the abstracts to all users until two screening decisions are made; then, the abstracts are sent forward for adjudication. 
 + 
 +{{ :undefined:screenshot_2023-12-06_at_15.29.19.png?nolink |}} 
 + 
 +In Dual modes, it can be useful to view the number of prior reviewers for the current record. This is displayed to the right of the advance button (see below). It is also displayed next to the include button in Full Text screening. 0 means no decisions have been made about the current record, 1 means 1 reviewer has made a decision, and so on. 
 + 
 +{{ :undefined:screenshot_2023-12-06_at_15.30.20.png?nolink |}} 
 + 
 +Even in Abstract Screening, you may want to view the full text, to do so go to the Full Text tab. In Dual Screening types, the status of whether the full text has been uploaded or not is hidden. This is to avoid bias as the knowledge that the other user has uploaded the record's full text may influence your screening decision. You still have the option to show the full text upload status as well as the full text regardless by clicking “Show Anyways.” This action does not affect your screening decisions and is not shown in the Full Text Screening module since all full texts will be uploaded.
  
-{{  :wiki:autolit:screening:5.png?nolink&  }} 
  
 ==== 2. Adjudicate decision for abstracts ==== ==== 2. Adjudicate decision for abstracts ====
Line 34: Line 36:
 There is an option to [[:wiki:autolit:screening:dual#optional_auto-adjudicate|auto-adjudicate]]. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision on the abstracts. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option. Once adjudicated, the studies will either be excluded or advanced and sent forward to Full Text Screening. There is an option to [[:wiki:autolit:screening:dual#optional_auto-adjudicate|auto-adjudicate]]. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision on the abstracts. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option. Once adjudicated, the studies will either be excluded or advanced and sent forward to Full Text Screening.
  
-{{  :wiki:autolit:screening:6.png?nolink&  }}+Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can [[wiki:autolit:admin:configure#blinding|Blind Adjudication in Settings.]] 
 + 
 +If you have Robot Screener turned on, its decisions will be shown in Abstract Adjudication and not Full Text Adjudication. This is because, in Dual Two Pass mode, Robot Screener only runs on abstracts. Learn more [[https://wiki.nested-knowledge.com/doku.php?id=wiki:autolit:screening:robot|here.]] 
 + 
 +{{ :undefined:screenshot_2023-12-06_at_15.31.44.png?nolink |}} 
  
 ==== 4. Screen the full-text of each study. ==== ==== 4. Screen the full-text of each study. ====
Line 40: Line 47:
 Two independent reviewers will need to review the full-text of every study and [[:wiki:autolit:screening:exclude|screen]] the abstracts using the same approach as Standard Screening Mode. AutoLit automatically queues the full-texts to all users until two screening decisions are made; then, the articles are sent forward for adjudication. Two independent reviewers will need to review the full-text of every study and [[:wiki:autolit:screening:exclude|screen]] the abstracts using the same approach as Standard Screening Mode. AutoLit automatically queues the full-texts to all users until two screening decisions are made; then, the articles are sent forward for adjudication.
  
-{{  :wiki:autolit:screening:10.png?nolink&  }}+{{ :undefined:screenshot_2023-12-06_at_15.35.35.png?nolink |}} 
  
 ==== 5. Adjudicate decisions for full-texts ==== ==== 5. Adjudicate decisions for full-texts ====
Line 46: Line 54:
 There is an option to [[:wiki:autolit:screening:dual#optional_auto-adjudicate|auto-adjudicate]]. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision on the full-texts. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option. Once adjudicated, the studies will either be excluded or included. There is an option to [[:wiki:autolit:screening:dual#optional_auto-adjudicate|auto-adjudicate]]. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision on the full-texts. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option. Once adjudicated, the studies will either be excluded or included.
  
-{{  :wiki:autolit:screening:9.png?nolink&  }}+Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can [[wiki:autolit:admin:configure#blinding|Blind Adjudication in Settings.]] 
 + 
 +{{ :undefined:screenshot_2023-12-06_at_15.37.05.png?nolink |}} 
 + 
 +==== Kappa Statistics for Interrater Reliability ==== 
 + 
 +After you finish Dual Screening, you can view the [[wiki:autolit:screening:kappastatistics|Kappa statistics]] in [[wiki:autolit:utilities:activity|Activity]].
  
 ==== Guidance on Dual Screening Best Practices ==== ==== Guidance on Dual Screening Best Practices ====
wiki/autolit/screening/dualtwopass.1668904077.txt.gz · Last modified: 2022/11/20 00:27 by nicole_hardy_alumni.brown.edu

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki