Nested Knowledge

Bringing Systematic Review to Life

User Tools

Site Tools

Trace:
wiki:autolit:screening:dual

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
wiki:autolit:screening:dual [2023/04/24 17:46]
jthurnham [1. Screen each study twice] 		wiki:autolit:screening:dual [2024/04/01 11:43]
jthurnham [2. [OPTIONAL] Auto-Adjudicate]
Line 16: Line 16:
 To configure dual screening in a nest, click on the "Settings" link under Nest Home. Once there, scroll down to the Screening section. Then, click on the "Dual" option in the ( <font inherit/inherit;;#e74c3c;;inherit>red box</font> ). To configure dual screening in a nest, click on the "Settings" link under Nest Home. Once there, scroll down to the Screening section. Then, click on the "Dual" option in the ( <font inherit/inherit;;#e74c3c;;inherit>red box</font> ).
  
-{{:wiki:autolit:screening:img66.png?nolink&  }}+{{ :undefined:screenshot_2023-12-05_at_16.33.45.png?nolink |}}
  
 Once this is complete, a new "Adjudicate Screening" option will appear in the Nest Menu for all Admins: Once this is complete, a new "Adjudicate Screening" option will appear in the Nest Menu for all Admins:
  
-{{:wiki:autolit:screening:img67_1.png?nolink&  }}+{{ :undefined:screenshot_2023-12-05_at_16.40.54.png?nolink |}}
  
 <WRAP center round important 60%> **Note:** Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, so all records without an adjudicated Include or Exclude decision will be reverted to Unscreened and **all data associated with individual users' decisions will be lost**! </WRAP> <WRAP center round important 60%> **Note:** Toggling back from Dual Screening to Standard Screening (or switching to Two-Pass Screening) will ONLY save final adjudications, so all records without an adjudicated Include or Exclude decision will be reverted to Unscreened and **all data associated with individual users' decisions will be lost**! </WRAP>
Line 31: Line 31:
 In Dual Screening, it can be useful to view the number of prior reviewers for the current record. This is displayed to the right of the include button (see below). 0 means no decisions have been made about the current record, 1 means 1 reviewer has made a decision, and so on. In Dual Screening, it can be useful to view the number of prior reviewers for the current record. This is displayed to the right of the include button (see below). 0 means no decisions have been made about the current record, 1 means 1 reviewer has made a decision, and so on.
  
-{{ :undefined:prior_reveiewrs.png?nolink |}}+{{ :undefined:screenshot_2023-12-05_at_16.47.10.png?nolink |}}
  
 However, in Dual modes the status of whether the full text has been uploaded or not by the other reviewer is hidden. This is to avoid bias as the knowledge that the other user has uploaded the record's full text may influence your screening decision. You still have the option to show the full text upload status as well as the full text regardless by clicking "Show Anyways." This action does not affect your screening decisions. However, in Dual modes the status of whether the full text has been uploaded or not by the other reviewer is hidden. This is to avoid bias as the knowledge that the other user has uploaded the record's full text may influence your screening decision. You still have the option to show the full text upload status as well as the full text regardless by clicking "Show Anyways." This action does not affect your screening decisions.
  
-{{ :undefined:show_anyway.png?nolink |}}+{{ :undefined:screenshot_2023-12-05_at_16.48.21.png?nolink |}}
  
-==== 2. [OPTIONAL] Auto-Adjudicate ====+==== 3. Adjudicate Disagreements ====
  
-All studies that have undergone two screening decisions are sent forward for adjudication, and any study that is either Included by both Screeners or Excluded by both Screeners is eligible for Auto-Adjudication.+For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option ( <font inherit/inherit;;#e74c3c;;inherit>red box</font>  ). Once adjudicated, the studies will either be excluded or included and sent forward to [[:wiki:autolit:tagging|]].
  
-To Auto-Adjudicate all eligible studies, navigate to Adjudicate Screening, and in the upper right, select "Auto-adjudicate {x} studies" ( <font inherit/inherit;;#e74c3c;;inherit>red box</font> ). This will automatically include all studies that both Screeners included, and exclude all studies that both Screeners excluded.+Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can [[:wiki:autolit:admin:configure#blinding|Blind Adjudication in Settings.]]
  
-If Screener 1 and Screener 2 selected different Exclusion Reasons, the Auto-Adjudication will select only one of these and apply it as the final Exclusion Reason.+==== Kappa Statistics for Interrater Reliability ====
  
-{{:wiki:autolit:screening:img68.png?nolink&  }}+After you finish Dual Screening, you can view the [[:wiki:autolit:screening:kappastatistics|Kappa statistics]] in [[:wiki:autolit:utilities:activity|Activity]]. 
 + 
 +==== Guidance on Dual Screening Best Practices ==== 
 + 
 +For guidance on best practices in Dual Screening, click [[:wiki:guide:research:screen#dual_screening|here]].
  
  
Line 52: Line 56:
 For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option ( <font inherit/inherit;;#e74c3c;;inherit>red box</font> ). Once adjudicated, the studies will either be excluded or included and sent forward to [[:wiki:autolit:tagging|]]. For any study that is not Auto-Adjudicated, an Admin will need to manually adjudicate in order to provide a final screening decision. The Admin should choose between selecting the decision of Screener 1 or Screener 2, or if both are incorrect, provide a different option ( <font inherit/inherit;;#e74c3c;;inherit>red box</font> ). Once adjudicated, the studies will either be excluded or included and sent forward to [[:wiki:autolit:tagging|]].
  
 +Note: by default, the names of the reviewers will be displayed alongside their decisions. You may want to reduce bias by hiding this information. To do so you can [[wiki:autolit:admin:configure#blinding|Blind Adjudication in Settings.]]
 +==== Kappa Statistics for Interrater Reliability ====
 +
 +After you finish Dual Screening, you can view the [[wiki:autolit:screening:kappastatistics|Kappa statistics]] in [[wiki:autolit:utilities:activity|Activity]].
  
 ==== Guidance on Dual Screening Best Practices ==== ==== Guidance on Dual Screening Best Practices ====
wiki/autolit/screening/dual.txt · Last modified: 2024/04/01 11:59 by jthurnham

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki