Nested Knowledge

Bringing Systematic Review to Life

User Tools

Site Tools

Trace:
wiki:autolit:tagging:smart_tags_guidelines

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
wiki:autolit:tagging:smart_tags_guidelines [2024/01/27 19:47]
jthurnham 		wiki:autolit:tagging:smart_tags_guidelines [2024/02/10 02:09] (current)
kevinkallmes
Line 2: Line 2:
  
 Smart Tag Recommendations is most effective when the tag hierarchy is kept as simple and concrete as possible. See below for tips on how to optimise smart tag recommendations: Smart Tag Recommendations is most effective when the tag hierarchy is kept as simple and concrete as possible. See below for tips on how to optimise smart tag recommendations:
 +
 +{{youtube>p59whSrkzR0}}
  
 ===== Configuring the Tag Hierarchy ===== ===== Configuring the Tag Hierarchy =====
Line 7: Line 9:
 **Mode** **Mode**
  
-  * Form-based Tagging mode (default) works best  +   * Form-based Tagging mode (default) works best 
-     * Smart Tags work in Standard mode too, but does not have explicit Questions to feed to the LLM+      * Smart Tags work in Standard mode too, but does not have explicit Questions to feed to the LLM
  
 **Tags** **Tags**
  
 Ensure you provide: Ensure you provide:
-  * Tag Name + 
 +  * Tag Name
   * Question Type (where relevant),   * Question Type (where relevant),
   * Question and Description text to give details beyond the   * Question and Description text to give details beyond the
-  * Tag Name, +  * Tag Name,
   * Aliases, if there are common synonyms or acronyms   * Aliases, if there are common synonyms or acronyms
   * All of the above, as well as the general hierarchy structure, is fed to the LLM   * All of the above, as well as the general hierarchy structure, is fed to the LLM
Line 22: Line 25:
 **Tag Name** **Tag Name**
  
-  * Limit to 4-5 words max, spell out acronyms, be careful with special characters ($, :, -, % etc.) only use if the term is commonly reported +  * Limit to 4-5 words max, spell out acronyms, be careful with special characters ($, :, -, % etc.) only use if the term is commonly reported
   * Ensure terms are spelled correctly but don’t worry about variations (e.g. UK vs US English)   * Ensure terms are spelled correctly but don’t worry about variations (e.g. UK vs US English)
- 
  
 **Question type** **Question type**
 +
   * Use Single Apply where possible   * Use Single Apply where possible
-  * Use Select types if total answers/child tags are less than 8, too many tags  can overwhelm the LLM+  * Use Select types if total answers/child tags are less than 8, too many tags can overwhelm the LLM
   * Currently only one recommendation is generated per question, which can be less helpful for Multiple Select. Single Apply is more likely to identify area of paper containing various answers   * Currently only one recommendation is generated per question, which can be less helpful for Multiple Select. Single Apply is more likely to identify area of paper containing various answers
- 
  
 **Questions** **Questions**
 +
   * Keep it simple and direct, don’t add too much detail   * Keep it simple and direct, don’t add too much detail
   * Present or past tense questions starting with “What” or “Where” tend to work best   * Present or past tense questions starting with “What” or “Where” tend to work best
  
 **Aliases (Optional)** **Aliases (Optional)**
 +
   * When the tag name cannot be identified (this includes both question and child tags for select question types), the LLM searches for aliases   * When the tag name cannot be identified (this includes both question and child tags for select question types), the LLM searches for aliases
   * This is most helpful to provide for field-specific terms   * This is most helpful to provide for field-specific terms
  
 **Contents (Tag Tables and Options)** **Contents (Tag Tables and Options)**
 +
   * Where possible, use free-text Tags; if you use a Tag Table, the LLM can scroll you to relevant text, but cannot assist with extraction   * Where possible, use free-text Tags; if you use a Tag Table, the LLM can scroll you to relevant text, but cannot assist with extraction
   * Similarly, if employing Options, the LLM can scroll you to relevant text, but will not select Options on your behalf   * Similarly, if employing Options, the LLM can scroll you to relevant text, but will not select Options on your behalf
Line 58: Line 63:
 Only after your tag hierarchy is configured should you toggle on Smart Tags: Only after your tag hierarchy is configured should you toggle on Smart Tags:
  
-In Settings → Tag Recommendations → Choose Type: Smart.  +In Settings → Tag Recommendations → Choose Type: Smart. By default, Smart Tags searches Full Texts but you may also wish to turn on Abstract Tag Recommendations, which may be toggled on in the same section of Settings. The LLM will then also highlight evidence on the Abstract page.
-By default, Smart Tags searches Full Texts but you may also wish to turn on Abstract Tag Recommendations, which may be toggled on in the same section of Settings. The LLM will then also highlight evidence on the Abstract page.+
  
 ===== Recommendation Limits ===== ===== Recommendation Limits =====
  
-**Abstracts:** Smart Tags is currently able to run on 1,000 total abstracts or less, and runs on all abstracts in your nest +**Abstracts:**  Smart Tags is currently able to run on 1,000 total abstracts or less, and runs on all abstracts in your nest
- +
-**Full Texts:** Smart Tags is currently able to run on 250 full texts or less, and runs only on included reports.+
  
 +**Full Texts:**  Smart Tags is currently able to run on 250 full texts or less, and runs only on included reports.
  
 ===== Using Smart Tags ===== ===== Using Smart Tags =====
  
-Smart Tags can take some time to generate. Feel free to click out of the modal and let it run in the background, you can check back in on its progress anytime using the      icon. +Smart Tags can take some time to generate. Feel free to click out of the modal and let it run in the background, you can check back in on its progress anytime using the icon.
  
 As applicable, Abstract or Full Tag Recs are displayed as a right hand tab (in red below) wherever you can view a record: Screening & Tagging modules and Study Inspector. Ensure you have Questions open (in blue below), with a question selected for the Smart Tag Recommendation to be displayed (in green). Clicking the rec will auto-scroll to its suggested text excerpt within the Abstract/Full Text for you to review and apply or reject. If a recommendation cannot be found, there will be none listed. As applicable, Abstract or Full Tag Recs are displayed as a right hand tab (in red below) wherever you can view a record: Screening & Tagging modules and Study Inspector. Ensure you have Questions open (in blue below), with a question selected for the Smart Tag Recommendation to be displayed (in green). Clicking the rec will auto-scroll to its suggested text excerpt within the Abstract/Full Text for you to review and apply or reject. If a recommendation cannot be found, there will be none listed.
  
-{{ :undefined:screenshot_2024-01-27_at_11.52.51.png?nolink |}}+{{  :undefined:screenshot_2024-01-27_at_11.52.51.png?nolink&  }}
  
 ===== Advanced: Bulk Apply Smart Tags ===== ===== Advanced: Bulk Apply Smart Tags =====
  
-While we encourage users to oversee the LLM by reviewing and either accepting or rejecting each recommendation, you may want to apply all the LLM recommendations at once. For instance, when completing a scoping review, you may want to view all LLM recommendations without any curation. +While we encourage users to oversee the LLM by reviewing and either accepting or rejecting each recommendation, you may want to apply all the LLM recommendations at once. For instance, when completing a scoping review, you may want to view all LLM recommendations without any curation.
  
-To do so, head to [[wiki:autolit:utilities:inspector|Study Inspector]], filter to your studies of choice, select [[wiki:autolit:utilities:inspector:bulk_actions|Bulk Actions]] → Apply Tag Recommendations, Smart Tags and Full Text are selected by default, select Apply. This will apply all Tag Recommendations, which you can then view in Qualitative Synthesis.  +To do so, head to [[:wiki:autolit:utilities:inspector|]], filter to your studies of choice, select [[:wiki:autolit:utilities:inspector:bulk_actions|]] → Apply Tag Recommendations, Smart Tags and Full Text are selected by default, select Apply. This will apply all Tag Recommendations, which you can then view in Qualitative Synthesis.
- +
-<WRAP center round important 60%> +
-Bulk Actions is an advanced tool, make sure to familiarize yourself with the use of Bulk Actions and Study Inspector first before going ahead with this action! +
-</WRAP>+
  
 +<WRAP center round important 60%> Bulk Actions is an advanced tool, make sure to familiarize yourself with the use of Bulk Actions and Study Inspector first before going ahead with this action! </WRAP>
  
 ===== FAQs ===== ===== FAQs =====
  
-**What data does OpenAI GPT 4 use? Are there restrictions in place on the use of my data? +**What data does OpenAI GPT 4 use? Are there restrictions in place on the use of my data? **
-**+
  
 When using Smart Tag Recommendations, your tag hierarchy and all relevant text (abstract and full text) content are sent to OpenAI via their API. OpenAI is blinded to your identity, and processes this information for the express purpose of providing Smart Tag Recommendations, per standard Data Processing Addendum. Your data will not be used to inform or train OpenAI products. When using Smart Tag Recommendations, your tag hierarchy and all relevant text (abstract and full text) content are sent to OpenAI via their API. OpenAI is blinded to your identity, and processes this information for the express purpose of providing Smart Tag Recommendations, per standard Data Processing Addendum. Your data will not be used to inform or train OpenAI products.
Line 98: Line 97:
 First, make sure recommendations were successfully generated in Background Jobs. Then make sure you are on either the Abstract or Full Text page, you have the right hand Questions and Tag Recs tabs open and a Question is selected. Try going through a few questions in the form, it may be that fewer recommendations were generated than expected. Still nothing? You may need to simplify your hierarchy and regenerate recs. If that doesn’t work, contact support@nested-knowledge.com. First, make sure recommendations were successfully generated in Background Jobs. Then make sure you are on either the Abstract or Full Text page, you have the right hand Questions and Tag Recs tabs open and a Question is selected. Try going through a few questions in the form, it may be that fewer recommendations were generated than expected. Still nothing? You may need to simplify your hierarchy and regenerate recs. If that doesn’t work, contact support@nested-knowledge.com.
  
-** +**The recommendation is highlighting the same excerpt for multiple questions or isn’t applicable to my question, can I retrain the AI?**
-The recommendation is highlighting the same excerpt for multiple questions or isn’t applicable to my question, can I retrain the AI?**+
  
 OpenAI GPT 4 works to find the most succinct answer to the question, which means it often highlights the abstract/similar excerpts. Currently there is no way to retrain the AI but we are working on having multiple recommendations generated for you to choose from… coming soon! OpenAI GPT 4 works to find the most succinct answer to the question, which means it often highlights the abstract/similar excerpts. Currently there is no way to retrain the AI but we are working on having multiple recommendations generated for you to choose from… coming soon!
Line 106: Line 104:
  
 No, after configuring new questions, Smart Tags must be regenerated in Settings. Note: each nest is limited to one regeneration due to LLM costs. If you need to regenerate Smart Tags after using your one regeneration, contact support@nested-knowledge.com. No, after configuring new questions, Smart Tags must be regenerated in Settings. Note: each nest is limited to one regeneration due to LLM costs. If you need to regenerate Smart Tags after using your one regeneration, contact support@nested-knowledge.com.
- 
  
  
wiki/autolit/tagging/smart_tags_guidelines.1706384842.txt.gz · Last modified: 2024/01/27 19:47 by jthurnham

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki