wiki:guide:qc
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
wiki:guide:qc [2022/01/17 19:39] katcow |
wiki:guide:qc [2022/04/18 21:05] (current) khutchison |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Quality Control (QC) ====== | ====== Quality Control (QC) ====== | ||
| - | ===== Nest Overview ===== | + | ===== Overview ===== |
| - | The first step in quality control is to understand the research question and proposal if available. Once the project goals are clear, identify the aim of the QC by asking the following | + | The first step in quality control is to understand the research question and proposal if available. Once the project goals are clear, identify the aim of the QC by asking the following: |
| * Is the main goal of this QC to finalize the data before statistical analysis? | * Is the main goal of this QC to finalize the data before statistical analysis? | ||
| - | * Are there specific tasks from the project leads to look out for or complete while inside the nest? | ||
| In the case that the project is unclear and the research design needs to be refined, one of the goals of QCing will be to refine the research question. Evaluate studies for common themes while reviewing the data extraction with the following guiding questions: | In the case that the project is unclear and the research design needs to be refined, one of the goals of QCing will be to refine the research question. Evaluate studies for common themes while reviewing the data extraction with the following guiding questions: | ||
| Line 12: | Line 11: | ||
| * Does the research design make sense and is the aim of the research clear? | * Does the research design make sense and is the aim of the research clear? | ||
| * Are the primary outcomes (or the data elements that are of highest interest) clear and reported in a standard, measurable way? | * Are the primary outcomes (or the data elements that are of highest interest) clear and reported in a standard, measurable way? | ||
| - | * Are there common outcomes or interventions mentioned in the included studies that are missing | + | * Are there common outcomes or interventions mentioned in the included studies that are missing? |
| ===== Screening ===== | ===== Screening ===== | ||
| Line 19: | Line 18: | ||
| * Low data density or quality (does not have a significant amount of data extraction or does not incude the main outcomes of interest) | * Low data density or quality (does not have a significant amount of data extraction or does not incude the main outcomes of interest) | ||
| - | * Subpopulation that could skew results in a certain | + | * Subpopulation that could skew results in a certain |
| * The cohort is not homogenous (ex: all are considered COVID-19 patients but some patients are suspected and some are PCR-confirmed positive) | * The cohort is not homogenous (ex: all are considered COVID-19 patients but some patients are suspected and some are PCR-confirmed positive) | ||
| * Study type (consider if lower level evidence such as case reports or case series should be included for the project) | * Study type (consider if lower level evidence such as case reports or case series should be included for the project) | ||
| * Does not report primary outcome of interest (ex: reports 30-day mortality when the primary outcome of interest is 90-day mortality) | * Does not report primary outcome of interest (ex: reports 30-day mortality when the primary outcome of interest is 90-day mortality) | ||
| - | In the case that the study should not be included, note the reason that exclusion is recommended | + | In the case that the study should not be included, note the reason that exclusion is recommended for secondary review. |
| - | ===== Tagging ===== | + | When performing an SR/MA, you should |
| - | + | ||
| - | * Look at the sunburst diagram in qualitative synthesis and evaluate gap teeth. Gap teeth occur when a study has been tagged at the higher level instead of the lower, most specific level. If studies are tagged with Study Design instead of the most specific tag, this may result in gap teeth. Gap teeth can also result when there is a small amount | + | |
| - | * For tagging only nests, evaluate each study for tagging similar to the process of checking the extraction. | + | |
| - | * Look at quantitative synthesis | + | |
| ===== Extraction ===== | ===== Extraction ===== | ||
| - | Examine | + | Examine the data elements for extraction. There may be inconsistencies that occurred as a result of variances in study reporting that can only be uncovered after reviewing the extraction of several studies. When examining the extraction for each study, look out for the following common data element errors: |
| * Clarify any data elements that are vague (ex: how are we defining clinical improvement? | * Clarify any data elements that are vague (ex: how are we defining clinical improvement? | ||
| * Studies that have a limited amount of data elements extracted (low data density). Comment directly on the study to be evaluated for secondary review. | * Studies that have a limited amount of data elements extracted (low data density). Comment directly on the study to be evaluated for secondary review. | ||
| - | * Data elements may have been configured incorrectly, | + | * If you notice that a data element is being reported more often as a mean instead of a median, this may need to be changed. |
| - | * Look at quantitative synthesis and note data elements that are " | + | * Check that outcomes are reported on consistent scales with the same units (ex: if pain is reported on a VAS vs NRS scale, then there is a need to create two separate data elements). |
| - | * Check that outcomes are reported on consistent scales with the same units (ex: if pain is reported on a VAS vs NRS scale, then there is a need to create two separate data elements) | + | * Check units of data elements to ensure they are consistently reported the same way. |
| - | * Check units of data elements to ensure they are consistently reported the same way in the nest. | + | * Make sure that the tagged intervention groups match the extracted intervention groups. |
| - | Examine the nest at the study level. | + | Go through each study and check that each element, intervention, |
| === Common extraction errors to correct: === | === Common extraction errors to correct: === | ||
| - | |||
| * Elements that are collected as outcomes do not have loss to follow up reflected in total. | * Elements that are collected as outcomes do not have loss to follow up reflected in total. | ||
| * Combined means were not combined correctly | * Combined means were not combined correctly | ||
| * Continuous data elements are being reported using the incorrect statistical measure (standard error is not the same as standard deviation) | * Continuous data elements are being reported using the incorrect statistical measure (standard error is not the same as standard deviation) | ||
| - | * Make sure that elements are being extracted at the same time point or that the wording of the study is consistent with what is being collected (ex: 30-day mortality is not the same as 90-day mortality). Clarify | + | * Make sure that elements are being extracted at the same time point or that the wording of the study is consistent with what is being collected (ex: 30-day mortality is not the same as 90-day mortality). Clarify if unsure of what the time point of interest is. Look out for inconsistencies across studies and note if there are timepoints that are commonly reported but are missed. |
| - | + | ||
| - | Note any thoughts or concerns you have in the comments section of the study. If you have done extensive revisions for the study, note the problem with the first extraction and how you fixed it. Justify any recommendations for inclusion/ | + | |
| ===== Statistical Errors and Clarifications ===== | ===== Statistical Errors and Clarifications ===== | ||
| Line 59: | Line 51: | ||
| QC requires a thorough understanding of the statistical measurements that are encountered when reading scientific literature. Refer to the following list if clarification is needed. | QC requires a thorough understanding of the statistical measurements that are encountered when reading scientific literature. Refer to the following list if clarification is needed. | ||
| - | Mean - The average of a set of numbers. This measure can be combined across populations if there are two means reported. | + | | <font 11pt/ |
| + | | <font 11pt/ | ||
| + | | <font 11pt/ | ||
| + | | <font 11pt/ | ||
| + | | <font 11pt/ | ||
| + | | <font 11pt/ | ||
| - | Median | + | ====== NK-specific QC ====== |
| - | Standard Deviation (SD) - A measure of the amount of variance within a set of numbers. Typically reported with a mean value using the symbol ' | + | //To be done in addition to steps outlined above. // |
| - | Standard Error (SE) - The standard deviation of a sample of the larger population. This is NOT the same thing as SD and should not be used in replacement of SD. | + | ===== Tagging ===== |
| - | Range - The difference between | + | //QC after review Screening section and review before Extraction section.// |
| + | * Look at the sunburst diagram in qualitative synthesis | ||
| + | * For tagging only nests, evaluate each study for tagging similar to the process of checking the extraction. | ||
| + | * Look at quantitative synthesis and note data elements that are " | ||
| - | Interquartile Range (IQR) - The difference between the upper and lower quartiles. This is NOT the same thing as Range and should not be used as a replacement. | + | ===== Extraction ===== |
| + | |||
| + | * Look at quantitative synthesis | ||
| + | |||
| + | Note any thoughts or concerns you have in the comments section of the study. If you have done extensive revisions for the study, note the problem with the first extraction | ||
| ===== Final Steps ===== | ===== Final Steps ===== | ||
| * Let the team or project lead know that QC is complete and include notes and recommendations in the email. | * Let the team or project lead know that QC is complete and include notes and recommendations in the email. | ||
| - | * In the Nest Comments, type up a QC report detailing any changes that have made to the nest and any clarifications or new standards that have been set. Reccomendations | + | * In the Nest Comments, type up a QC report detailing any changes that have made to the nest and any clarifications or new standards that have been set. Recommendations |
wiki/guide/qc.1642448346.txt.gz · Last modified: 2022/01/17 19:39 by katcow
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
