DQ001 - Data Quality Guidelines

DQ001 - Data Quality Guidelines and Principal Investigator Verification of Compliance
Review Committee: Data Start Date: 10/28/2008
Attachments: None Last Revised Date: 6/5/2024
Forms: Last Reviewed Date: 6/5/2024

Introduction:

The TBIMS have established data quality guidelines and strategies to ensure that data is being collected the same way throughout the TBIMS.

Purpose:

Through standardized practices for data quality, integrity of data will be preserved.

Scope:

All TBIMS centers and TBIMS longitudinal follow-up centers that participate in data collection for the TBIMS National Database.

Responsibilities:

Each TBIMS Project Director is responsible for the integrity of data being collected within his/her center. The Project Director is also responsible for tracking and assuring that the Data Quality Guidelines outlined below are followed in his/her center. It is recommended that each TBIMS maintain a log documenting that these guidelines are being followed.

Procedural Steps:

  1. FIM® Instrument: It is the responsibility of each center to assure that all staff who perform FIM® assessments and those performing the data quality checks (Form 1 and Form 2) are trained/certified and training/certification is repeated bi-annually for the duration of the time that they collect data/assess patients for the TBIMS National Database. FIM® training and certification is to be completed using the online Training and Certification materials provided on the TBIMS NDSC website.

  2. Disability Rating Scale (DRS): It is the responsibility of each center to assure that all staff who perform DRS ratings and those performing the data quality checks for Form 1 are trained and certified through the website at “www.tbims.org/combi/drs/”. All staff should be re-certified bi-annually.

  3. Pre-Injury History Data: The Pre-Injury History should be used to collect relevant data for the TBIMS National Database Form 1. Exact wording of questions is provided. The Pre-Injury History form is available on the NDSC website. Centers should not be using their own data collection methods or forms.

  4. Intracranial CT Diagnosis: TBIMS staff coding the CT Diagnosis data collection should be certified. Certification materials can be obtained from the National Data and Statistical Center. The Database Syllabus contains the CT data collection form with guidelines. Staff coding the CT forms need to be recertified every 10 years.

  5. Data Collected from Medical Charts: Each center will have an independent certified staff member re-abstract the entire Form 1 on one case entered each quarter. If any errors in coding are found they should be corrected and another one of the cases for the quarter should be re-abstracted and any errors corrected. This should be done prior to each quarterly submission deadline (e.g., March 31, June 30, September 30, and January 15).

  6. Form 1 Certification: Each center will have the staff responsible for completion of the Form 1 data collection for the TBIMS National Database complete the Form 1 Certification bi-annually. For new employees, contact the NDSC for materials to complete the certification process, otherwise, the NDSC will notify centers when to complete the bi-annual process.

  7. Cultural Certification: It is the responsibility of each center to assure that all staff responsible for enrollment, Form 1 Pre-Injury History collection or Form 2 Data collection complete the Cultural Certification provided in the “Cultural” section under the Members tab on the TBINDSC website.

  8. Data Entry: Data entry for a random sample of at least 10% of Form 1s will be verified by having a different staff member compare the complete data collection form to the data entered into the database. If any errors are detected, all forms should be verified. This should be done prior to each quarterly submission deadline. The same procedure will be used for Form 2 data entry if paper forms are used.

  9. Data Errors Discovered During Analysis: All data errors (or questionable data) identified during any analysis of data in the national database, by any center staff, will be reported to the NDSC for distribution and correction by the other centers.

  10. Data Collected by Interview: Each center, on an annual basis, should have another center staff person sit in on an interview (via phone, recording, or in-person) and code and compare a Form 2 for each Form 2 data collector. If any errors are found, they should be corrected and another interview should be coded by the independent staff person.

  11. Error Analysis Report: Each center should review the database Errors/Inconsistencies report after all data entry is completed for a quarterly submission deadline and attempt to correct all errors before that quarterly deadline. All cases containing errors at the time of submission will be excluded from distributed datasets.

  12. Coding Consistency: Each center should run the Errors/Inconsistencies report after all data entry tasks have been completed for a quarterly submission deadline and review all inconsistencies before that quarterly deadline. Centers should attempt to correct all inconsistencies where an error has occurred, and disregard any inconsistencies where an error has not occurred.

  13. Missing Data Reports: Each center should run the database Completeness reports for both Form 1 and Form 2 missing data after all data entry is completed for a quarterly submission deadline and attempt to complete all missing data before the quarterly deadline.

  14. Screening Report: Each center should verify that screening data entry is kept up to date throughout the quarter and prior to each quarterly submission deadline.

  15. Form 2’s Overdue: Each center should run the database Follow-Up Cases report after all data entry is completed for a quarterly submission deadline and complete a Form 2 for any cases appearing on the list as overdue on the Cases Overdue tab of the report. If no information is available on the participant, complete the Form 2 considering the participant as lost to follow-up. If any Form 2s are overdue for any quarterly submission deadline, they will be considered lost to follow-up in the calculation of follow-up rates for the data quality targets.

  16. Best Practices for Follow-up: Each center should complete the “Lost” tab in the database for every participant that is submitted to the national database as lost to follow-up (that is either they could not be located or they did not respond to contact) to assure that all best practices for follow-up have been completed. The Lost Cases: Guidelines and Strategies report should then be run to verify that each lost case has been updated.

  17. Data Quality Targets: Each center should attempt to meet the Data Quality Targets established by the TBIMS. Those targets are: 1) annually enroll the projected number of participants stated in their grant proposal; 2) enroll 80% of eligible participants each quarter and year; 3) successfully follow 90% of participants for the Form 2 year 1 and year 2 follow-ups each quarter and year (successful follow-up = those followed, expired or incarcerated); 4) successfully follow 80% of participants for the Form 2 year 5,10, 15, etc. follow-ups each quarter and year; 5) maintain less than 10% missing data on all Form 1 and Form 2 variables each year.

  18. Data Integrity Pledge: Each center should ensure that all TBIMS staff with data collection or data management duties, including screening and consenting sign the Data Integrity Pledge annually. Project Directors do not need to sign.

  19. Cost Sharing: Each Center is expected to attend and share in the cost of holding the Project Directors meetings and Data Collector Conferences. Each center will be held responsible for a share of the registration fees split appropriately across centers, whether the meeting is held in person, virtually or a combination of these formats, and independent of that center’s ability to attend.

  20. Guidelines Sign-off by Project Directors: On an annual basis, each center Project Director should submit a signed copy of this SOP to the NDSC, to indicate that these guidelines are being followed within his/her center. All signed guidelines will be archived by the NDSC.

Training requirements:

Data quality will be a continual topic of discussion both at the Project Directors Conference as well as Data Collectors in person conferences and quarterly data collector’s teleconferences.

Compliance:

All TBIMS centers will be asked to discuss data quality procedures conducted at their center with the NDSC during quality support visits.

References:

None

History:

Date Action
10/28/2008 Completely revised version from data committee used to create this SOP approved by Data Committee
11/01/2008 Transferred to SOP template
11/17/2008 Approved by SOP Review Committee
12/12/2008 Approved by Planning Committee and Project Directors
07/01/2009 Clarified what is considered “lost” for #16
10/01/2009 Clarified that a person conducting the re-abstraction should be certified
10/01/2009 Change re-abstraction rate from 10% to just one
10/01/2011 Removed item that discussed correcting errors identified by the NDSC prior to each quarterly data submission redundant with Error Reports item (#10)
10/01/2011 Eliminated requirement for having an independent staff member review 10% of data collection forms for coding consistency
10/01/2011 Added requirement for running coding consistency reports and reviewing all inconsistencies (#11)
01/01/2013 Updated #3 - Pre-Morbid to Pre-Injury; Updated #4 CT certification instructions to contact the NDSC instead of Santa Clara
04/01/2014 Updated #10 to include “All cases containing errors at the time of submission will be excluded from distributed datasets.”
08/21/2014 Removed statement that the Northern California TBIMS (Santa Clara Valley Medical Center) sends out reminders for DRS certification.
11/22/2016 Updated #4 to include certification requirement of every 10 years
11/22/2016 Added ITHealthTrack to method of FIM certification
11/22/2016 Change DRS certification to be data collectors and data quality checkers for Form1
10/01/2019 Added Cultural Certification requirement.
08/27/2020 Added Data Integrity Pledge requirement. Updated ITHealthTrack as the method of FIM certification. Updated instructions for Form 2 data entry checks.
04/09/2021 Updated Enrollment Report to Screening Analysis Report
04/09/2021 Removed Quarterly Report bullet
09/08/2022 Updated to reflect newer report names
09/12/2022 Removed “Interview” and “Questionnaire” from PreInjury items
06/05/2024 Updated FIM training and certification procedures, added bullet regarding requirement that each center share in the costs of Project Director meetings and Data Collector Conferences, updated names of various NDSC reports

Review schedule:

At least every 5 years.