Last month, we discussed the significance of patient tracking and registry capture to improve data quality in clinical orthopaedic registries. Ensuring that all patients are captured within the registry, and verifying that the information stored for them is correct is just one component of improving registry quality. A second equally important component is ensuring that the actual data that is recorded throughout the preoperative to postoperative cycle is validly collected and correctly stored. Patient drop-out and missing data are common sources of bias in longitudinal clinical studies [1,2], and if the data cannot be assessed or evaluated over time, there stands little reason for collecting it in the first place.
Data entry: The patient factor
Data entry into clinical registries occurs at several levels: it is self-reported by the patients in the form of patient-reported outcome measures (PROMs), it is collected by healthcare providers such as nurses, anesthetists and physiotherapists, and it is entered into the registry by administrative staff, resident medical officers and the consulting physician. The transcription of data as it changes hands and moves across systems can be routinely monitored and rectified. What cannot be fixed however, is if the patient-reported data is incomplete or incorrect.
Let's take a look at data collected for a rotator cuff injury of the shoulder. One of the validated PROMs questionnaires is the Western Ontario Rotator Cuff Index (WORC), which requires the patient to submit their responses on a visual analogue scale. An example of one of the questions is as follows:
The type of response required from the patient is shown above in red. At most, the patient is allowed to mark a spot on the horizontal line with a cross, instead of a vertical line. It is not uncommon however, to receive invalid responses in the form of circles, statements, crossed-out phrases and horizontal lines indicating a threshold of pain:
An invalid response is the equivalent of no response at all - it cannot be entered into the registry, or quantitatively analysed. Electronic data capture is one solution to overcoming these types of problems, and more clinicians are slowly embracing electronic processes over traditional paper forms. But there remains limited guidance with respect to electronic data capture and the factors that can be monitored (and thus subsequently modified) to improve patient compliance with PROMs questionnaires.
An important question that arises with improving PROMs compliance is: Can we predict which patients return incomplete data? And can this information be used to allow researchers to refine their current recruitment and data collection procedures to improve registry data quality? To see if we could answer these, we investigated PROMs compliance for 2000 patients in a quality-controlled clinical registry we planned and implemented for one of our clinical partners.
Registry snapshot
The registry comprised of thirteen observational, prospective cohort studies of upper limb pathologies. Patients eligible for intervention were recruited to the registry and asked to complete electronic forms via a link sent to them by email, or at their clinic visit with a tablet prior to their treatment. Patients were classified as Shoulder, Elbow or Hand-Wrist relative to their presenting pathology, and those that missed at least one questionnaire were classified as Missing. The dataset of 1997 patients was extracted from the registry database (Socrates v3.5, Ortholink Pty Ltd, Australia) and a binary logistic regression model was developed to link patient factors (Age, Gender, Type of Presentation, Date of Presentation) and injury factors (Side, Joint) with the Missing classification.
Predicting compliance
A dataset of 1997 patients was extracted from a registry database (Socrates v3.5, Ortholink Pty Ltd, Australia) and a binary logistic regression model developed to link patient factors (Age, Gender, Type of presentation, Date of Presentation) and injury factors (Side, Joint) with the Missing classification.
We found a significant association between PROMs compliance and which joint the patient presented for assessment (Table 1). Specifically, Elbow and Hand-Wrist patients were at lower risk of partial compliance with the PROMs pack compared to shoulder patients. This could be attributed to the lower number of questionnaires in total for these patients (typically 2-3), compared to the shoulder patients (between 4-6). More recent date of examination was also associated with a significant, but small, improvement in compliance that was independent of other patient factors, which may reflect the evolution in processes to capture PROMs over time for this particular registry.
However, there was a lack of significant association with any other patient factors, indicating that are still unknown variables that need to accounted for which will better help establish the likelihood of a patient successfully completing a set of PROMs via electronic data capture.
Improving compliance
Our findings point to a few key things that should be considered to improve patient compliance:
The number of questionnaires matters. Elbow and Hand-Wrist patients were at lower risk of partial compliance not because of their pathology, but because they had half the number of questionnaires for Shoulder patients. This comes as no surprise - filling out a seemingly endless set of forms with questions like "How much sharp pain do you experience in your shoulder?" followed by "How much constant, nagging pain do you experience in your shoulder?" is both tedious and time-consuming. Which brings us to the next point...
The amount of time patients have to complete the questionnaires is also relevant. We make this inference from the significant association between a later date of examination and improved compliance, underscored by changes in the practice and registry workflows over time. Since the implementation of the registry, the lead time for patients being sent their scores before treatment has increased, giving them more time to complete the questionnaires. Patients who miss questionnaires prior to surgery are also flagged and identified to clinic staff, who then have the opportunity to chase them up when they arrive for their treatment.
A third point worth mentioning is that scores for Shoulder patients are delivered to them at staggered time points. Elbow and Hand-Wrist patients however, get all the questionnaires once, upfront. It is possible that when Shoulder patients receive the later questionnaires, they are under the impression that they don't need to complete them (as they've already filled some out before), or lose interest in them. With that in mind, it is intuitively better to deliver the questionnaires together where possible, but we need more evidence to back that up.
Another report has additionally identified that being younger, being a new patient and having an English-speaking background as well as having a longer wait time were associated with a higher PROMs response rate, whereas gender, pathology and the type of PROMs were not [3]. A comprehensive understanding of the factors that affect patient compliance with PROMs can help inform registry workflow and processes to maximise patient contributions, and thus improve registry data quality.
While further work remains to fully elucidate these factors and their impact, a first modifiable step is the registry design itself, taking into account the classification of cohorts, required questionnaires and data collection time points. To date, EBM Analytics has successfully designed and implemented five clinical orthopaedic registries, one of which has recently undergone two expansions. Our experience enables us to advise our clinical partners of the best way to structure their cohorts, design core data sets that are evidence-backed, minimise the research footprint on existing clinical activities, and maximise their registry output. Let us know what is important for you to get out of your registry, and we'll let you know how we can make that happen.
References
Powney M, et al., Trials. 15:237-248, 2014.
Kristman V, et al., European Journal of Epidemiology. 19(8): 751-760, 2004.
Hi A, et al., Patient Related Outcome Measures. 10:217-226, 2019.