AKT Description

The AKT is an instrument with evidence of validity and reliability of construct developed by Obamiro et al.^[14]. It measures a patient’s knowledge of her/his anticoagulant therapy through self-administered questions and calculated scores. The AKT has 28 items (open-ended and multiple-choice questions) divided into two sections (A and B) with a maximum score of 35 points and 25 points, respectively, for VKA users and DOAC users. This difference between scores is because section B of the tool is exclusive dedicated for VKA users. Section B has eight items and a maximum score of 10 points. Section A (applied to all OAC users - both VKA and DOAC) presents 20 items and has a maximum score of 25 points. This section covers general questions about anticoagulants. To each given answer, a zero (incorrect answer) or a one (correct answer) is attributed by the clinicians to assess a patient’s knowledge^[14].

Cultural and Linguistic Validation

We performed a culturally acceptable translation of AKT into Brazilian Portuguese. This step followed the International Society for Pharmacoeconomics and Outcomes Research (or ISPOR) recommendations^[17]; it involved researchers to translate, back translate, apply, and validate the instrument.

A direct translation from English to Portuguese was performed by two independent Portuguese mother language translators (V.F, K.S), both with previous knowledge about anticoagulation. The consensus of this stage was reached through a discussion panel involving the two translators and the key-country consultant (R.P) and the project manager (F.M). The back-translation process was performed independently by two other translators (A.F, F.T), both without previous knowledge of the AKT, leading to a literal translation of the document. The consensus of this second stage occurred through a discussion panel involving R.P and F.M that compared the back translation with the original tool to finally obtain the AKT-Br version (see Supplementary Material).

Quantitative validity of the AKT-Br was achieved through the content validity index (CVI) and followed the minimum recommendations described by Lawshe^[18] and Lynn^[19]. This phase involved a panel, with five anticoagulation experts, to obtain a level of agreement on the tool items. The clarity of the text, relevance, and quality of the back translation of each item were discussed. Items with a CVI < 1 were reassessed, deleted, or replaced. The clarity and relevance of the items were assessed using a four-point Likert scale (1 = not clear/not relevant; 4 = highly clear/highly relevant) and the back translation correspondence regarding the original instrument (1 = does not match; 4 = totally matches)^[19]. The CVI was calculated using two approaches (item level [I-CVI] and scale level [S-CVI]), considering the average of the scores of the I-CVIs and error scale^[20]. The I-CVI is the index that expresses the proportion of agreement among the evaluators for a given item and, according to Lynn^[19], in a panel of “five or fewer specialists, everyone must agree with the validity of the content for its classification to be considered a reasonable representation of the universe of possible classifications”. Any result < 100% agreement (< 1) deserves due attention and possible reassessment until reaching unanimous agreement among the evaluators. The S-CVI is equivalent to the average percentage of agreement between the items and, according to Waltz et al.^[21], the recommended standard acceptability criterion is 0.90.

Assessment of Psychometric Proprieties

A cross-sectional study with convenience sampling was performed, following the protocol conducted in the original AKT^[14]. The samples were obtained in a non-probabilistic manner for convenience sampling of three groups with different levels of knowledge about OAC: pharmacists (the pharmacist group), users of oral anticoagulants (the patient group), and the general population (the population group). Pharmacists were eligible to participate if they had at least one year of clinical experience or worked in community pharmacies. Patients were eligible if there were over 18 years of age and used any OAC. Eligible individuals from the general population were over 18 years old who were not receiving treatment with OAC or did not have relatives or close friends undergoing treatment with these therapies. Participants were informed of the anonymity and confidentiality of their responses. To participate in the study, participants needed to provide written informed consent. The exclusion criteria for patients were as follows: incomplete questionnaire completion and OAC treatment duration < 3 months. For illiterate patients or those with reading difficulties, the questions and answers options were read by the interviewers exactly as written, minimising possible bias, while for the others the questionnaires were self-administered^[22].

The questionnaires were applied from September 2019 to January 2020. Pharmacists were interviewed using an electronic form, while the general population was interviewed under the supervision of an interviewer; both groups were asked to respond based on their knowledge on anticoagulation. For the patient group, the questionnaires were applied in a tertiary hospital in the South Region of Brazil (Curitiba, Paraná) using the printed version of the AKT-Br and under the supervision of an interviewer.

Statistical Analysis

The original 28-item structure of the AKT was maintained for all analyses^[14]. Descriptive statistical analyses were performed to describe the characteristics of the sample. The variables age, treatment, gender, and educational level were considered non-normally distributed. The results are presented as relative and absolute frequencies for categorical variables and as the median and the interquartile range (IQR) (presented in square brackets) for continuous variables. Contrasted group analysis was conducted to assess differences in the mean scores between the three groups. One-way analysis of variance (ANOVA) with Tukey’s post hoc analysis was used to explore statistical differences^[23].

Statistical tests that assess internal consistency and test-retest reliability are commonly used in self-administered instruments to ensure the reliability of the instrument^[24]. The evaluation of internal consistency was performed through Cronbach’s alpha coefficient that uses a scale of 0 to 1, where values close to 0.7 are considered acceptable and values > 0.9 are redundant^[25,26]. In addition, for instruments with more than 15 items it is recommended to apply the interitem correlation^[27]. The test-test reliability was obtained by reapplying the test to the same group, considering an appropriate time interval (14 days). Reliability coefficients between 0.7 and 0.8 (on a scale of 0 to 1) are considered acceptable^[24]. The level of significance of each test was set at 0.05 (two-tailed). All statistical analyses were conducted using RStudio (version 1.3.1073).

Ethics

The research was conducted within the standards required by the Declaration of Helsinki and approved by the Ethics Committee of the Hospital de Clínicas of the Universidade Federal do Paraná (Curitiba, Paraná, Brazil) under registration number CAAE: 16858719.1.0000.0096.

Cultural and Linguistic Validation

The final version of the AKT-Br (see Supplementary Material) was obtained through the consensus discussion of five anticoagulation specialists. Two hundred people were invited to participate in the study (75 pharmacists, 75 patients, and 50 people from the general population), of whom 148 met all eligibility criteria and were included for statistical analyses (55 pharmacists, 57 patients, and 36 people from the general population). Of this sample, 96 were women (64.9%), with a median age of 36 [IQR 28-53] years. About 67% of patients had used OAC for > 2 years (see Table 1).

Table 1 - Demographic characteristics of the sample.

Quantitative and Qualitative Content Validity

The unilateral ANOVA did not reveal a correlation between the total score and age (F=4.8365; P<0.001), but there were correlations for gender, education, and group (F=10.0121, 11.0706, 172.0956, respectively; P<0.001), where women, individuals with higher education, and pharmacists presented the highest scores.

The evaluated items of the tool presented I-CVIs ranging from 0.7 to 1 and an S-CVI of 0.92 (see Table 2). Cronbach’s alpha for the pharmacist and patient groups were 0.71 and 0.65, respectively. The test-retest reliability resulted in r=0.99 (P<0.001). The textual analysis showed that the AKT-Br presents clear and relevant items, without the need for further modifications.

Table 2 - Content validity index.

Construct Validity

There were significant differences between the scores of the three evaluated groups (N=148) in the construct validity through the analysis of contrasted groups (Figure 1). The total mean scores of the pharmacist group, patient group, and population group were 29 [25-32], 17 [14-21], and 7.5 [3.75-10.25], respectively (F=209.49, P<0.001). Tukey’s post-hoc test showed significant differences for all comparisons (P<0.001).

Fig. 1 - Total score comparison between groups/Tukey’s post hoc test showed significant differences (P<0.001) in comparing the means of all the groups (pharmacists vs. patients; pharmacists vs. population; patients vs. population).

Download

Fig. 1 - Total score comparison between groups/Tukey’s post hoc test showed significant differences (P<0.001) in comparing the means of all the groups (pharmacists vs. patients; pharmacists vs. population; patients vs. population).

Limitations

The results obtained with the AKT-Br tool highlight the validity of its translation and cross-cultural adaptation to Brazilian Portuguese. Nonetheless, our study has some limitations. First, it lacks a criterion validity using a measure of adherence (e.g., Morisky score) and testing a priori framework, where patients with higher adherence should be those with greater knowledge. The data from the group of patients was obtained from a tertiary hospital in Curitiba, Paraná (South Region of Brazil), who may not reflect the cultural features of the entire country nor be representative of the Brazilian population taking OAC. Further analyses can be performed in other regions to guarantee the validity of the tool. In addition, some educational limitations from the population led the interviewer to assist in the interpretation of the questions, a factor that may generate bias. In addition, 90% (n=50) of the evaluated patients were warfarin users. This may imply a possible selection bias, which is compatible with the convenience sampling or a need to a readdress the management strategy of Brazilian patients prescribed OAC. This factor may also limit the assessment of data from patients only using DOAC. The main objective of the study was to validate the AKT-Br as a psychometric analysis tool. However, clinicians can also benefit from the development of a “scale of knowledge” according to the obtained scores. For example, to minimise adherence issues and OAC adverse events, patients who score between five and 10 points (out of 35) probably need to receive different educational interventions compared to those who score 20-25 points. This is especially important in regions or countries with greater socioeconomic inequalities.