Victor Polignano Godoy1, Fernanda Gomes da Mata2, Bárbara Romaneli Conde3, Caroline Antunes de Oliveira e Souza1, Ana Luiza Guimarães Martins1, Paulo Mattos4, Débora Marques de Miranda5, Leandro Fernandes Malloy-Diniz5
1 Federal University of Minas Gerais (UFMG); Research Laboratory of Clinical Neuroscience (LINC) of UFMG, Belo Horizonte, MG, Brazil.
2 Monash University, Melbourne, Australia.
3 Postgraduate Program in Neuroscience at UFMG, Belo Horizonte, MG, Brazil.
4 Psychiatry Institute of Federal University of Rio de Janeiro (UFRJ); CNA/D’or Institute, Rio de Janeiro, RJ, Brazil.
5 Department of Pediatrics of UFMG, Belo Horizonte, MG, Brazil.
Received: 7/5/2015 – Accepted: 6/11/2015
Address for correspondence: Leandro Fernandes Malloy-Diniz. Av. Alfredo Balena, 190 – 30130-090 – Belo Horizonte, MG, Brazil. E-mail: email@example.com
Background: Considering the importance of Executive Functions to clinical and nonclinical situations, Barkley proposed a new theory of executive functioning based on an evolutionary neuropsychological perspective and clinical research using large samples of clinical and community identified adults and children as well as children with ADHD followed to adulthood. Objective: The present study aims to adapt the Barkley Deficits in Executive Functions Scales (BDEFS) to Brazilian Portuguese and also assess its construct validity in a sample of normal Brazilian adults. Methods: The original version of scale was adapted to Brazilian Portuguese according to the guideline from the ISPOR Task Force. To assess the semantic equivalence between the original and adapted version, both of them were applied into a sample of 25 Brazilian bilingual adults. Finally, 60 Brazilian adults completed the BDEFS and the Brazilian versions of Barratt Impulsiveness Scale (BIS-11) and Adult Self-Report Scale (ASRS-18) to assess convergent validity. Results: The BDEFS Brazilian Portuguese version has semantic correspondence with the original version indicating that the adaptation procedure was successful. The BDEFS correlated significantly with the impulsivity and attention scores from the BIS-11 and ASRS-18 supporting its construct validity. Cronbach’s alpha (α = 0.961) indicated that the BDEFS translated version has satisfactory internal consistency. Discussion: Together, these findings indicate the successful adaptation of the BDEFS to Brazilian Portuguese and support its utility in that population.
Godoy VP et al. / Arch Clin Psychiatry. 2015;42(6):147-52
Keywords: Executives functions, neuropsychology, psychometrics, assessment scales.
Executive Functions (EF) are among the most important cognitive processes due to their role in complex cross-temporal behavior, adaptation to novel situations, problem solving, and decision-making. These cognitive processes are some of the most-studied constructs in neurosciences1,2. Several theoretical models have tried to explain both the structure and functioning of EF3. These models differ in terms of the number of processes thought to be involved (single or multiple processes) in the construct. Despite some divergence between different models of Executive Function, authors seem to agree that EF processes are pivotal for self-regulation and goal-directed behavior4,5.
Several psychopathologies are associated with impairments in EF processes, such as Attention Deficit Hyperactivity Disorder (ADHD)6, Bipolar Disorder7, Obsessive Compulsive Disorder8, and Schizophrenia9. In clinical samples, as in ADHD, EF deficits are related to poor functional outcomes, such as educational fields10. In non-psychiatric samples, EF seems to be related to others diseases and to everyday activities. Patients with increased severity of type 2 diabetes mellitus have lower performance in a working memory task and in a measure of general EF probably because chronic hyperglycemia may end in neural apoptosis and, as consequence, decreased the cognitive functioning. In addition, EF mediates the diabetes self-care11 (glucose monitoring, diet, medication administration, etc.). Moreover the executive functioning is related to driving behavior and the probability of traffic accidents12 and to productivity at work13.
Considering the importance of EF to clinical and nonclinical situations, Barkley proposed a new theory of executive functioning based on an evolutionary neuropsychological perspective and clinical research using large samples of clinical and community identified adults and children as well as children with ADHD followed to adulthood. According to Barkley14, executive functions could be understood by considering a hierarchic relationship between inhibitory control and other cognitive processes. Barkley argues that three inhibitory processes – (a) inhibition of the prepotent response; (b) stopping of an ongoing response; and (c) interference control – support higher abilities like working memory, internalization of speech, self-regulation of affect-motivation-arousal, reconstitution, and motor-control-fluency-syntax15. These cognitive abilities operate as tools for the performance of major activities of daily life since the beginning of humanity. That is because EF is proposed to mediate complex behaviors such as reciprocity and social exchange, social cooperation, vicarious learning, mimesis, and self-regulation for self-defense that underlie the evolutionary process of the human species and the development of our culture14.
Despite the plethora of neuropsychological tests designed to assess EF, there is evidence that many of them do not have ecological validity considering their lack of correlation with daily-life activities or observations and ratings of EF in natural settings6,16. In this context, Barkley created the Barkley Deficits in Executive Functioning Scale (BDEFS), which consists of a self-reported questionnaire that assesses different components of executive functioning related to daily-life impairments of psychiatric patients16. Since the publication of BDEFS, this scale has accumulated evidence of validity17,18, as well as utility in clinical practice for the assessment of psychiatric patients19,20. The present work presents the adaptation of the BDEFS to Brazilian Portuguese, and provides evidence of its reliability and construct validity in a non-clinical community sample of Brazilian adults.
Materials and methods
Two convenience samples were recruited through local advertisements and the only exclusion criteria was age (less than 18 years old and more of 89 years old). For the translation phase, 25 bilingual Brazilian adults (n = 25; 17 women) answered the both versions of BDEFS. The participants had an average age of 26.40 years old, with a minimum of 19 years old and a maximum of 41 (Standard Deviation = 5.515). All participants had at least 11 years of formal education.
For the pilot study of psychometric properties of the Portuguese Brazilian version of BDEFS, 60 Brazilian individuals were recruited (n = 60; 21 men), all of whom had at least eight years of formal education. The participants in this phase had an average age of 27.3 years old, with a minimum age of 18 years old and a maximum of 55 years old (Standard Deviation = 12.3 years).
Barkley Deficits in Executive Functioning Scale (BDEFS)
The self-report version of the BDEFS-LF is intended for adults ages 18 to 81. The patient completes the scale by indicating the frequency of certain behaviors and thoughts believed to reflect EF deficits. The ascertainment window for this self-assessment is the past six months, which should ensure a more stable measurement and representative sampling of the EF deficits over time than do typically short-duration cognitive tests. Answers to the 89 items comprise a four-point Likert-type scale, as follows: never or rarely (1), sometimes (2), often (3), and very often (4). The factor analysis of the BDEFS revealed a large, single factor along with four smaller factors. These became the five subscales that assess specific domains of EF in daily life: self-management of time (items 1 to 21), self-organization/problem solving (items 22 to 45), self-restraint (items 46 to 64), self-motivation (items 65 to 76), and self-regulation of emotion (items 77 to 89).
The Scoring System involves the computing individual scores of the five subscales; the Total Score across all five subscales; the Executive Functioning Symptom Count which indicates clinical symptoms of deficits in executive functioning (items answered with 3 or 4 are added to obtain this score); and the ADHD-EF Index (items 1, 6, 14, 16, 24, 49, 50, 55, 60, 65 and 69 have their scores added). The ADHD-EF Index is only intended as a predictor of the risk of having ADHD and is not diagnostic of it per se.
Barratt Impulsiveness Scale – Eleventh edition, Brazilian version (BIS -11)
The BIS-11 is a self-report instrument with 30 items for measure impulsivity of the Ernest Barratt model21. The respondents have to select the best option based in a four-point Likert type-scale to describe the frequency of certain behaviors and situations linked with impulsivity. The BIS-11 has three subscales: attentional impulsivity (items 6, 5, 9, 11, 20, 24, 26, 28), motor impulsivity (items 2, 3, 4, 16, 17, 19, 21, 22, 23, 25, 30) and non-planning impulsivity (items 1, 7, 8, 10, 12, 13, 14, 15, 18, 27, 29). Furthermore, the BIS-11 has a total score ranges from 30 to 120 points. Higher scores mean that the participant tends to behave impulsively.
Adult Self-Report Scale, Brazilian version (ASRS-18)
The ASRS-18 is a self-report measure of symptoms of the Attention-Deficit Hyperactivity Disorder (ADHD) based in the A diagnostic criteria of the Fourth Edition of the Diagnostic and Statistical Manual (DSM-4) adapted to adults22. The scale was composed by 18 questions: 9 for evaluate the attention-deficit symptoms and 9 for measure the hyperactivity-impulsivity symptoms. To be suspect of having ADHD is necessary at least 6 symptoms in one subscale or 12 symptoms in both (for the combined type). For clinical diagnosis is necessary fill the others criteria.
There are two main phases of the present study: (1) the transcultural adaptation and (2) internal consistency and construct validity. First, the BDEFS was translated into and adapted to Brazilian Portuguese, following the principles and steps recommended by the ISPOR Task Force for Translation and Cultural Adaptation21. The steps of translation and adaptation were (a) preparation: research design and obtaining authorization from Barkley and Guilford Press to translate and transculturally adapt the scale; (b) forward translation: translation for the Portuguese language; (c) reconciliation: comparison and synthesis of translations into Portuguese; (d) back translation: translation of the Portuguese version back into the English language; (e) review of back translation: comparison of the back-translated version with the original; (f) cognitive debriefing: testing of the translated version with a small sample; (g) review of cognitive debriefing results and finalization: interpretation of the results of cognitive debriefing; (h) proofreading: correction of any type of error; and (i) final report. Only the step of harmonization was not followed, because it is not applicable to this study. The comparison between the original version and the adapted version used the same procedure described in Malloy-Diniz et al.22,23. Namely, both original and translated versions of the BDEFS were administered to the sample of 25 bilingual Brazilian participants. The original scale was administered first. After seven days, the translated version was administered, with the time between administrations decreasing the possibility that the participants would remember the order of and their responses to the items.
In a second phase, relationships between the Brazilian versions of the BDEFS, the Adult Self Report Scale (ASRS 1822), and the Barrat Impulsiveness Scale (BIS 1121) were assessed. The three scales were applied in the sample of 60 Brazilian participants.
SPPS 21.0 software (Statistical Package for Social Sciences, version 21.0) was used to analyze the distribution of each item of the BDEFS Brazilian Portuguese version, finding that all the items did not follow normal distribution (p < 0.05) by the Shapiro-Wilk test (n < 50). This finding is typical of rating scales that assess symptoms or deficits. Accordingly, Spearman correlation was employed between the answers to the translated and the original instruments in order to verify the validity of the construct and the association between items. Finally, to assess the internal consistency of BDEFS Brazilian Portuguese version was calculated the inter-correlation among the items (the coefficient of Cronbach’s Alpha).
A Bland-Altman plot was constructed to analyze the agreement between the two versions of the BDEFS. This statistical method involves a scatter plot that has as the y-axis the difference between the two measures, with the x-axis representing the mean of the two measures. Upper and lower limits are established based on +-1.96 SD. If the bias has a normal distribution, 95% of the differences will be between the limits. A standard deviation of 1.96 is not clinically representative, so if the differences are within +-1.96 SD, the two measures correspond and may be used interchangeably.
Thirty items of the scale showed a strong correlation index (0.70 < rho < 1), fifty-five items presented a moderate correlation (0.30 < rho < 0.70), and three items showed a weak correlation (0.1 < rho < 0.3). Only one item showed no correlation between the original and Brazilian versions. All five correlations were strong (0.7 < rho < 1) between the five subscale scores of the translated and the original versions. Finally, the total score, the FE Count Symptoms, and the ADHD Index showed strong correlations (0.7 < rho < 1). These results point to a good correspondence between the original and translated BDEFS. These correlations are presented in table 1.
Cronbach’s Alpha was calculated in order to measure the internal consistency of the BDEFS Brazilian Portuguese Version for all scales and for each subscale, as done in the validity and normative study of the BDEFS English Version15. For all scales, Cronbach’s Alpha was 0.961 (α = 0.961). For each subscale – self-management of time, self-organization, self-restraint, self-motivation, and self-regulation of emotion – the results of Cronbach’s Alpha (α) were, respectively, 0.925, 0.920, 0.879, 0.790, and 0.970. These results are quite satisfactory and are close to those Barkley found in 2011 (0.949, 0.958, 0.930, 0.914, and 0.946, respectively). These results indicate that the BDEFS translated version has satisfactory internal consistency.
To build the Bland-Altman plot, the total score of both BDEFS were used to compute the differences and the means for each participant. Next, the standard deviation was calculated for the sample (SD = 9.65091). The mean of differences between the total scores on the BDEFS Brazilian Portuguese Version and the total scores on the BDEFS English Version was -0.16 (m = -0.16), and the maximum differences were -4.1437 and 3.8237. Finally, the SD was multiplied by +-1.96, added to the mean of differences (-0.16), and the upper and down limits were thereby discovered. A strong relationship was expected between the scales because, in theory, they measure exactly the same construct. Based on this statistical information, the Brand-Altman plot (Figure 1) shows that all the results are within the 95% confidence range, indicating that the two versions have good clinical agreement.
The next step was to correlate the five subscales, the indexes, and total score of BDEFS with the two factors (inhibition response and lack of planning) and the total score of BIS 11 and with the two factors (inattention and hyperactivity), the total score, and the total symptoms of ASRS 18. The results are shown in tables 2 and 3.
Overall, the results of the Spearman correlations between the BDEFS Brazilian Portuguese version, BIS 11, and ASRS 18 indicate satisfactory concurrent validity.
The present study reports evidence that our transcultural adaptation of the BDEFS to Brazilian Portuguese was quite satisfactory and successful. Furthermore, the adapted version of the BDEFS showed evidence of concurrent validity when compared with two other instruments (BIS 11 and ASRS 18), which measure different types of impulsive behavior, attention, and hyperactivity. The results also suggested that the internal consistency of the BDEFS Brazilian Portuguese was not only satisfactory, but also straightforward in its relationship to the corresponding data for the BDEFS English version. The subscale “self-regulation of emotion” of the BDEFS Brazilian Portuguese has a Cronbach’s Alpha higher than the same statistical procedure for the BDEFS English version. This is also strong evidence of the correspondence and validity between the versions.
To validate the original scale, Barkley and colleagues did two studies (UMASS and Milwaukee Studies; Barkley16) using several neuropsychological tests of executive and other functions. Those studies found a low correlation between the scores of the BDEFS English Version and the scores of these tests, consistent with many other studies. This absence of a strong correlation between ratings and tests reflects a lack of ecological validity of neuropsychological tests. Other studies using the BDEFS English Version have shown similar results17-20,24,25. For example, as found by Vasconcelos et al.26 the BIS 11 Brazilian Portuguese Version presents almost non-significant correlations with the Iowa Gambling Task, which support the theoretical and statistical difference between these types of neuropsychological methods. Due to the frequently reported weak correlation between neuropsychological tests and scales, in the present study we opted to use scales measuring similar constructs (i.e., ASRS-18 and BIS-11) and similar methods to assess the convergent validity of BDEFS. Even though the constructs measured by BIS-11 (impulsivity), ASRS-18 (ADHD’s cardinal symptoms), and BDEFS (executive functioning) are not equivalent, they are hypothesized by Barkley to be substantially related to each other. Poor inhibition (self-restraint) is a major EF component in Barkley’s and others’ theories of EF. Thus Barkley has argued that ADHD is primarily a disorder of EF beyond simply being one of inattention. Supporting these theoretical positions, we found significant correlations between the scores derived from those scales. This result suggests that the Brazilian adaptation of the BDEFS presents evidence of construct validity.
Other studies using the BDEFS English Version have made important contributions regarding the validity and the potential for use of the scale. Dehili et al.20 analyzed the capacity of BDEFS to predict symptoms of ADHD in comparison to a visual-search task in a college population. The results showed that the BDEFS is more predictive of ADHD than the task. Also, the relationship between the two measures was found to be low (just as in the studies of Barkley). Knouse25 used the BDEFS successfully to examine the relationship of EF to understand and predict the mediation of self-regulation in academic performance. That study19 also showed that the BDEFS could be used to predict depressive symptoms, especially using the self-management of time, self-organization, and problem-solving subscales, in a sample with symptoms of ADHD and depression. Future studies should address the relationship between the Brazilian adaptation of BDEFS and functional outcomes in a Brazilian context.
The results of the present study should be interpreted in the context of its limitations. One such limitation was that we assumed that the factorial structure of the adapted BDEFS was the same as the original version and did not examine that factor structure directly. Because our sample was small, factor analysis could not be performed. This study was executed only for adapt the BDEFS for our culture and, for that objective, the sample could be small. Future studies should assess the factor structure of BDEFS in the Brazilian Population using both confirmatory and exploratory methods. These statistical analysis and the standardization of the BDEFS implies in a recruitment of a larger sample than the one used in the present study.
The null correlation between the item 69 of the both versions of BDEFS probably can be explained for cultural differences. Although, in the future studies of the standardization of BDEFS Portuguese version, this result needs to be investigated more deeply.
It is important to note that possible clinical use of this adaptation of BDEFS depends on future studies of criterion validity in clinical populations, as well as on factor analytical procedures, and the collection of additional norms for a Brazilian population that is representative of different regions, other important demographic factors, and socioeconomic levels. Yet the evidence found here suggests that such future studies could rely on this adaptation.
The authors wish to thank Russell Barkley, Ph.D. for his availability to serve as a consultant to this project, for his review of an earlier draft of this manuscript, and for authorization to translate his scale for use in this research.
Conflict of interest
None of the authors has any conflict of interest. This research doesn’t have any financial support.
1. Diamond A. Executive functions. Annu Rev Psychol. 2013;64:135-68.
2. Verdejo-García A, Bechara A. Neuropsicología de las funciones ejecutivas. Psicothema. 2010;22(2):227-35.
3. Barkley RA, editor. Attention-deficit hyperactivity disorder: a handbook for diagnosis and treatment. New York: Guilford Publications; 2014.
4. Barkley RA, Murphy KR. Impairment in occupational functioning and adult ADHD: the predictive utility of executive function (EF) ratings versus EF tests. Arch Clin Neuropsychol. 2010;25(3):157-73.
5. Fuster JM. The prefrontal cortex – an update: time is of the essence. Neuron. 2001;30(2):319-33.
6. Barkley RA, Murphy KR. The nature of executive function (EF) deficits in daily life activities in adults with ADHD and their relationship to performance on EF tests. J Psychopathol Behav Assess. 2011;33(2):137-58.
7. Malloy-Diniz LF, Neves F, Corrêa H. Neuropsychological aspects of bipolar disorder. Rev Bras Psiquiatr. 2009;31(2):183-5.
8. da Rocha FF, Alvarenga NB, Malloy-Diniz L, Corrêa H. Decision-making impairment in obsessive-compulsive disorder as measured by the Iowa Gambling Task. Arq Neuropsiquiatr. 2011;69(4):642-7.
9. Lepage M, Bodnar M, Bowie CR. Neurocognition: clinical and functional outcomes in schizophrenia. Can J Psychiatry. 2014;59(1):5-12.
10. Biederman J, Monuteaux MC, Doyle AE, Seidman LJ, Wilens TE, Ferrero F, et al. Impact of executive function deficits and attention-deficit/hyperactivity disorder (ADHD) on academic outcomes in children. J Consult Clin Psychol. 2004;72(5):757-66.
11. Gatlin PK, Insel KC. Severity of type 2 diabetes, cognitive function, and self-care. Biol Res Nurs. 2015;17(5):540-8.
12. Roca J, Crundall D, Moreno-Ríos S, Castro C, Lupiáñez J. The influence of differences in the functioning of the neurocognitive attentional networks on drivers’ performance. Accid Anal Prev. 2013;50:1193-206.
13. Ownsworth T, Shum D. Relationship between executive functions and productivity outcomes following stroke. Disabil Rehabil. 2008;30(7):531-40.
14. Barkley RA. The executive functions and self-regulation: an evolutionary neuropsychological perspective. Neuropsychol Rev. 2001;11(1):1-29.
15. Barkley RA.Behavioral inhibition, sustained attention, and executive functions: constructing a unifying theory of ADHD. Psychol Bull. 1997;121(1):65-94.
16. Barkley RA. Barkley deficits in executive functioning scale (BDEFS). New York: Guilford Press; 2011.
17. Coffman TP. The Psychometric Properties of the Barkley Deficits in Executive Functioning Scale (Bdefs) in a College Student Population. Florida State University; 2014.
18. Kamradt JM, Ullsperger JM, Nikolas MA. Executive function assessment and adult attention-deficit/hyperactivity disorder: tasks versus ratings on the Barkley deficits in executive functioning scale. Psychol Assess. 2014;26(4):1095-105.
19. Knouse LE, Barkley RA, Murphy KR. Does executive functioning (EF) predict depression in clinic-referred adults? EF tests vs. rating scales. J Affect Disord. 2013;145(2):270-5.
20. Dehili VM, Prevatt F, Coffman TP. An Analysis of the Barkley Deficits in Executive Functioning Scale in a College Population: Does It Predict Symptoms of ADHD Better Than a Visual-Search Task? J Atten Disord. 2013.
21. Malloy-Diniz LF, Mattos P, Leite WB, Abreu N, Coutinho G, De Paula JJ, et al. Tradução e adaptação cultural da Barratt Impulsiveness Scale (BIS-11) para aplicação em adultos brasileiros. J Bras Psiquiatr. 2010;59(2):99-105.
22. Mattos P, Segenreich D, Saboya E, Louzã M, Dias G, Romano M. Transcultural adaptation of the Adult Self-Report Scale into Portuguese for evaluation of adult attention-deficit/hyperactivity disorder (ADHD). Rev Psiquiatr Clín. 2006;33(4):188-94.
23. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94-104.
24. Nogueira CYS, Carvalho AM, Gauer G, Tavares N, Santos RMM, Ginani G, et al. Translation and adaptation of Impulsive Behavior Scale (UPPS) to the Brazilian population. Clinical Neuropsychiatry. 2013;10(2):79-85.
25. Knouse LE, Feldman G, Blevins EJ. Executive functioning difficulties as predictors of academic performance: examining the role of grade goals. Learn Individ Differ. 2014;36:19-26.
26. Vasconcelos AG, Sergeant J, Corrêa H, Mattos P, Malloy-Diniz L. When self-report diverges from performance: the usage of BIS-11 along with neuropsychological tests. Psychiatry Res. 2014;218(1-2):236-43.