Journal of Personality Assessment

ISSN: 0022-3891 (Print) 1532-7752 (Online) Journal homepage: http://www.tandfonline.com/loi/hjpa20

Disentangling Gratitude: A Theoretical and

Psychometric Examination of the Gratitude
Resentment and Appreciation Test–Revised Short
(GRAT–RS)

Joseph H. Hammer & Rachel E. Brenner

To cite this article: Joseph H. Hammer & Rachel E. Brenner (2017): Disentangling
Gratitude: A Theoretical and Psychometric Examination of the Gratitude Resentment and
Appreciation Test–Revised Short (GRAT–RS), Journal of Personality Assessment, DOI:
10.1080/00223891.2017.1344986

To link to this article: http://dx.doi.org/10.1080/00223891.2017.1344986

Published online: 14 Jul 2017.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=hjpa20

Download by: [Cornell University Library] Date: 14 July 2017, At: 09:13
JOURNAL OF PERSONALITY ASSESSMENT
https://doi.org/10.1080/00223891.2017.1344986

Disentangling Gratitude: A Theoretical and Psychometric Examination of the Gratitude

Resentment and Appreciation Test–Revised Short (GRAT–RS)
1 2
Joseph H. Hammer and Rachel E. Brenner
1
Department of Educational, School, and Counseling Psychology, University of Kentucky; 2Department of Psychology, Iowa State University

ABSTRACT ARTICLE HISTORY

This study extended our theoretical and applied understanding of gratitude through a psychometric Received 13 February 2017
examination of the most popular multidimensional measure of gratitude, the Gratitude, Resentment, and Revised 1 June 2017
Appreciation Test–Revised Short form (GRAT–RS). Namely, the dimensionality of the GRAT–RS, the model-
based reliability of the GRAT–RS total score and 3 subscale scores, and the incremental evidence of validity
for its latent factors were assessed. Dimensionality measures (e.g., explained common variance) and
confirmatory factor analysis results with 426 community adults indicated that the GRAT–RS conformed to
a multidimensional (bifactor) structure. Model-based reliability measures (e.g., omega hierarchical)
provided support for the future use of the Lack of a Sense of Deprivation raw subscale score, but not for
the raw GRAT–RS total score, Simple Appreciation subscale score, or Appreciation of Others subscale
score. Structural equation modeling results indicated that only the general gratitude factor and the lack of
a sense of deprivation specific factor accounted for significant variance in life satisfaction, positive affect,
and distress. These findings support the 3 pillars of gratitude conceptualization of gratitude over
competing conceptualizations, the position that the specific forms of gratitude are theoretically distinct,
and the argument that appreciation is distinct from the superordinate construct of gratitude.

We can only be said to be alive in those moments when our hearts Others suggest that gratitude manifests itself not as a singular con-
are conscious of our treasures. struct, but as independent-yet-related forms of gratitude (Diessner
—Wilder (2007, p. 149)
& Lewis, 2007; Thomas & Watkins, 2003). This corresponds to an
Gratitude is a general disposition toward acknowledging and oblique (i.e., correlated, common factors) model. In turn, the three
appreciating the positive in the world (Wood, Froh, & pillars of gratitude conceptualization suggests that gratitude simul-
Geraghty, 2010). Distinguished from future-focused emotions, taneously exists in three specific forms but also as a superordinate
such as optimism and hope, feelings of gratitude are focused on construct (Watkins, 2009, as cited in Watkins, 2014; Watkins,
the present moment and without expectation for the future. Woodward, Stone, & Kolts, 2003). The superordinate gratitude
The extant literature underscores the benefits of gratitude, construct can be conceptualized as a general factor that exists inde-
which has been consistently linked to greater well-being, life pendently of its specific forms (i.e., operationalized by a bifactor
satisfaction, self-esteem, positive affect, and healthier relation- model) or as a higher order factor that is defined by the common
ship functioning (Bernstein & Simmons, 1974; see Wood et al., element that runs through all three specific forms (i.e., operational-
2010, for a review). Gratitude could also hold clinical relevance, ized by second-order model). A compelling psychometric rationale
having demonstrated associations with reduced depressive for preferring one of these four theoretical structures over the
symptomology (Disabato, Kashdan, Short, & Jarden, 2016; others remains unarticulated. In addition, embedded within this
Fredrickson, Tugade, Waugh, & Larkin, 2003; Wood, Maltby, debate is the question of whether or not appreciation (sometimes
Gillet, Linley, & Joseph, 2008; Van Dusen, 2014), suicidal idea- considered a specific form of gratitude) is distinct from the superor-
tion and past suicide attempts (D. Li, Zhang, Li, Li, & Ye, dinate construct of gratitude (Fagley, 2012; Manela, 2013; Wood,
2012), and posttraumatic symptoms (Van Dusen, Tiamiyu, Maltby, Stewart, et al., 2008). Beyond the appreciation debate, there
Kashdan, & Elhai, 2015). Moreover, longitudinal studies sug- is also continued disagreement on whether specific forms of grati-
gest a causal relationship wherein gratitude predicts depressive tude are theoretically distinct (Wood, Maltby, Stewart, et al., 2008)
symptoms over time (e.g., Disabato et al., 2016; Van Dusen, or provide incremental clinical utility beyond the general gratitude
2014; Wood, Maltby, Gillet, et al., 2008). factor (Fagley, 2012).
There is ongoing debate about the theoretical structure of grati- Intertwined with these questions regarding gratitude theory
tude. Some scholars have argued that gratitude is best conceptual- and clinical utility are questions about the reliability and valid-
ized as a unitary construct (McCullough, Emmons, & Tsang, 2002; ity of gratitude assessments. One of the most widely used multi-
Wood, Maltby, Stewart, & Joseph, 2008). In factor analytic terms, dimensional measures of gratitude is the Gratitude Resentment
gratitude would be operationalized with a unidimensional model. and Appreciation Test–Revised Short form (GRAT–RS;

CONTACT Joseph H. Hammer [email protected] Department of Educational, School, and Counseling Psychology, University of Kentucky, 243 Dickey Hall,
Lexington, KY 40506.

© 2017 Taylor & Francis

2 HAMMER AND BRENNER

Thomas & Watkins, 2003). The GRAT–RS was designed to GRAT–RS subscale scores have demonstrated moderate to
operationalize the three pillars of gratitude (Thomas & high intercorrelations (i.e., r D .25–.83; Magno & Orillosa,
Watkins, 2003; Watkins, 2009; Watkins et al., 2003) via three 2012). This shared variance among the subscales suggests that
GRAT–RS subscales. Lack of a Sense of Deprivation (LOSD) the GRAT–RS might be defined by a strong general factor that
purports to measure the absence of the belief that life has is distinct from the GRAT–RS subscale factors. A second-order
treated oneself unfairly. Simple Appreciation (SA) purports to factor model, however, would specify three first-order factors
measure the appreciation of simple day-to-day pleasures. (LOSD, SA, and AO) and one second-order gratitude factor
Appreciation of Others (AO) purports to measure the recogni- that is defined by the shared variance of LOSD, SA, and AO. In
tion of the importance of appreciating and expressing apprecia- other words, the second-order gratitude factor only measures
tion of the contribution that others have in their lives. The what the three pillars of gratitude share in common, rather
developers contend that the GRAT–RS subscale scores reliably than merely being (a) an amalgam of the entire content
and validly measure their respective forms of gratitude, and the domains of the three forms of gratitude, or (b) an independent
GRAT–RS total score reliably and validly measures the super- general gratitude factor distinct from the three specific forms.
ordinate gratitude construct (Thomas & Watkins, 2003). How- In contrast, a bifactor model specifies one general gratitude
ever, moderate to strong correlations observed among the three factor reflecting the common variance of all items as well as
GRAT–RS subscales, consistent with the theoretical debate three separate, narrower specific factors (LOSD, SA, and AO)
regarding the extent to which facets of gratitude are distinct, reflecting the variance of their assigned items. That is, each
indicate that the dimensionality of the GRAT–RS, the model- item simultaneously loads on the general factor and its assigned
based reliability of its total and subscale scores, and incremental specific factor. The general and specific factors are first-order
evidence of validity for its latent factors require additional factors set orthogonal to each other. Thus, the three specific
investigation through a bifactor analysis lens. factors account for variance in their respective items after parti-
In summary, there are outstanding questions about the theo- aling out the variance accounted for by the general factor.
retical structure of gratitude; the distinctness of appreciation A bifactor CFA and ancillary bifactor measures (see
from gratitude; the incremental clinical utility of the specific Hammer & Toland, 2016a, for a video walkthrough) allow
forms of gratitude: and the dimensionality, reliability, and researchers to answer unresolved questions about the
validity of the GRAT–RS scores. Therefore, using promising dimensionality and model-based reliability of instrument
but underutilized methods of bifactor analysis and ancillary scores (Reise, Bonifay, & Haviland, 2013). However, the possi-
bifactor measures, this study sought to address these theoretical bility of a second-order or bifactor solution for the GRAT–RS
and applied questions. has not been investigated. This study examined the dimension-
ality of the GRAT–RS by testing four alternative models (e.g.,
unidimensional, three-factor oblique, second-order, bifactor)
Dimensionality of the GRAT–RS
and examining ancillary bifactor measures pertaining to
Thomas and Watkins (2003) derived the 16-item GRAT–RS dimensionality. Determining which of these models best fits
from the original three subscales of the 44-item Gratitude the GRAT–RS data will help provide a psychometric rationale
Resentment and Appreciation Test (GRAT; Watkins et al., for preferring one theoretical structure over the rest. If a bifac-
2003). Froh and colleagues (2011) subjected the GRAT–RS to tor solution provides the best fit, this would also provide an ini-
an exploratory factor analysis (EFA) and confirmatory factor tial affirmative answer to the question of whether appreciation
analysis (CFA) using a sample of 681 college-enrolled adults. (operationalized in two forms via the SA and AO specific fac-
After dropping one item, Froh and colleagues concluded the tors) is distinct from the general factor of gratitude.
results supported the anticipated three-factor oblique structure. In addition to refining our theoretical understanding of grat-
Subsequently, Froh and colleagues conducted a CFA using the itude, these results can clarify the dimensionality of a promi-
15-item version of the GRAT–RS in a sample of youth ranging nent gratitude assessment, the GRAT–RS. Dimensionality
in age from 10 to 19 (N D 1,405), concluding the GRAT–RS evidence (e.g., explained common variance) has a direct bearing
demonstrated the same dimensionality as in the adult sample. on how users can permissibly model and score the GRAT–RS.
Unidimensional, second-order, or bifactor solutions were not For example, even when a unidimensional measurement model
explored in either sample. provides a poor fit for a given instrument, under certain cir-
To date, no published studies have subjected the GRAT–RS cumstances it remains permissible for test users to model the
to an oblique CFA in an adult sample independent from the instrument using a unidimensional solution (Rodriguez, Reise,
EFA sample. As noted by Hammer and Toland (2016b), “It is & Haviland, 2016b). In practice, some researchers or clinicians
important to use an independent sample to confirm an instru- might wish to treat the GRAT–RS as unidimensional for the
ment’s internal structure (Worthington & Whittaker, 2006) purposes of scoring, but this could result in model misspecifica-
because it is possible for an initial data set to be idiosyncratic, tion that can lead to factor loadings that are too high or low. In
leading to the identification of an initial factor structure that turn, this can lead to inaccurate conclusions regarding group
does not generalize well to subsequent samples (i.e., a lack of differences on gratitude or how strongly gratitude correlates
structural generalizability)” (p. 2). This study conducted an with other constructs (Rodriguez et al., 2016b). Thus, the
independent sample CFA examining the dimensionality of the degree of average relative measurement parameter bias
GRAT–RS among adults. (ARMPB) associated with forcing the GRAT–RS into a unidi-
Furthermore, the GRAT–RS has not been subjected to a mensional solution was also calculated in this study. The pres-
bifactor CFA nor a second-order (i.e., higher order) CFA. The ence of significant ARMPB would suggest the possibility of
 GRATITUDE 3

inaccurate parameter estimates and, in turn, indicate that 2014; Watkins, Cruz, Holben, & Kolts, 2008), and inversely
researchers and clinicians should avoid operationalizing grati- linked with negative affect, thought suppression, envy, cyni-
tude via a unidimensional solution for the GRAT–RS. cism, material values, pathological narcissism, and ruminating
about sad memories (Diessner & Lewis, 2007; Magno &
Orillosa, 2012; Solom et al., 2017; Watkins et al., 2008). Impor-
Reliability of the raw GRAT–RS scores
tant to clinical contexts, the GRAT–RS has demonstrated pre-
Cronbach’s alpha (a) estimates for the GRAT–RS total (i.e., dictive validity through a longitudinal relationship with
composite) score have ranged from .86 to .92 among adults depression. Specifically, in one study, the Time 1 GRAT–RS
(Spangler, 2010; Thomas & Watkins, 2003). Estimates for the total score significantly predicted depression 6 weeks later
three subscales scores have ranged from .80 (LOSD), .76 to .87 (Time 2), controlling for Time 1 levels of self-reported depres-
(SA), and .75 to .76 (AO) among adults (Diessner & Lewis, sion, life satisfaction, positive and negative affect, and Time 2
2007; Diessner, Solom, Frost, Parsons, & Davidson, 2008). levels of self-reported academic stress (Van Dusen, 2014). This
However, if the GRAT–RS best conforms to a bifactor struc- relationship remained significant when life satisfaction at Time
ture, this would indicate that the raw scores’ internal consis- 2 was added to the model.
tency estimates are dictated by both general and specific Researchers have stated that the GRAT–RS subscale scores
sources of common variance (Rodriguez et al., 2016b). In other demonstrated convergent evidence of validity via correlations
words, the GRAT–RS total score’s alpha coefficient might be with theoretically linked criterion variables (e.g., life satisfac-
artificially inflated by reliable variance from the specific factors, tion, depression, positive affect, negative affect, materialism,
and the GRAT–RS subscale scores’ alpha coefficients might be spiritual transcendence, achievement emotions; Diessner &
artificially inflated by reliable variance from the general factor. Lewis, 2007; Froh et al., 2011; Magno & Orillosa, 2012; Thomas
Therefore, model-based reliability measures (e.g., omega hierar- & Watkins, 2003; Toussaint & Friedman, 2009). However,
chical) are needed to determine whether the raw GRAT–RS incremental convergent evidence of validity for the GRAT–RS
total score and subscale scores are reliable measures of their factors has not been reported. This investigation used structural
intended constructs. This complies with the Standards for Edu- equation modeling (SEM) to examine incremental convergent
cational and Psychological Testing, which state that it is only evidence of validity for the general gratitude factor and the
appropriate to use scores—particularly subscale scores—that three specific factors (LOSD, SA, and AO). This can provide an
demonstrate evidence of “distinctiveness and reliability” (Stan- initial answer to the question of whether specific forms of grati-
dard 1.14; American Educational Research Association, Ameri- tude—particularly those measuring appreciation (SA and
can Psychological Association, National Council on AO)—have incremental clinical utility beyond the general grat-
Measurement in Education, & Joint Committee on Standards itude factor (Fagley, 2012). The finding that certain factors fail
for Educational and Psychological Testing, 2014, p. 27). This to account for unique variance in key outcomes would hold
study used model-based reliability measures to determine implications for the theoretical and clinical utility of these grati-
whether it might be appropriate to use the raw total or subscale tude factors. For example, if AO was uniquely linked to well-
scores of the GRAT–RS in future research and clinical practice. being beyond general gratitude, clinicians using gratitude might
The finding that certain scores are unreliable indicators of their consider supplemental interventions aimed toward increasing
intended gratitude construct would have significant implica- expression of appreciation to others. Factors failing to demon-
tions for the clinical utility of gratitude assessment with the strate added value might require enhanced scrutiny in future
GRAT–RS. research, and possibly theoretical revision.

Evidence of validity for the GRAT–RS factors This study

Regarding convergent evidence of validity, the total GRAT–RS In summary, this study had three primary goals. The first was
score shows moderate to strong associations with other estab- to examine the theoretical structure of gratitude through the
lished measures of gratitude (Froh et al., 2011; Solom, Watkins, comparison of four models. A unidimensional model will cor-
McCurrach, & Scheibe, 2017; Toussaint & Friedman, 2009; respond the conceptualization of gratitude as a unitary con-
Van Dusen, 2014), including the original GRAT (i.e., r D .95 struct (e.g., Wood, Maltby, Stewart, et al., 2008). An oblique
[Toussaint & Friedman, 2009]; r D .97 [Watkins, 2014]). The model comprised of three latent correlated factors will corre-
GRAT–RS total score has also demonstrated concurrent evi- spond to the view of gratitude as comprised of three indepen-
dence of validity through correlations with theoretically linked dent-yet-related forms of gratitude (e.g., Diessner & Lewis,
criterion variables. For example, broaden and build theory 2007). In turn, the three pillars of gratitude conceptualization,
asserts that gratitude should be linked to action tendencies that which suggests that gratitude simultaneously exists in three
promote social engagement (Fredrickson, 2004), and indeed specific forms but also as a superordinate construct, will be
the GRAT–RS total score has been positively associated with operationalized by a higher order model and bifactor model.
interpersonal forgiveness and inversely associated with inter- A higher order model considers gratitude as a common element
personal revenge and avoidance (Toussaint & Friedman, 2009). (i.e., second-order factor) that runs through all three first-order
The GRAT–RS has also been linked with greater life satisfac- factors. A bifactor approach will model the superordinate grati-
tion, psychological well-being, happiness, positive affect, spiri- tude construct as a general factor that exists independent of its
tual transcendence, and self-esteem (Diessner & Lewis, 2007; specific forms (i.e., operationalized by a bifactor model). Given
Solom et al., 2017; Toussaint & Friedman, 2009; Van Dusen, the moderate to strong correlations among the three GRAT–RS
4 HAMMER AND BRENNER

subscales reported in the literature (Magno & Orillosa, 2012), the sample identified as White, 6.6% as African American or
and the unique substantive value argued for specific gratitude Black, 5.4% as Latino or Latina, 3.3% multiracial, 3.3% Asian
facets (Fagley, 2012), we anticipated that a bifactor structure American or Pacific Islander, 1.4% American Indian or Native
would best account for the item covariation for the GRAT–RS. American, and 0.7% other race or ethnicity; 1% preferred not
Assuming the GRAT–RS conforms to a bifactor structure, to answer. Approximately 88.3% of participants identified as
ancillary bifactor measures pertaining to dimensionality would heterosexual, 6.6% bisexual, 2.5% lesbian, 0.7% questioning,
be examined to determine whether the GRAT–RS should be 0.7% asexual, 0.2% gay, and 0.8% chose to self-identify with a
viewed as having a primarily unidimensional or primarily mul- text response (e.g., pansexual). Approximately 50% reported
tidimensional structure and, in turn, inform our theoretical being married or in a committed relationship or civil union,
understanding of gratitude as a unidimensional or multifaceted 31% were single, 17% were separated or divorced, 2% were wid-
construct. owed, and 1% preferred not to answer. Approximately 10.6%
The second goal, assuming the GRAT–RS conforms to a reported having some high school education or a high school
bifactor structure, was to use model-based reliability estimates diploma, 7.0% earned a technical and further education
to determine whether it might be appropriate to use the raw (TAFE) degree, 62.4% had college experience or earned a
GRAT–RS total score or any of the three raw subscale scores. 4-year college degree, and 19.9% earned a graduate or profes-
Past research determined that some multidimensional instru- sional degree.
ments should only be scored as univocal measures (e.g.,
Brewster, Hammer, Sawyer, Eklund, & Palamar, 2016; Brouwer,
Measures
Meijer, & Zevalkink, 2013; Hammer & Toland, 2016b), whereas
others should only have certain subscales scored and the total Gratitude
score and remaining subscales scores should be abandoned The GRAT–RS is designed to measure gratitude. Items are rated
(e.g., C. R. Li, Toland, & Usher, 2016). Given extant research on a Likert-type scale with scores ranging from 1 (strongly dis-
on the GRAT–RS’s mixed-strength interfactor correlations, it agree) to 9 (strongly agree). Scores were derived by averaging
was anticipated that model-based reliability estimates would individual item scores. Higher scores reflect higher levels of
support the use of some raw scores but not others. reported gratitude. Extant evidence for the psychometric proper-
Third, assuming the GRAT–RS conforms to a bifactor struc- ties of the GRAT–RS was discussed in the introduction. In this
ture, SEM would then be used to examine incremental conver- study, the internal consistency estimates (a) were as follows:
gent evidence of validity. The gratitude literature consistently GRAT–RS D .82, 95% CI [.80, .85], LOSD D .86, 95% CI [.84,
suggests that gratitude is linked to greater life satisfaction and .88], SA D .85, 95% CI [.83, .87], and AO D .84, 95% CI [.82,
positive affect (see Wood et al., 2010, for a review) as well as .86]. The means for the scores were as follows: GRAT–RS D 6.68
lower distress (Disabato et al., 2016; Fredrickson et al., 2003; (SD D 1.20), LOSD D 5.89 (SD D 1.77), SA D 7.21 (SD D 1.37),
Van Dusen, 2014; Wood, Maltby, Gillet, et al., 2008). Therefore, and AO D 7.06 (SD D 1.47).
we used measures of these constructs to examine incremental
convergent evidence of validity for the GRAT–RS general and Distress
specific factors. Given the inclusion of two appreciation-based The K6 (Kessler et al., 2002) is a 6-item scale measure of non-
specific factors (i.e., SA and AO), findings would contribute to specific psychological distress developed for use in the U.S.
the burgeoning line of research assessing the incremental utility National Health Interview Survey. Participants read the sen-
of appreciation over and above gratitude (Fagley, 2012). tence stem, “During the past 30 days, about how often did you
feel …” and rate items such as “nervous” and “hopeless” on a
5-point Likert scale from 0 (all the time) to 4 (none of the time).
Method The six item scores are summed to calculate a total score, with
higher scores indicating greater tendency toward mental illness.
Participants and procedure
The K6 has been linked to blinded clinical diagnoses in general
Participants were 426 adults (300 women, 126 men) who com- U.S. population samples (Kessler et al., 2003), and self-report
pleted the online survey study through Amazon’s Mechanical ratings of severe mental illness in samples from 14 countries
Turk (MTurk). Results from research examining the viability of (Kessler et al., 2010). Internal consistency estimates have been
samples recruited through MTurk suggests that the data are as reported for undergraduate (a D .84; Lannin, Vogel, Brenner,
reliable as data obtained through traditional methods Abraham, & Heath, 2016) and community adult samples
(Buhrmester, Kwang, & Gosling, 2011; Goodman, Cryder, & (.89 < as < .92; Kessler et al., 2002). In this study, the K6
Cheema, 2013; Paolacci, Chandler, & Ipeirotis, 2010; Shapiro, yielded an internal consistency estimate of .89, 95% CI [.87,
Chandler, & Mueller, 2013). Participants were informed that .90] and a mean of 2.31 (SD D .90).
participation was voluntary and they received $0.10 for their
participation. Interested participants were directed to an online Positive affect
survey that began with an informed consent page, followed by The positive affect subscale from the Positive and Negative
the instrument battery and demographic items, and ended with Affect Schedule (PANAS; Watson, Clark, & Tellegen, 1988)
a debriefing page. Approval was obtained for this project from was used to measure positive affect over the past week. The 10-
Iowa State University’s Office of Responsible Research. item positive affect subscale (PANAS PA) consists of items,
Participants ranged in age from 18 to 74 years old such as “excited” and “strong” (Watson et al., 1988, p. 1070).
(M D 36.7, SD D 12.7, median D 33). Approximately 79.3% of Participants indicate the extent to which they experienced each
 GRATITUDE 5

emotion over the past week on a 5-point Likert-type scale rang- bifactor—were examined. We used Mplus’s MLR option for
ing from 1 (very slightly or not at all) to 5 (extremely). The maximum likelihood estimation, which calculates the scaled
items are averaged to create a total score, with higher scores chi-square test statistic (scaled x2) and associated fit indexes
indicating more positive affect. Previous studies produced con- that use it, to protect against deviations from multivariate nor-
current evidence of validity for the PANAS PA through associ- mality. Model fit was evaluated using the scaled x2 statistic,
ations with other measures of positive and negative mood root mean square error of approximation (RMSEA), compara-
(Watson et al., 1998), as well as happiness in undergraduate tive fit index (CFI), Tucker–Lewis index (TLI), and standard
and adult community samples (Wei, Liao, Ku, & Shaffer, 2011). root mean square residual (SRMR). The following fit criteria
The PANAS PA yielded an internal consistency estimate of reli- were used: RMSEA  .06, CFI  .95, TLI  .95, and SRMR <
ability of .89 in an adult community sample (Wei et al., 2011) .08 (Weston & Gore, 2006). The metric for each factor was
and .89 in this study, 95% CI [.88, .91] and a mean of 3.30 (SD scaled by setting the variance of each factor to one. All analyses
D .79). were done at the 5% significance level. Only the bifactor solu-
tion met all fit criteria (see Table 1). Thus, the CFA results sug-
Life satisfaction gest that the GRAT–RS best conforms to a bifactor structure
The Satisfaction with Life Scale (SWLS; Diener, Emmons, rather than a unidimensional, three-factor oblique, or second-
Larsen, & Griffin, 1985) is a five-item measure of life satisfaction. order structure.
Participants indicate the extent to which the items reflect how We next calculated the explained common variance (ECV;
they view their lives using a 7-point Likert scale ranging from 1 Reise, Moore, & Haviland, 2010), an index of unidimensional-
(strongly disagree) to 7 (strongly agree). An example item is “In ity attributable to the general factor and each of the three spe-
most ways, my life is close to my ideal.” The items are averaged cific factors. ECV quantifies how unidimensional versus
to create a total score, with higher scores indicating greater life multidimensional a set of items is. Rodriguez et al. (2016b)
satisfaction. The SWLS has demonstrated positive correlations indicate that an ECV > .70 and a percent of uncontaminated
with interviewer ratings of 1-hr interviews of life satisfaction variance (PUC) > .70 suggest that the presence of some multi-
(Diener et al., 1985) and self-reports of happiness (Wei et al., dimensionality is not severe enough to disqualify the interpre-
2011). The SWLS has demonstrated internal consistency in tation of the instrument as primarily unidimensional. PUC is
community adults (a D .90; Wei et al., 2011) and yielded test– the percentage of “unique correlations in a correlation matrix
retest reliabilities of .84 for 2-week and 1-month intervals across that are influenced by a single factor. … The higher the PUC,
undergraduate students (Pavot, Diener, Colvin, & Sandvik, the more the matrix is saturated with information relevant to
1991). In this study, the SWLS demonstrated an internal consis- estimating the parameters of a single factor and the less likely
tency of .92, 95% CI [.87, .90] and a mean of 4.15 (SD D 1.57). the parameter estimates in a unidimensional model will be
biased” (Rodriguez, Reise, & Haviland, 2016a, p. 146).
Quality check Table 2 summarizes the factor loadings for the unidimen-
The survey contained two instructed response items (e.g., sional and bifactor solutions for the GRAT–RS. Four findings
respond with “strongly disagree” for this item). The survey inform the dimensionality of the GRAT–RS. First, the ECV
included a yes–no self-reported single-item indicator known as (.45) and PUC (.70) failed to reach the cutoff for essential uni-
the SRSI Use Me (“In your honest opinion, should we use your dimensionality. The ECV of .45 means that only 45% of the
data?”; Meade & Craig, 2012, p. 16). The SRSI Use Me has been common variance in the set of items is due to the general
highly correlated with number of bogus items missed and pro- dimension. Second, half of the items (particularly the SA items)
vides a clear cutoff point (Meade & Craig, 2012). demonstrated stronger general factor loadings than specific fac-
tor loadings, and the other half of the items demonstrated the
Results opposite pattern (particularly the LOSD items). Third, half of
the items were better measures of their respective specific fac-
Data cleaning tors than the general factor, indicated by individual explained
The initial data set contained 487 individuals. Cases with any common variance (IECV) coefficients below .50 for 8 of the 16
missing data (n D 0), incorrect responses to one or both items (see Table 2). Fourth, the average difference between
instructed response items (n D 28), with a duplicate person items’ loading in the unidimensional solution and on the
response (n D 26) indicated by participants’ anonymous MTurk
Worker ID, and with responses of “no” to the SRSI Use Me item Table 1. Goodness of fit statistics for all tested measurement models.
(n D 7) were deleted, resulting in a final sample of N D 426. The Model Scaled x2 df RMSEA [90% CI] CFI TLI SRMR
final sample was used for all analyses and reliability estimates. No
GRAT–RS unidimensional 958.61 104 .139 [.131, .147] .632 .576 .139
variables exceeded cutoffs of 3 and 10 for high univariate skew- GRAT–RS Three-factor 301.77 101 .068 [.060, .077] .914 .897 .076
ness and kurtosis values, respectively (Weston & Gore, 2006). oblique
GRAT–RS second order 301.77 101 .068 [.060, .077] .914 .897 .076
GRAT–RS bifactor 181.99 88 .050 [.040, .060] .960 .945 .040
Dimensionality of the GRAT–RS
Note. The chi-square for all models was statistically significant at the p < .001 level.
The dimensionality of the GRAT–RS was first tested via a series GRAT–RS D Gratitude Resentment and Appreciation Test–Revised Short. Statis-
of CFAs with Mplus version 6.11 (L. K. Muthen & Muthen, tics are based on MLR estimation. Scaled x2 D scaled chi-square test statistic;
RMSEA D root mean square error of approximation; CI D confidence interval;
1998–2012). Specifically, four competing measurement mod- CFI D comparative fit index; TLI D Tucker–Lewis index; SRMR D standardized
els—unidimensional, three-factor oblique, second-order, and root mean square residual.
6 HAMMER AND BRENNER

Table 2. Confirmatory factor analysis standardized loadings, IECVs, and ARMPBs.

Bifactor

Uni Gen F1 F2 F3 IECV RMPB

Specific Factor 1: Lack of a sense of deprivation

2. Life has been good to me. .52 .46 .39 .58 .13
3. There never seems to be enough to go around and I never seem to get .30 .24 .68 .11 .25
my share.
6. I really don’t think that I’ve gotten all the good things that I deserve in .34 .27 .67 .14 .23
life.
10. More bad things have happened to me in my life than I deserve. .27 .19 .74 .06 .40
11. Because of what I’ve gone through in my life, I really feel like the world .38 .33 .58 .24 .15
owes me something.
15. For some reason I don’t seem to get the advantages that others get. .28 .20 .83 .06 .36
Specific Factor 2: Simple pleasures
4. Oftentimes I have been overwhelmed at the beauty of nature. .47 .45 .51 .44 .05
7. Every Fall I really enjoy watching the leaves change colors. .46 .44 .49 .44 .05
9. I think that it’s important to “Stop and smell the roses.” .77 .79 .19 .95 .02
12. I think that it’s important to pause often to “count my blessings.” .80 .83 –.05 1.00 .03
13. I think it’s important to enjoy the simple things in life. .79 .81 .08 .99 .03
16. I think it’s important to appreciate each day that you are alive. .81 .86 –.03 1.00 .05
Specific Factor 3: Social appreciation
1. I couldn’t have gotten where I am today without the help of many .54 .45 .54 .41 .21
people.
5. Although I think it’s important to feel good about your .67 .58 .53 .55 .15
accomplishments, I think that it’s also important to remember how others
have contributed to my accomplishments.
8. Although I’m basically in control of my life, I can’t help but think about .68 .62 .46 .64 .11
all those who have supported me and helped me along the way.
14. I feel deeply appreciative for the things others have done for me in my .80 .76 .33 .84 .05
life.

Note. Uni D unidimensional model; Gen D general factor; F1 D lack of a sense of deprivation specific factor; F2 D simple pleasures specific factor; F3 D social appreciation
specific factor; IECV D individual explained common variance; RMPB D relative measurement parameter bias. Loadings are standardized and based on MLR estimation.
All bolded standardized loadings significant at p < .05.

general factor in the bifactor solution was .04, with differences common factors (i.e., the general factor and all three specific
ranging from a low of .02 to a high of .09 (see Table 2). Thus, factors). Coefficient omega hierarchical (vH) measures the pro-
the average relative measurement parameter bias (ARMPB; portion of explained total score variance that can be attributed
Hammer, 2016; Rodriguez et al., 2016b) associated with forcing to the general factor after accounting for the three specific fac-
the GRAT–RS into a unidimensional solution was 14%, which tors. It quantifies the degree to which the raw total score is a
is close to violating the 10% to 15% upper limit discussed by B. univocal indicator of the general factor (Rodriguez et al., 2016a,
Muthen, Kaplan, and Hollis (1987). The ARMPB is the mean p. 145). The percentage of reliable variance (PRV) for the total
of “the difference between each item’s loading in the unidimen- score (i.e., vH divided by v for the total score) offers a more
sional solution and its general factor loading in the bifactor contextualized view of the total score’s reliability. Namely, by
(i.e., the truer model), divided by the general factor loading in only including reliable variance and not taking error into
the bifactor” (Rodriguez et al., 2016a, p. 145). In summary, account, the PRV assesses overall explained total score variance
these four findings collectively suggest that the dimensionality in a way that vH does not. An vH > .75 and total score PRV
of the construct of gratitude, as operationalized by the GRAT– of > 75% would both indicate that the raw GRAT–RS total
RS, is best represented as a multidimensional (bifactor) solution score is an appropriate measure of the general gratitude factor
and that the use of a unidimensional solution might be mis- (D. Li et al., 2016; Reise et al., 2013, p. 137).
leading. This also indicates that specific forms of gratitude, Results indicated that the general gratitude factor only
including appreciation (operationalized by the SA and AO spe- accounted for 65% of the explained variance (vH D .65) and
cific factors), are dimensionally distinct from the superordinate 70% of the reliable variance (total score PRV D 70%) in the raw
construct of gratitude (operationalized by the general gratitude GRAT-RS total score. Therefore, the evidence did not strongly
factor). support the use of the raw GRAT-RS total score as a measure
of the latent general construct of gratitude.
Model-based reliability of the GRAT–RS total score
Model-based reliability of the GRAT-RS subscale
Coefficient omega (v) is similar to Cronbach’s alpha, but a more
scores
suitable indicator of reliability for instruments with a bifactor
structure. It is “a factor analytic model-based estimate of the Coefficient Omega for subscales measures the proportion of
reliability (true score variance over observed score variance) of explained subscale score variance that can be attributed to all
unit-weighted test scores” (Rodriguez et al., 2016a, p. 145). common factors (i.e., the general factor and the specific factor
Coefficient omega for the total score measures the proportion corresponding with that subscale). Coefficient Omega Hierar-
of explained total score variance that can be attributed to all chical Subscale (vHS) measures the proportion of explained
 GRATITUDE 7

subscale score variance uniquely accounted for by the corre- scores that demonstrated a small PRV (see earlier) are the least
sponding specific factor, after partialling out the variance likely to demonstrate incremental evidence of validity. How-
accounted for by the general factor. A low vHS precludes ever, it is best to conduct a direct test, particularly when exam-
meaningful interpretation of the raw subscale score as a clear ining latent factors with borderline-sufficient model-based
indicator of a specific factor (Rodriguez et al., 2016a; p. 146). reliability.
The PRV for the subscale score (i.e., vHS for that subscale In line with extant research, we specified three separate SEM
score divided by v for that subscale score) likewise offers a structural models in which the (a) GRAT–RS items were set to
more contextualized view of the subscale score’s reliability. An load in accordance with the bifactor model, (b) the criterion
vHS > .75 and subscale score PRV of > 75% would indicate variable items (i.e., life satisfaction, positive affect, or distress)
that the raw subscale score in question is an appropriate mea- were set to load on the latent criterion variable factor, and (c)
sure of its corresponding specific factor (C. R. Li et al., 2016; the GRAT–RS general and specific factors were simultaneously
Reise et al., 2013, p. 137). Table 3 summarizes the v, vH, vHS, regressed onto the latent criterion variable factor. Results indi-
PRV, and ECV coefficients for the bifactor solution of the cated that the general factor (life satisfaction b D .32, positive
GRAT–RS. affect b D .44, distress b D ¡.39) and LOSD specific factor (life
The LOSD specific factor (vHS D .74, PRV D 85%) approxi- satisfaction b D .46, positive affect b D .12, distress b D ¡.41)
mated or exceeded the cutoffs, whereas the SA (vHS D .09, accounted for significant variance in all three criterion varia-
PRV D 10%) and AO (vHS D .32, PRV D 38%) specific factors bles, whereas the SA and AO specific factors never did
did not. Thus, evidence supports the use of the raw LOSD sub- (ps > .14). Therefore, the general gratitude factor and the
scale score as a measure of the LOSD specific factor, but not LOSD specific factors were the only factors to demonstrate
does support the use of the raw SA or AO subscale scores. incremental convergent evidence of validity. The SA and AO
specific factors did not demonstrate incremental clinical utility
beyond the general gratitude factor.
Evidence of validity for the GRAT–RS latent factors
Although model-based reliability measures provide guidance
Discussion
regarding the utility of raw total and subscale scores, these
methods do not address the validity of the general and specific This study reexamined the factor structure of the GRAT–RS and
latent factors themselves. Importantly, the specific factors’ extended our theoretical understanding of the gratitude con-
demonstration of theoretical dimensional distinctness does not struct. An advantage of the GRAT–RS over existing gratitude
necessary mean that they will demonstrate incremental clinical measures is its provision of a more comprehensive assessment
utility beyond the general factor. Latent factors can only be of gratitude through its inclusion of three purportedly distinct
modeled in the context of SEM, where their relationship with yet related facets of gratitude. However, there is debate regarding
theoretically linked criterion variables can be studied with the composition of gratitude: Is gratitude a unitary construct
greater precision. The use of bifactor CFA to separate general (Wood, Maltby, Stewart, et al., 2008), or are there aspects of grat-
factor variance from specific factor variance allows researchers itude, such as appreciation, that fall under the umbrella of grati-
to examine incremental convergent evidence of validity for tude but are uniquely associated with psychological health and
each of the four GRAT–RS latent factors. Factors with raw well-being over and above general trait gratitude (Adler & Fag-
ley, 2005; Fagley, 2012)? Although the GRAT–RS is one of the
Table 3. Ancillary bifactor measures. more commonly used multidimensional measures of gratitude,
sparse examination of its factor structure presented a serendipi-
Gen F1 F2 F3
tous gap in the literature where theory, practice, and psychomet-
Omega (v) .92 .87 .88 .85 rics intersect. That is, we extended our theoretical and applied
Omega hierarchical (vH) .65 .23 .02 .03 understanding of gratitude through an investigation of the
Omega hierarchical subscale (vHS) .74 .09 .32
Percentage of rreliable variance (PRV) .70 .85 .10 .38 dimensionality of the GRAT–RS, the model-based reliability of
Explained common variance (ECV) .45 .41 .06 .08 total and subscale scores, and incremental evidence of validity
for its latent factors.
Note. Gen D general factor; F1 D lack of a sense of deprivation specific factor;
F2 D simple pleasures specific factor; F3 D social appreciation specific factor. All
bolded coefficients significant at p < .05. Coefficient omega for the total score
(Gen column) measures the proportion of explained total score variance that can Dimensionality
be attributed to all common factors (i.e., the general factor and all three specific
factors). Coefficient omega hierarchical (vH) measures the proportion of The GRAT–RS best conformed to a bifactor structure. This
explained total score variance that can be attributed to the general factor after suggests that the covariation among the 16 GRAT–RS items
accounting for the three specific factors. The percentage of reliable variance could be best accounted for by (a) a single general gratitude fac-
(PRV) for the total score (Gen column) is vH divided by v for the total score.
Coefficient omega for the subscale score measures the proportion of explained tor that reflects the common variance across all items, and (b)
subscale score variance that can be attributed to all common factors (i.e., the three specific factors that capture additional (i.e., unique) com-
general factor and the specific factor corresponding with that subscale). Coeffi- mon variance among clusters of items (Reise, 2012, p. 688).
cient omega hierarchical subscale (vHS) measures the proportion of explained
subscale score variance uniquely accounted for by the corresponding specific Ancillary bifactor measures reinforced the finding that the
factor, after partialing out the variance accounted for by the general factor. The GRAT–RS’s dimensionality is best represented with a multidi-
PRV for the subscale score is the vHS for that subscale score divided by v for mensional (bifactor) solution. In contrast, results did not pro-
that subscale score. The explained common variance (ECV) is an index of unidi-
mensionality of the common variance attributable to the general factor and each vide support for the use of a unidimensional, three-factor
of the three specific factors. oblique, or second-order measurement model to conceptualize
8 HAMMER AND BRENNER

remeasure the general gratitude factor, rather than the specific

factors they were designed to measure. In contrast, because
85% of the reliable subscale score variance was driven by the
LOSD specific factor, there is sufficient evidence that raw
LOSD subscale scores can be interpreted as representing mean-
ingful information about the LOSD specific factor.
Thus, model-based reliability estimates provided strong sup-
port for the use of one of the four raw GRAT–RS scores. This
outcome is not uncommon when examining well-respected
instruments from a bifactor framework (e.g., C. R. Li et al.,
2016; Rodriguez et al., 2016a). When a factor cannot reliably be
measured by its intended raw score, it should instead be mea-
sured as a latent SEM factor. Researchers and clinicians using
the GRAT–RS are encouraged to carefully consider the risks of
using insufficiently reliable raw scores. This can result in mis-
leading interpretations, which threatens the clinical utility of
gratitude assessment via the GRAT–RS.

Figure 1. Bifactor measurement model displaying standardized factor loadings. Validity

Covariances, error terms, and disturbance terms are not pictured for readability.
GEN D general factor; LOSD D lack of a sense of deprivation specific factor; Building on the model-based reliability measures, which pro-
SP D simple pleasures specific factor; SA D social appreciation specific factor. vided guidance regarding the utility of raw total and subscale
scores, the use of bifactor CFA to separate general factor vari-
the GRAT–RS. Given the 14% ARMPB found in this study, test ance from specific factor variance allowed for the examination
users should carefully consider how much measurement of incremental convergent evidence of validity for each of the
parameter bias they are willing to accept before choosing to four GRAT–RS latent factors. Results indicated that the general
force the GRAT–RS into a unidimensional model. gratitude factor and the LOSD specific factor each accounted
Regarding theoretical implications, these findings support for significant variance in all three criterion variables (i.e., life
the three pillars of gratitude conceptualization (Watkins, 2009, satisfaction, positive affect, or distress), whereas the SA and AO
as cited in Watkins, 2014), wherein the superordinate gratitude specific factors did not. This incremental convergent evidence
construct functions as a general factor that exists independent of validity in favor of the general gratitude factor and LOSD
of its pillars (operationalized by the orthogonal specific factors). specific factor support the continued measurement and study
Findings also support the theoretical distinction of the specific of these two constructs. In contrast, additional research on the
forms of gratitude (Fagley, 2012), and the distinction of appre- incremental convergent validity of the SA and AO specific fac-
ciation from the superordinate construct of gratitude (Fagley, tors in the context of other criterion variables is needed.
2012; Manela, 2013; Wood, Maltby, Stewart, et al., 2008). The finding that only certain latent factors operationalized
by an instrument accounted for variance in key criteria is like-
wise common in the bifactor literature (e.g., Brenner, Heath,
Reliability
Vogel, & Crede, 2017; Brewster et al., 2016; Schroder, Dawood,
Contrary to popular opinion, “the finding that an instrument Yalch, Donnellan, & Moser, 2016). However, the finding that
conforms to a bifactor structure does not, by itself, provide a the appreciation factors (i.e., SA and AO) did not account for
compelling rationale for the calculation and interpretation of incremental variance in these key variables is notable, as this
the total or subscale scores of that instrument” (Hammer & was a fundamental clinical utility question of interest raised by
Toland, 2016b, p. 11). Therefore, model-based reliability meas- Fagley (2012). Namely, Fagley found that certain appreciation-
ures were calculated to determine whether it might be appro- based subscales predicted life satisfaction above a general mea-
priate to use the raw GRAT–RS total and subscale scores in sure of gratitude in a hierarchical regression and, in turn, con-
research and clinical settings. Given that only 70% of the reli- cluded that this supports the clinical utility of appreciation
able total score variance was driven by the general gratitude fac- above and beyond gratitude. In their study, however, Fagley
tor, there is a lack of strong evidence in favor of using the raw used regression of correlated manifest variables rather than
GRAT–RS total score as a measure of the general gratitude fac- partialing out a general gratitude factor. If appreciation is con-
tor. Rather, researchers wishing to measure general trait grati- ceptually subordinate to gratitude, as supported by this study,
tude using the GRAT–RS would likely best capture this shared variance between the appreciation and gratitude mea-
construct through a latent bifactor structure. sure(s) might present as a confound and overinflate the contri-
Given that only 10% and 38% of the reliable subscale score bution of a specific factor (Tracey, 2012). To build on our
variance was driven by the SA and AO specific factors, respec- understanding of the relationship between appreciation and
tively, there is a lack of strong evidence in favor of using the gratitude, future researchers can examine unidimensional, obli-
raw SA and AO subscale scores as measures of the SA and AO que, higher order, and bifactor models using responses to a
specific factors. In other words, these scores primarily variety of purported gratitude and appreciation measures.
 GRATITUDE 9

This research also presents implications for clinicians using interest, then developing and retaining items that primarily
gratitude. The finding that the general gratitude factor was measure these constructs rather than the general gratitude con-
uniquely associated with lower distress as well as greater life struct would be desirable. Researchers could examine the inter-
satisfaction and positive affect further supports the use of gen- nal structure of the full-length GRAT, which might yield a
eral gratitude exercises. Exercises using appreciation of simple different factor structure or more reliable factor scores than
pleasures or interpersonal relationships might be useful as part found with the GRAT–RS in this study. Finally, the GRAT–RS,
of this general gratitude contribution, rather than offering like many popular social science instruments, would benefit
incremental value above other general gratitude interventions. from investigation of how sensitive the GRAT–RS scores are to
Given the incremental convergent evidence of validity of LOSD changes in the gratitude construct they purport to measure
over and above the general gratitude factor in predicting these (Borsboom, Mellenbergh, & van Heerden, 2004). It is essential
outcomes, it might behoove clinicians to use interventions to demonstrate that gratitude has a causal effect on responding
aimed at addressing how fairly one believes he or she has been on the GRAT–RS.
treated in life. This might include mindfulness exercises aimed
toward decreasing rumination about being treated unfairly.
Focusing on LOSD might offer incremental clinical utility ORCID
beyond other gratitude and appreciation interventions. Such Joseph H. Hammer http://orcid.org/0000-0002-4028-5878
clinical suppositions must be tested directly in future research. Rachel E. Brenner http://orcid.org/0000-0003-1069-9591

Cautions, limitations, and additional future directions

References
As of this writing, scholarship on bifactor modeling is rapidly
Adler, M. G., & Fagley, N. S. (2005). Appreciation: Individual differences in
evolving, with guidelines for analysis and interpretation in flux. finding value and meaning as a unique predictor of subjective well-
Thus, we offer these interpretations as key information for the being. Journal of Personality, 73, 79–114.
ongoing conversation regarding the theoretical structure of American Educational Research Association, American Psychological
gratitude and the psychometric properties of the GRAT–RS, Association, National Council on Measurement in Education, & Joint
rather than definitive conclusions. It is also important to reiter- Committee on Standards for Educational and Psychological Testing.
(2014). Standards for educational and psychological testing. Washing-
ate the utility of bifactor analysis and ancillary bifactor meas- ton, DC: AERA.
ures, as demonstrated in this article. These promising yet Bernstein, D. M., & Simmons, R. G. (1974). The adolescent kidney donor:
underutilized methods offer researchers the ability to under- The right to give. American Journal of Psychiatry, 131, 1338–1343.
stand the dimensionality of instruments and the constructs Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004). The concept
they operationalize with a much greater degree of precision of validity. Psychological Review, 111, 1061–1071. doi:10.1037/0033-
295X.111.4.1061
than merely comparing alternative CFA models (Rodriguez Brenner, R. E., Heath, P. J., Vogel, D. L., & Crede, M. (2017). Two is more
et al., 2016a). valid than one: Examining the factor structure of the Self-Compassion
In addition, the generalizability of these findings to other Scale (SCS). Journal of Counseling Psychology. Advance online publica-
versions, adaptations, and translations of the GRAT–RS cannot tion. doi:10.1037/cou0000211
be assumed. For example, it is not certain to what degree these Brewster, M. E., Hammer, J., Sawyer, J. S., Eklund, A., & Palamar, J. (2016).
Perceived experiences of atheist discrimination: Instrument develop-
results would replicate themselves with the original GRAT. ment and evaluation. Journal of Counseling Psychology. Advance online
Future researchers could examine the original GRAT using publication. doi:10.1037/cou0000156
analyses like those used in this study. Likewise, given the com- Brouwer, D., Meijer, R. R., & Zevalkink, J. (2013). On the factor structure
munity adult makeup of this sample, it cannot be assumed that of the Beck Depression Inventory–II: G is the key. Psychological Assess-
these findings will automatically generalize to other popula- ment, 25, 136–145. doi:10.1037/a0029228
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical
tions. Most of the sample was also female, White, and reported Turk: A new source of inexpensive, yet high-quality, data? Perspectives
moderate to strong gratitude. Thus, the generalizability of these on Psychological Science, 6(1), 3–5.
findings should be tested directly in future research, rather Diener, E. D., Emmons, R. A., Larsen, R. J., & Griffin, S. (1985). The Satis-
than assumed. For example, perhaps the GRAT–RS demon- faction with Life Scale. Journal of Personality Assessment, 49, 71–75.
strates a different factor structure or more reliable gratitude, Diessner, R., & Lewis, G. (2007). Further validation of the Gratitude,
Resentment, and Appreciation Test (GRAT). The Journal of Social Psy-
SA, and AO factor scores among university students, religious chology, 147, 445–447.
practitioners, or populations scoring low in gratitude. Diessner, R., Solom, R. D., Frost, N. K., Parsons, L., & Davidson, J. (2008).
The results reported here prompt future research beyond the Engagement with beauty: Appreciating natural, artistic, and moral
clinical and theoretical directions mentioned earlier in this dis- beauty. The Journal of Psychology, 142, 303–332.
cussion. First, the SA and AO specific factors would benefit Disabato, D. J., Kashdan, T. B., Short, J. L., & Jarden, A. (2016). What pre-
dicts positive life events that influence the course of depression? A lon-
from additional investigation of the evidence—incremental and gitudinal examination of gratitude and meaning in life. Cognitive
otherwise—for or against their validity. Second, if these find- Therapy and Research. Advance online publication. doi:10.1007/
ings are replicated with diverse samples, a redevelopment of s10608-016-9785-x
gratitude theory and the GRAT–RS might be indicated. Such a Fagley, N. S. (2012). Appreciation uniquely predicts life satisfaction above
redevelopment could benefit from the use of exploratory bifac- demographics, the Big 5 personality factors, and gratitude. Personality
and Individual Differences, 53, 59–63.
tor analysis to help guide the creation of an instrument that Fredrickson, B. L. (2004). The broaden-and-build theory of positive emo-
will reliably measure the constructs of most interest. For exam- tions. Philosophical Transactions of the Royal Society of London: Series
ple, if the specific factors (LOSD, SA, and AO) are of most B Biological Sciences, 359, 1367–1378.
10 HAMMER AND BRENNER

Fredrickson, B. L., Tugade, M. M., Waugh, C. E., & Larkin, G. R. (2003). Reise, S. P., Moore, T. N., & Haviland, M. G. (2010). Bifactor models and
What good are positive emotions in crisis? A prospective study of resil- rotations: Exploring the extent to which multidimensional data yield
ience and emotions following the terrorist attacks on the United States univocal scale scores. Journal of Personality Assessment, 92, 544–559.
on September 11th, 2001. Journal of Personality and Social Psychology, Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016a). Applying bifactor
84, 365–376. statistical indices in the evaluation of psychological measures. Journal
Froh, J. J., Fan, J., Emmons, R. A., Bono, G., Huebner, E. S., & Watkins, P. of Personality Assessment, 98, 223–237.
(2011). Measuring gratitude in youth: Assessing the psychometric Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016b). Evaluating bifactor
properties of adult gratitude scales in children and adolescents. Psycho- models: Calculating and interpreting statistical indices. Psychological
logical Assessment, 23, 311–324. Methods, 21, 137–150.
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a Schroder, H. S., Dawood, S., Yalch, M. M., Donnellan, M. B., & Moser, J. S.
flat world: The strengths and weaknesses of Mechanical Turk samples. (2016). Evaluating the domain specificity of mental health-related
Journal of Behavioral Decision Making, 26, 213–224. mind-sets. Social Psychological and Personality Science, 7, 508–520.
Hammer, J. H. (2016, October). Average relative parameter bias (ARPB) Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013). Using Mechanical Turk
calculator: A Microsoft Excel–based tool to calculate the ARPB statistic. to study clinical populations. Clinical Psychological Science, 1, 213–220.
Retrieved from http://DrJosephHammer.com/ Solom, R., Watkins, P. C., McCurrach, D., & Scheibe, D. (2017). Thieves of
Hammer, J. H., & Toland, M. D. (2016a). Bifactor analysis in Mplus [Video file]. thankfulness: Traits that inhibit gratitude. The Journal of Positive Psy-
Retrieved from http://sites.education.uky.edu/apslab/upcoming-events/ chology, 12, 120–129.
Hammer, J. H., & Toland, M. D. (2016b). Internal structure and reliability Spangler, K. (2010). Does gratitude enhance self-esteem? (Unpublished
of the Internalized Stigma of Mental Illness Scale (ISMI–29) and brief master’s thesis). Eastern Washington University, Cheney.
versions (ISMI–10, ISMI–9) among Americans with depression. Stigma Thomas, M., & Watkins, P. (2003, May). Measuring the grateful trait:
and Health. Advance online publication. doi:10.1037/sah0000049 Development of revised GRAT. Paper presented at the annual conven-
Kessler, R. C., Andrews, G., Colpe, L. J., Hiripi, E., Mroczek, D. K., Nor- tion of the Western Psychological Association, Vancouver, BC, Canada.
mand, S. L., … Zaslavsky, A. M. (2002). Short screening scales to moni- Toussaint, L., & Friedman, P. (2009). Forgiveness, gratitude, and well-
tor population prevalences and trends in non-specific psychological being: The mediating role of affect and beliefs. Journal of Happiness
distress. Psychological Medicine, 32, 959–976. Studies, 10, 635–654. doi:10.1007/s10902-008-9111-8
Kessler, R. C., Barker, P. R., Colpe, L. J., Epstein, J. F., Gfroerer, J. C., Hiripi, Tracey, T. J. G. (2012). Problems with single interest scales: Implications of
E., … Zaslavsky, A. M. (2003). Screening for serious mental illness in the general factor. Journal of Vocational Behavior, 81, 378–384.
the general population. Archives of General Psychiatry, 60, 184–189. doi:10.1016/j.jvb.2012.10.001
doi:10.1002/mpr.310 Van Dusen, J. P. (2014). Relationships amongst gratitude, well-being and
Kessler, R. C., Green, J. G., Gruber, M. J., Sampson, N. A., Bromet, E., Cui- depression (Unpublished doctoral dissertation). University of Toledo,
tan, M., … Lara, C. (2010). Screening for serious mental illness in the Toledo, OH.
general population with the K6 screening scale: Results from the WHO Van Dusen, J. P., Tiamiyu, M. F., Kashdan, T. B., & Elhai, J. D. (2015).
World Mental Health (WMH) survey initiative. International Journal Gratitude, depression and PTSD: Assessment of structural relation-
of Methods in Psychiatric Research, 19(1), 4–22. ships. Psychiatry Research, 230, 867–870.
Lannin, D. G., Vogel, D. L., Brenner, R. E., Abraham, T. W., & Heath, P. J. Watkins, P. C. (2009, April). Gratitude: Amplifier of the good. Invited paper
(2016). Does self-stigma reduce the probability of seeking mental presented to the 89th annual convention of the Western Psychological
health information? Journal of Counseling Psychology, 63, 351–358. Association, Portland, OR.
doi:10.1037/cou0000108 Watkins, P. C. (2014). What is gratitude and how can it be measured? In P.
Li, C. R., Toland, M. D., & Usher, E. L. (2016). Dimensionality, scoring, and C. Watkins (Ed.), Gratitude and the good life (pp. 13–40). doi:10.1007/
interpretation of the Short Grit Scale. Manuscript in preparation. 978-94-007-7253-3_2
Li, D., Zhang, W., Li, X., Li, N., & Ye, B. (2012). Gratitude and sui- Watkins, P. C., Cruz, L., Holben, H., & Kolts, R. L. (2008). Taking care of
cidal ideation and suicide attempts among Chinese adolescents: business? Grateful processing of unpleasant memories. The Journal of
Direct, mediated, and moderated effects. Journal of Adolescence, Positive Psychology, 3, 87–99.
35(1), 55–66. Watkins, P. C., Woodward, K., Stone, T., & Kolts, R. L. (2003). Gratitude and
Magno, C., & Orillosa, J. (2012). Gratitude and achievement emotions. happiness: Development of a measure of gratitude and relationships with
Philippines Journal of Counseling Psychology, 14(1), 29–43. subjective well-being. Social Behavior and Personality, 31, 431–452.
Manela, T. (2013). Gratitude and appreciation. American Philosophical Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and valida-
Quarterly, 53, 281–294. tion of brief measures of positive and negative affect: The PANAS
McCullough, M. E., Emmons, R. A., & Tsang, J. A. (2002). The grateful dis- scales. Journal of Personality and Social Psychology, 54, 1063–1070.
position: A conceptual and empirical topography. Journal of Personal- Wei, M., Liao, K., Ku, T., & Shaffer, P. A. (2011). Attachment, self-compas-
ity and Social Psychology, 82, 112–127. doi:10.1037//0022-3514.82.1.112 sion, empathy, and subjective well-being among college students and
Meade, A. W., & Craig, S. B. (2012). Identifying careless responses in sur- community adults. Journal of Personality, 79, 191–221.
vey data. Psychological Methods, 17, 437–455. Weston, R., & Gore, P. (2006). A brief guide to structural equation model-
Muthen, B., Kaplan, D., & Hollis, M. (1987). On structural equation ing. The Counseling Psychologist, 34, 719–751.
modeling with data that are not missing completely at random. Psycho- Wilder, T. (2007). The cabala and the woman of Andros [HarperCollins e-
metrika, 52, 431–462. book version]. Retrieved from https://www.harpercollins.com
Muthen, L. K., & Muthen, B. O. (1998–2012). Mplus user’s guide (7th ed.). Wood, A. M., Froh, J. J., & Geraghty, A. W. A. (2010). Gratitude and well-
Los Angeles, CA: Authors. being: A review and theoretical integration. Clinical Psychology Review,
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on 30, 890–905.
Amazon Mechanical Turk. Judgment and Decision Making, 5, 411–419. Wood, A. M., Maltby, J., Gillett, R., Linley, P. A., & Joseph, S. (2008). The
Pavot, W. G., Diener, E., Colvin, C. R., & Sandvik, E. (1991). Further vali- role of gratitude in the development of social support, stress, and
dation of the Satisfaction With Life Scale: Evidence for the cross- depression: Two longitudinal studies. Journal of Research in Personal-
method convergence of well-being measures. Journal of Personality ity, 42, 854–871.
Assessment, 57, 149–161. Wood, A. M., Maltby, J., Stewart, N., & Joseph, S. (2008). Conceptualizing
Reise, S. P. (2012). The rediscovery of bifactor measurement models. Mul- gratitude and appreciation as a unitary personality trait. Personality
tivariate Behavioral Research, 47, 667–696. and Individual Differences, 44, 621–632.
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013). Scoring and model- Worthington, R. L., & Whitaker, T. A. (2006). Scale development research:
ing psychological measures in the presence of multidimensionality. A content analysis and recommendations for best practices. The
Journal of Personality Assessment, 95, 129–140. Counseling Psychologist, 34, 806–838.