Psychological Bulletin

1978, Vol. 85, No. 6, 1275-1301

The Classification of Child Psychopathology:

A Review and Analysis of Empirical Efforts
Thomas M. Achenbach and Craig S. Edelbrock
National Institute of Mental Health, Bethesda, Maryland

Empirically derived syndromes of child behavior problems are reviewed, and

those having counterparts in two or more studies are identified. A distinction
between broad-band and narrow-band syndromes seems warranted because many
syndromes derived from first-order factor analyses can be subsumed by a few
second-order factors. Broad-band Undercontrolled and Overcontrolled syndromes
and narrow-band Aggressive, Delinquent, Hyperactive, Schizoid, Anxious, De-
pressed, Somatic, and Withdrawn syndromes were found in diverse samples of
disturbed children. Two other broad-band and six other narrow-band syndromes
were found in a few studies. Test-retest reliabilities and stabilities of syndrome
scores were more adequate than interrater reliabilities, which increased with
the degree of similarity between the types of raters and between the types of
situations in which they saw subjects. Cross-instrument and cross-population
consistencies corroborated some empirically derived syndromes, but the lack of
independent criteria for categorizing disturbed children makes it difficult to
establish criterion-referenced validity. Because categorization of children by
syndromes has been limited primarily to the broad-band undercontrolled-over-
controlled dichotomy, more efforts are needed to translate syndromes into cat-
egories for use by practitioners as well as researchers. It is concluded that
further work in this area is likely to benefit disturbed children only if it is more
systematically linked to the existing mental health system and to efforts at re-
forming this system.

The study of psychopathology in children and Statistical Manual (DSM) were Adjust-
has long lacked a coherent taxonomic frame- ment Reaction and Childhood Schizophrenia,
work within which training, treatment, epi- Adult categories of the DSM could be ap-
demiology, and research could be integrated, plied to children, but a national survey
Not only were children's disorders omitted showed that 70% of the children seen in
from the system that originally formed the psychiatric clinics were either unclassified or
basis for psychiatric taxonomy (Kraepelin, were classified as having adjustment reactions
1883) but, even until 1968, the only cate- (Rosen, Bahn, & Kramer, 1964).
gories provided for children in the American Although there were a few early efforts to
Psychiatric Association's (1952) Diagnostic develop differentiated taxonomies of child-
hood disorders (e.g., Jenkins & Glickman,
1946), it was not until the 1960s that the
need for SUch a S Stem WaS aCUtelv fdt
The authors wish to thank Sue Fleisher for her y ;
many contributions to this work; Howard Moss and Efforts to meet this need took a variety of
Sally Ryan for performing reliability checks on forms. Following the psychiatric tradition of
the categorization of syndromes; and Roger Slash- formulating classifications through consensual
field, Gale Inoff, Lovick Miller, and John Weisz . , , ,-, , ., ._, „ _ _ „
for critically reading the manuscript. committee work, the Group for the Advance-
Requests for reprints should be sent to Thomas ment of Psychiatry (GAP; Note 1) pro-
M. Achenbach, Laboratory of Developmental Psy- posed an elaborate system that combined
chology, Building 1SK, National Institute of Men- , . , . , ,. ., u --.--t,.
tal Health, 9000 Rockville Pike, Bethesda, Maryland developmental considerations with a variety
20014. of theoretical viewpoints. In the second edi-

In the public domain

1275
1276 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

tion of its Diagnostic and Statistical Manual more clinically useful, reliable, and empiri-
oj Mental Disorders (DSM-II), the Ameri- cally based foundation for the study of child
can Psychiatric Association (1968) also disorders. Moreover, the increasing availa-
added several categories of behavioral reac- bility of powerful multivariate techniques
tions of childhood, including the hyperki- offers an opportunity to establish an empiri-
netic, withdrawing, overanxious, runaway, cally based taxonomy that may circumvent
unsocialized aggressive, and group delinquent many of the problems of the traditional
reactions. Like those of the GAP system, the adult classification system. As a result, the
categories of the DSM-H are based on mix- 1960s and 1970s have witnessed numerous
tures of theoretical inferences and general- attempts to identify empirically syndromes
ized descriptions of behavior, with no pro- that pould serve as a basis for the classifica-
cedures for operationalizing them. The tion of childhood psychopathology. Because
reliability of both systems is mediocre, with the welter of findings is in need of consolida-
agreement between diagnosticians averaging tion, the present article reviews the efforts to
about 60% for major categories, such as develop empirically based systems for classi-
psychotic versus neurotic disorders, and fication of child psychopathology, identifies
considerably less for specific disorders within unsolved problems, and establishes guidelines
these categories (Freeman, 1971; Sandifer, for future efforts. We focus on descriptive
Pettus, & Quade, 1964; Tarter, Templer, & classification of the broad range of child
Hardy, 1975). Agreement averaging 67c/o psychopathology having no known organic
has been achieved by English child psychia- etiology and occurring in children presumed
trists who classified case histories according to be of normal intelligence. Studies of per-
to the major syndromes of the International sonality dimensions in nonclinical samples
Classification of Diseases and the Multi-axial (e.g., Schaefer, Note 2) and studies re-
Classification drafted by the World Health stricted to particular diagnostic subgroups,
Organization (Rutter, Shaffer, & Shepherd, such as psychotic (e.g., Prior, Boulton,
1975). However, the reliability estimates in Gajzago, & Perry, 1975) and retarded chil-
all these studies would be lowered by correc- dren (e.g., Lambert & Nicoll, 1976), are not
tions for chance agreement (Spitzer & Fleiss, considered unless they have implications for
1974). Current drafts of the DSM-OI offer the establishment of syndromes in a broad
little hope of improvement in the child area. range of clinically disturbed children. We
On another front, the mounting concern use the term syndrome generically to desig-
with classification of childhood disorders led nate groups of behavior problems found to be
the federal government to sponsor the Pro- statistically associated with one another. No
ject on the Classification of Exceptional assumptions about the etiologies or appropri-
Children (Hobbs, 1975a, 1975b). The diverse ate conceptual models for behavior disorders
experts contributing to the project generally are intended.
agreed that most current labels for excep- None of the studies reviewed was without
tional children are not only of little use but flaws, and no study has by itself produced
may be harmful because of the stigmata they definitive results. However, because the
confer and the categorical nature of services flaws and biases vary from study to study,
they dictate. Although it was agreed that convergent findings that emerge from di-
classification in some form is necessary, there verse studies may be worthy of confidence.
was little agreement on what type of classifi- Rather than emphasizing the defects of each
cation would be most appropriate. study, we therefore attempt to identify com-
mon findings that can serve as a benchmark
Empirical Approaches to Classification of for subsequent work. For purposes of deter-
Child Psychopathology mining which syndromes have been estab-
lished, we omit findings that are almost cer-
The lack of entrenched categories for child tain to be unreliable because they have been
psychopathology may ultimately prove a derived from analyses employing less than
blessing if it promotes efforts to develop a twice as many subjects as variables or less
 CHILD PSYCHOPATHOLOGY 1277

than 100 subjects (Lessing, Black, Barbara, lations among 125 behavior problems re-
& Seibert, 1976; Patterson, 1964; Wysocki corded from the guidance clinic records of
& Wysocki, 1970). Furthermore, to avoid 2,113 boys and 1,181 girls aged 6-17. Al-
undue attention to findings that do not re- though Ackerson himself did not derive syn-
flect substantial groupings of items, we omit dromes statistically, Jenkins and Glickman
syndromes comprising less than five items (1946) used Ackerson's data to identify pairs
having factor loadings or average intercorre- of highly correlated items. They then suc-
lations of .30 or higher, as well as efforts cessively added items that had the next high-
that produced less than two such syndromes est correlations with each of these items.
(Fanshel, Hylton, & Borgatta, 1963; Spi- Decisions about when to stop adding items to
vack & Swift, 1966; Wolff, 1971). The ex- a cluster were subjective, but no items were
clusion of a few studies for failure to meet added unless their average correlation with
our criteria produces a slightly more con- the other items of a cluster was .20 or higher.
servative array of syndromes than if these In this way, Jenkins and Glickman found
studies were included. However, data from five syndromes for both sexes, which they
some of these studies are considered where labeled Overinhibited, Socialized Delinquent,
they are relevant to questions other than the Unsocialized Aggressive, Brain Injured, and
establishment of syndromes, such as correla- Schizoid. The first three were similar to three
tions among raters and among instruments. found by Hewitt and Jenkins (1946) when
The diversity of subject samples, rating they applied a combination of clinical judg-
instruments, raters, and methods of analysis ment and correlational criteria to 94 traits
makes it difficult to aggregate the findings reported in 500 other clinic case histories.
of various studies in summary form. To pro- Applying a more formal cluster analysis to
vide the reader with essential details, we the correlations among 90 of these traits,
therefore present the relevant studies one by Jenkins (1966) later derived syndromes re-
one before attempting a synthesis of findings sembling the five derived by Jenkins and
and conclusions on which to base future work. Glickman (1946) from Ackerson's data, but
Because children rarely seek treatment or Jenkins gave them slightly different names,
report their own problems, adults' observa- Shy-Seclusive, Socialized Delinquent, Undo-
tions typically provide the data base for mesticated, Hyperactive-Distractible, and
assessment of children's disorders. Since the Overanxious-Neurotic, respectively.
perspectives of adult informants inevitably In one of the first applications of factor
affect the data they supply, we group studies, analysis to behavior problems, Himmelweit
for purposes of review, according to whether (cited in Eysenck, 1953) performed a cen-
the primary data were supplied by mental troid analysis of 50 of the items Ackerson
health workers, teachers, or parents. Unless (1942) reported for boys. Two major factors
otherwise noted, the majority of subjects in were found: One was interpreted as a general
each sample were boys. abnormality dimension; the other was in-
terpreted as indicative of personality prob-
Syndromes Derived From Mental Health lems versus conduct problems or introversion
Workers' Reports versus extraversion. Lorr and Jenkins (1953)
obtained somewhat similar results when they
One approach to the empirical derivation converted the five Jenkins-Glickman (1946)
of syndromes has been to analyze the be- clusters for boys into factors, which they then
havior problems reported in clinical case his- subjected to a second-order centroid analysis
tories compiled by mental health workers.
with oblique rotation. One second-order fac-
A second approach has been to have mental
health workers directly report their observa- tor, labeled Rebellion, resembled the conduct
tions on structured checklists. The results of problems pole of Himmelweit's second factor,
these two approaches are reviewed separately. and the second, labeled Maladaptation, re-
Case history data. In a pioneering study, sembled Himmelweit's general abnormality
Ackerson (1942) computed tetrachoric corre- factor.
1278 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

Achenbach (1966) applied a more com- An additional rotated factor for boys, la-
prehensive factor-analytic approach to be- beled Sexual Problems, and four additional
havior problems reported in the case histories rotated factors for girls, labeled Hyperreac-
of 300 4- to 15-year-olds of each sex, includ- tive Behavior, Neurotic and Delinquent Be-
ing outpatients and short-term inpatients. havior, Obesity, and Depressive Symptoms,
Seventy-four items were subjected to princi- also classified children who were not uni-
pal components analysis for the boys, and 73 formly classified by the Internalizing and
were analyzed for the girls. The first princi- Externalizing syndromes. The remaining ro-
pal component for children of each sex re- tated factor for boys, labeled Delinquent
sembled Himmelweit's Personality Problems Behavior, classified boys who were also classi-
Versus Conduct Problems factor but was fied by the Externalizing syndrome, whereas
designated as Internalizing-Externalizing. the remaining rotated factor for girls, labeled
The second principal component, labeled Anxiety Symptoms, classified children who
Severe and Diffuse Psychopathology, was were also classified by the Internalizing syn-
unipolar and resembled Himmelweit's general drome. Classification of subjects by the sec-
abnormality factor in that items indicative ond principal component bore no consistent
of severe pathology had the highest loadings. relationship to classification by the rotated
In addition, six rotated factors for the boys factors.
and nine for the girls not only met our cri- Direct reports by mental health workers.
teria of five items with loadings of .30 or In a cross-cultural test of the syndromes
higher but also appeared consistently in rota- identified by Jenkins (1966; Jenkins & Click-
tions to three different criteria for simple man, 1946), Kobayashi, Mizushima, and
structure—varimax, quartimax, and oblimin. Shinohara (1967) had psychologists record
Four of the rotated factors were similar 68 items for 200 6- to 15-year-old boys eval-
for both sexes and were given the following uated in Tokyo guidance clinics. Most of the
labels: Aggressive Behavior; Obsessions, items had been employed by Jenkins, but
Compulsions, and Phobias; Somatic Com- some were added on the basis of local experi-
plaints; and Schizoid Thinking and Behavior. ence. Cluster analysis of the items produced
When children were classified first according syndromes resembling the Socialized Delin-
to the Internalizing-Externalizing dichotomy quent, Unsocialized Aggressive (Undomesti-
and then according to the rotated factors, a cated), and Brain-Injured (Hyperactive-
hierarchical relationship between the broad- Distractible) syndromes found by Jenkins. A
band syndromes and some of the narrow- fourth syndrome resembled both the Over-
band syndromes emerged that was similar for inhibited (Shy-Seclusive) and Schizoid
both sexes: Children whose problems (Overanxious-Neurotic) syndromes reported
matched those on the Aggressive Behavior by Jenkins, but the presence of "fantastic,
factor were classified by the first principal unrealistic thinking" and "suspiciousness" in
component as externalizers and children the syndrome suggests that it is most akin to
whose problems matched those on the So- Jenkins' Schizoid cluster.
matic Complaints and Obsessions, Compul- Factor analyses of data recorded directly
sions, and Phobias factors were classified by by clinicians have also been reported. In a
the first principal component as internalizers. study of 268 8- to 10-year-old English boys
However, children whose problems matched receiving psychiatric evaluations, Collins,
those on the Schizoid Thinking and Behavior Maxwell, and Cameron (1962) performed a
factor were not uniformly classified by the principal components analysis of 59 items.
Internalizing and Externalizing clusters. For (A separate analysis of 64 items for 98 girls
both sexes, some narrow-band syndromes is not discussed here because the N and ratio
thus appeared to represent clear-cut subtypes of subjects to variables fell below our cri-
of the Externalizing and Internalizing syn- teria.) The items included specific behavior
dromes, whereas the Schizoid syndrome was problems and scores for very general cate-
not subsumed by these broad-band syn- gories of complaint (e.g., "manifest disturb-
dromes. ance in relation to school"), early child-rear-
 CHILD PSYCHOPATHOLOGY 1279

ing data, family history, parental attitudes, Immature Conduct Problem and had highest
and intelligence level. The first principal loadings on excessive demands for attention,
component, designated Rebelliousness, disobedience, and immaturity. The second,
showed high loadings for such behavior prob- labeled Phobic-Suicidal and having highest
lems as disobedience and destructiveness. A loadings on truancy, school phobia, and sui-
dichotomy between parental neglect and cidal attempts, appears to have no clear
overconcern was manifest in a bipolar second counterpart in other studies.
factor labeled Rootlessness. The third factor Factor analysis has also been applied to
was also bipolar, comprising anxiety symp- mental health workers' ratings of children in
toms at one pole and aggressive and delin- residential treatment centers. One of the most
quent behaviors at the opposite pole, con- extensive efforts of this type was carried out
sistent with the dichotomy between these by Spivack and Spotts (Note 3) in develop-
items found in other studies. ing the Devereux Child Behavior Rating
In another study of both sexes in which Scale (DCB). Using versions of the DCB
only the boys' samples were large enough to ranging in length from 68 to 121 items, they
meet our criteria, Borgatta and Fanshel factor analyzed child care workers' ratings of
(1965) applied a principal factor - varimax several samples of children aged 5-14, many
analysis to 70 traits scored for children seen of whom were severely disturbed, retarded,
in 30 different guidance clinics. Separate or brain damaged (Spivack & Levine, 1964;
analyses were made for 272 boys aged 7-12 Spivack & Spotts, 1965, Note 3). In a further
and 150 boys aged 13-17. Of the 12 factors study, a factor analysis was performed on 125
rotated, 5 contained at least five behavior items scored for 640 adolescents aged 13-18
problems that met Borgatta and Fanshel's (Spivack & Spotts, 1967). In all these stud-
cutoff point of .35 for loadings in both age ies, large numbers of factors were rotated by
groups. These factors were labeled Defiance, the equamax method, although only factors
Unsocialized, Tension-Anxiety, Learning Dif- having at least five items with adequate
ficulty, and Overcleanliness. Three others met loadings are mentioned here.
the criteria only for the older sample. These In the younger samples, six factors met
were labeled Infantilism, Lack of Affection, this criterion and were given the labels Emo-
and Sex Precociousness. tional Detachment, Proneness to Emotional
In a study designed to compare the factor Upset, Need for Adult Contact, Anxious-
structures of three different checklists, Les- Fearful Ideation, Impulse Ideation, and In-
sing and Zagorin (1971) had mothers of 102 ability to Delay (Spivack & Spotts, Note 3).
10-12-year-old clinic children fill out the A principal components - varimax analysis of
Peterson (1961) Problem Checklist (PPC) scores obtained by a new sample of institu-
and the Wichita Guidance Center Checklist tionalized boys on 12 of Spivack and Spotts's
(WGCC). In addition, a psychiatrist filled factors yielded three second-order factors
out the 36-item Institute for Juvenile Re- (Schaefer & Millman, 1973a). The first two
search Checklist (IJRC) based on an inter- were like the aggressive and inhibited syn-
view with each mother. Principal factor- dromes obtained in other studies, and the
binormamin analyses were performed on the third included high loadings for first-order
data collected with each checklist. Because factors indicative of extreme disorganiza-
58 PPC items and 55 WGCC items were tion and immaturity. In their adolescent sam-
factored, these analyses did not meet our cri- ple, Spivack and Spotts (1967) found nine
terion of at least twice as many subjects as factors having at least five items with high
variables for establishment of syndromes, al- loadings and labeled them as follows: Poor
though the degree of similarity found among
factors is considered below. The analysis of Emotional Control, Defiant-Resistive, Domi-
the psychiatrists' ratings on the 3,6 IJRC nating-Sadistic, Heterosexual Interest, Need
items produced two factors that had at least Approval - Dependency, Physical Inferior-
five items reaching the authors' cutoff point ity - Timidity, Hyperactive-Expansive, Schiz-
of .35 for loadings. One of these was labeled oid Withdrawal, and Poor Coordination.
1280 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

The factors found by Spivack and Spotts graders. The 26 items analyzed for the sev-
for institutionalized children appear to re- enth graders yielded the previously found
flect more molecular groupings of items than Conduct and Personality Problem factors,
even the narrow-band syndromes found in and the 28 items analyzed for the eighth
most studies of noninstitutionalized children. graders yielded an additional factor, Imma-
Relatively molecular groupings were also turity. Like a factor labeled Inadequacy-Im-
found in another study of institutionalized maturity for preadolescent delinquents
children, despite the use of a much shorter (Quay, 1966), the Immaturity factor had
behavior checklist (54 items) and the ex- high loadings on "preoccupation," "short at-
clusion of retarded and brain-damaged chil- tention span," "inattentiveness," and "day-
dren (Raff, Cromwell, & Davis, Note 4). dreaming" (Quay & Quay, 1965, p. 217).
The 107 children, aged 7-13, were rated by Applying similar analyses to teacher rat-
nurses, counselors, teachers, and others who ings of a more clinical sample—441 5- to
worked with them throughout the day. As 15-year-olds in classes for the emotionally
with the Spivack-Spotts studies, several of disturbed—Quay, Morse, and Cutler (1966)
the rotated factors had few items with high found three factors like the Conduct Prob-
loadings, but eight had at least five items lem, Personality Problem, and Immaturity
with loadings of .30 or higher. These factors factors found for normal eighth graders
were labeled Acting Out - Unsocialized, (Quay & Quay, 1965) and preadolescent de-
Withdrawn-Psychotic, Emotional Lability, linquents (Quay, 196,6). These three factors
Ideational or Language Disturbed Psy- thus appear to have considerable stability in
chotic, Impulsivity-Restraint, Interpersonal teachers' ratings on Peterson's Checklist, al-
Integrity, Acting Out - Bully, and Liar. though a failure to replicate more than the
Conduct Problem factor in teacher ratings of
Syndromes Derived From Teachers' Reports Oglala Sioux high school students suggests
that the factors may not be cross-culturally
Although our focus is on studies of psycho- consistent (O'Donnell & Cress, 1975).
pathology of clinical proportions, the most Ross, Lacey, and Parton (1965) adopted
widely cited syndromes of behavior problems a novel approach in the development of the
were originally derived from teachers' ratings Pittsburgh Adjustment Survey Scales
of children in regular public school classes. (PASS). On the assumption that scales for
The ratings were made on the Problem Check- aggressive, withdrawn, and prosocial behavior
list developed by Peterson (1961) and re- would adequately describe the social behavior
vised by Quay and Peterson (Note S). Fifty- of elementary school boys, Ross et al. as-
eight items were initially included on the sembled 111 items hypothesized to reflect
checklist because of their high frequency in these dimensions. Teachers rated boys who,
child guidance case histories, but three of unknown to the teachers, had been selected
the items were dropped because teachers re- as being aggressive, withdrawn, or well ad-
ported them for less than three percent of justed. The 94 items that significantly dis-
their pupils. Four separate centroid analyses criminated among the three categories of boys
were performed on teachers' ratings of 126 were retained for factor analysis of a new
kindergarteners, 273 first and second graders, sample of 209 randomly selected boys. A
229 third and fourth graders, and 239 fifth principal components - varimax analysis
and sixth graders (Peterson, 1961). Ten yielded four substantial factors, three of
centroid factors were extracted for each age which were defined mainly by the three scales
group, but only two were rotated to the vari- developed on the criterion groups. The fourth
max criterion. These two factors were similar factor, labeled Passive-Aggressive, consisted
in all four groups and were labeled Conduct of items describing resistive and covertly ag-
Problem and Personality Problem. Quay and gressive behavior. Applying a principal com-
Quay (1965) performed similar analyses of ponents - varimax analysis to teachers' rat-
Peterson items that were reported for at ings on a modified version of the PASS, Mil-
least 10% of 259 seventh and 259 eighth ler (1972) obtained factors somewhat similar
 CHILD PSYCHOPATHOLOGY 1281

to the Withdrawn and Prosocial factors, as Although nine factors were obtained in a
well as one that combined the Aggressive and centroid-varimax analysis, the authors chose
Passive-Aggressive factors. to retain only the two largest factors be-
In contrast with the general-purpose cause their "interest was primarily in a two-
checklists, Conners's (1969) Teacher Rating factor model" (p. 433). They also factor
Scale was specifically designed to measure analyzed the Kohn Social Competence Scale,
the effects of drug therapy. Using the origi- composed of 90 items describing social be-
nal 39-item version, teachers rated 103 chil- havior, filled out for the same sample as the
dren (mean age = 9 years 10 months) se- Kohn Symptom Checklist. Six factors were
lected for dextroamphetamine therapy for obtained in a centroid-varimax analysis, but
behavior disorders. A principal components - the authors again retained only the first two.
varimax analysis yielded five factors, Ag- High negative correlations between items
gressive Conduct, Daydreaming-Inattentive, having the highest loadings on Symptom
Anxious-Fearful, Hyperactivity, and Social- Factor I and those on Social Competence
Cooperative. The last factor, however, might Factor I and between those on Symptom
just as well be defined in terms of problems in Factor II and Social Competence Factor II
social relationships, since all the items with indicated that the two instruments tapped
high (negative) loadings reflected such prob- the same two dimensions but in reverse form.
lems. When combined, the social competence and
Another instrument developed for use with symptom dimensions were labeled Interest-
a particular treatment is the Teacher Re- Participation - Apathy-Withdrawal and Co-
ferral Form (TRF; Clarfield, 1974), which operation-Compliance - Anger-Defiance.
has been revised slightly and renamed the The Preschool Behavior Questionnaire em-
Classroom Adjustment Rating Scale (Lorion, ployed by Behar and Stringfield (1974) was
Cowen, & Caldwell, 1975). The TRF was a 36-item revision of the Children's Behav-
designed for referral and evaluation of chil- iour Questionnaire originated by Rutter
dren in the Primary Mental Health Project (1967). Teachers' ratings of 496 children in
(PMHP) carried out in the Rochester public regular preschools and 102 children in pre-
schools. A 49-item version of the TRF schools for the emotionally disturbed were
yielded three stable behavior problem factors subjected to a principal components - vari-
in a principal components - varimax analysis max analysis. Two of the factors were similar
of 373 primary school children referred to to those found by Kohn and Rosman and
the PMHP. These factors were labeled were labeled Hostile-Aggressive and Anxious-
Learning Problems, Acting Out, and Shy- Fearful. The third factor, from which only
Anxious. They correlated significantly with four items with high loadings were retained,
three factors labeled Aggressive-Outgoing, was labeled Hyperactive-Distractible.
Moody-Internalized, and Learning Disability
in the AML, a brief screening checklist com- Syndromes Derived From Parents' Reports
posed of 11 items (Cowen, Dorr, Clarfield,
Kreling, McWilliams, Pokracki, Pratt, Ter- In one of the first studies to use direct
rell, & Wilson, 1973). parental reports, parents of 80 normal and
Two efforts to identify behavioral syn- 353 clinic 6-13-year-olds were asked to indi-
dromes in preschoolers have been marked by cate which of 229 behavioral items described
their similarity of approach. In both, be- their children (Dreger, Lewis, Rich, Miller,
havior problem checklists were filled out by Reid, Overlade, Taffel, & Flemming, 1964).
preschool teachers for children aged 3-6, the Tetrachoric correlations of 142 behavioral
responses were factor analyzed, and discrimi- and demographic items were subjected to
native validity was assessed by comparing principal factor - oblimin analysis. The eight
scores obtained from clinical and nonclinical factors having at least five items with load-
groups. The Kohn Symptom Checklist, em- ings of .30 or higher were designated as Fac-
ployed by Kohn and Rosman (1972), con- tors A through H and were described as fol-
tained 58 items rated for. 407 preschoolers. lows: A—paranoid, aggressive, and isolative
1282 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

egocentricity; B—antisocial aggressiveness; factors labeled Social Withdrawal, Anxiety,

C—intellectual and scholastic retardation; and Sleep Disturbance. The remaining sec-
D—a psychoid type of desurgency; E—an ond-order factor was named Learning Dis-
appreciative, confident, and concerned socia- abilities, and it comprised first-order factors
bleness; F—an impulsive and excitable kind labeled Academic Disability and Immaturity.
of hyperactivity; G—a sadistic type of ag- Using the same response format as in his
gressiveness; H—disturbed sleep and dreams. Teacher Rating Scale, Conners (1970) ob-
A revised version of Dreger's checklist was tained ratings on behavior problems by
subsequently analyzed for 4-6-year-olds, but parents of 316 clinic and 365 nonclinic chil-
the results are of doubtful value for the dren aged 5-15. Nearly all of the clinic sam-
classification of psychopathology, since only ple had been diagnosed as either neurotic or
11 clinical cases were included, some of the hyperkinetic. For one analysis, 73 items were
factors had only one or two items with load- grouped under 24 headings, each of which
ings of .30 or higher, and other factors were subsumed from one to five individual items.
composed of very redundant items (Baker & Some of the groups contained items very
Dreger, 1973). similar in content, but others contained items
To provide a basis for comparison with that might in fact be negatively correlated,
the findings of Peterson (1961) and Dreger such as the group entitled "problems of eat-
et al. (1964), Miller (1967) included as ing," which included "will not eat enough"
marker variables one item from each of their and "overweight." The parents' responses to
checklists in his Louisville Behavior Check individual items were summed within each of
List (LBCL). Containing 163 items of devi- the 24 categories and subjected to principal
ant and prosocial behavior, the LBCL was components - varimax analysis, separately for
filled out by parents of 263 clinic boys aged the normal and clinic samples. Only two fac-
6-12. The 118 items reported for \Q%-W% tors in the clinic sample had at least five
of the sample were then subjected to princi- items with loadings of .30 or higher. These
pal factor - varimax analysis. Items with represented the familiar aggressive-inhibited
loadings of .30 or higher were retained for distinction and were labeled Aggressive -
scales defined by eight factors, and subjects' Conduct Disorder and Anxious-Immature. A
scores were computed for each factor, using subsequent factor analysis of 85 individual
unit weights for each item. Scores on the items for the combined normal and clinic
eight scales were T transformed and sub- samples (Conners, 1973) yielded seven fac-
jected to a second-order principal factor - tors meeting our criteria. These were labeled
varimax analysis, resulting in three factors Conduct Problem, Anxiety, Impulsive-Hyper-
which were then employed as broad-band active, Learning Problem, Psychosomatic,
scales. Antisocial, and Muscular Tension.
The first two broad-band scales, labeled Ag- Arnold and Smeltzer (1974) also factor
gression and Inhibition, resembled Peterson's analyzed 74 items adapted from the checklist
(1961) Conduct and Personality Problem from which Conners derived his items. Sepa-
factors, respectively. The marker variables rate principal factor - varimax analyses were
from the Peterson and Dreger et al. check- performed on checklists filled out by parents
lists both had high loadings on a first-order of 185 2-12-year-old and 166 13-18-year-old
factor labeled Infantile Aggression, which clinic patients. For both age groups, six fac-
loaded on the second-order Aggression factor. tors were found; these were labeled Unso-
Miller's Infantile Aggression factor is thus cialized Aggression, Inattentive Unproduc-
linked to Peterson's Conduct Disorder factor tiveness (primarily academic), Sociopathy,
and Dreger's Factor G, which Dreger called Hyperactivity, Withdrawal-Depression, and
"a sadistic type of aggressiveness." The other Somatic Complaint. An additional factor,
two first-order factors loading on the second- Sleep Disturbance, was also found for the
order Aggression factor were labeled Hyper- 13-18-year-olds.
activity and Antisocial. The second-order In- In a study that employed both clinical and
hibition factor comprised three first-order nonclinical samples, Ferguson, Partyka, and
 CHILD PSYCHOPATHOLOGY 1283

Lester (1974) obtained parents' responses to three factors, labeled Anxious-Fighting-De-

154 socially desirable and undesirable be- pressed, Mentation-Developmental, and De-
havior items for 112 clinic and 105 nonclinic linquency.
children aged 5-11. The 97 items yielding at In the final studies reviewed here, parents
least a 10% difference between clinic and of clinic children filled out the Child Behav-
nonclinic samples were retained for a princi- ior Checklist (CBCL), which consists of 118
pal factor - varimax analysis. Items were behavior problem items and 20 social compe-
employed to define a factor only if they tence items (Achenbach, 1978, in press;
loaded .45 or higher on the factor and had no Achenbach & Edelbrock, in press). Most of
loadings of less than .30 on any other factor the behavior problems were adapted from
or no loadings on other factors differing less Achenbach's (1966) factor analytic study of
than .20 from their highest loading. Accord- case history data but were reworded to make
ing to these stringent criteria, only two fac- them more appropriate for parents. The
tors, Impulsivity and Competence, were de- CBCL is scored in terms of the Child Be-
nned by a substantial number of items. Sub- havior Profile, of which separate editions are
sequent factor analyses were performed on standardized for each sex at ages 4-5, 6-11,
checklists filled out by mothers of the clinic and 12-16. Each edition consists of nonfac-
children, fathers of the clinic children, moth- torial social competence scales, plus be-
ers of the normals, and fathers of the nor- havior problem scales derived through prin-
mals. Although these samples were too small cipal components - varimax analyses of items
(ns = 85-407) to inspire much confidence, reported for between S% and 95% of clinic
the first two factors found for the total sam- children in the relevant age and sex group.
ple were replicated in each subsample. Factors were retained only if they appeared
A survey of behavior problems in a repre- in direct quartimin as well as varimax rota-
sentative sample of 1,034 urban 6-18-year- tions and if they contained at least six items
olds is relevant here because approximately with loadings of .30 or higher.
17^ had been referred for clinical services To date, we have completed editions of the
and the authors related their data to a vari- profile for both sexes at ages 6-11 and 12-
ety of other clinical criteria, including psy- 16, each edition based on CBCLs completed
chiatrists' ratings of impairment and prog- by parents of 450 clinic children of the rele-
nosis over a 5-year period, as discussed below vant age and sex. From 111 to 114 behavior
(Langner, Gersten, Eisenberg, Greene, & problem items have been retained for the
Herson, in press). Data were obtained from analyses, and nine robust factors were ob-
mothers' reports on 654 items. This number tained in each group, except the 12-16-year-
was reduced to 287 by grouping similar items old girls, for whom only eight were obtained.
and dropping items that were infrequent or Second-order principal components - varimax
age or sex contingent. A principal compo- and direct quartimin analyses of T scores on
nents - varimax analysis yielded 17 factors the first-order factor-based scales produced
that met our criteria, labeled as follows: two broad-band factors, labeled Internalizing
Sex Curiosity, Self-Destructive Tendencies, and Externalizing. The first-order factors
Mentation Problems, Conflict With Parents, varied somewhat from group to group, al-
Dependent-Unassertive, Regressive Anxiety, though factors labeled Somatic Complaints,
Weak Group Membership, Toilet Training, Withdrawal, Hyperactive, Aggressive, and
Undemanding, Repetitive Motor Behavior, Delinquent were found for all four groups.
Aggressive, Delusions-Hallucinations, Delin- Schizoid and Obsessive factors were found for
quent, Conflict With Siblings, Late Devel- three groups, but these were combined into
opment, Isolation, and Noncompulsive. The one factor for the 6-11-year-old girls. All but
last factor could just as well be labeled Com- the 12-16-year-old boys' sample yielded a
pulsive, since the defining items were denials factor labeled Depressed. Both boys' samples
of compulsivity (e.g., "Never checks on yielded a factor labeled Uncommunicative,
things several times"), which had high nega- whereas both girls' samples yielded a factor
tive loadings. A second-order analysis yielded labeled Cruel. Unique to a single sample
1284 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

were a factor labeled Sex Problems for the Achenbach & Edelbrock, in press; Lorr &
6-11-year-old girls and a factor labeled Im- Jenkins, 1953; Miller, 1967; Schaefer &
mature for the 12-16-year-old boys, although Millman, 1973a). Since a similar distinction
the Hyperactive factor found for the 12-16- was evident in the type and breadth of syn-
year-old girls included enough signs of imma- dromes yielded by studies that did not in-
turity to be called Immature-Hyperactive. clude second-order analyses, these studies
are listed in the broad-band group if they
Generality of Syndromes From produced no more than three syndromes and
Various Sources in the narrow-band group if they produced
four or more syndromes. This arbitrary cri-
Considering the variety of subject samples terion effectively divides a markedly bimodal
and procedures for collection and analysis of distribution of studies, with all but one study
data, comparison of the syndromes obtained (Kobayashi et al., 1967) producing either
in the various studies may be risky. Statisti- three or less or five or more syndromes. In
cally rigorous comparisons are precluded by fact, as can be seen from Table 1, most of
the lack of uniformity in the items employed. the studies producing broad-band syndromes
However, in order to provide an overview of without second-order analyses yielded no
findings to date, syndromes that appear to more than two syndromes that were repli-
have clear counterparts in more than one cated in at least one other study.
study are summarized in Table 1. Syndrome Despite the heterogeneity of approaches,
similarity was judged on the basis of simi- Table 1 reveals some substantial consisten-
larity in the content of items loading .30 or cies. There is no doubt that the broad-band
higher. After we established the categories syndromes here labeled Undercontrolled (ag-
and assigned syndromes to these categories, gressive, externalizing, acting out, conduct
two other psychologists independently sorted disorder) and Overcontrolled (inhibited, in-
the syndromes into the categories they con- ternalizing, shy-anxious, personality disorder)
sidered most appropriate. Our assignments are evident in all three sources of data. A
agreed with those of one judge for 91% of third broad-band syndrome, here labeled
the syndromes and with those of the other Pathological Detachment, was identified in
judge for 88f/e. The sources of the syndromes four studies but was not very uniform among
are those described in the previous section, these four. This syndrome was responsible
excluding any that did not employ formal for more disagreements than any other in
factor- or cluster-analytic procedures (Hew- the placement of syndromes into the cate-
itt & Jenkins, 1946) or explicitly clinical gories of Table 1. The Lorr and Jenkins
samples (e.g., Langner et al., in press; Peter- (1953) version was a second-order factor
son, 1961; Quay & Quay, 1965). The names comprising Jenkins and Glickman's Schizoid
given the syndromes are intended to reflect and Brain-Injured syndromes. The Quay,
the common elements in syndromes that Morse, and Cutler (1966) version was named
appear similar, whether or not the original Inadequacy-Immaturity and loaded highest
investigators used these particular names. on lack of interest, laziness in school, pre-
Only syndromes having at least five items occupation, daydreaming, and inattentive.
with factor loadings or average intercorrela- Achenbach (1966) called his version Severe
tions of .30 or higher are included. If the and Diffuse Psychopathology. For boys, the
original authors adopted a criterion more most heavily loaded items were bizarre be-
stringent than .30, syndromes are included havior, fantastic thinking, temper tantrums,
only if they comprised at least five items threatening people, and ideas of reference.
meeting the more stringent criterion. For girls, the most heavily loaded items were
The distinction between broad-band and withdrawn, bizarre behavior, confused, de-
narrow-band syndromes in Table 1 is based pression, and ideas of reference. The factor
on second-order analyses that have revealed obtained by Himmelweit (cited in Eysenck,
a hierarchical relationship between certain 1953) combined items like those found in
groupings of items (Achenbach, 1966, 1978; the studies by Lorr and Jenkins, Quay,
 CHILD PSYCHOPATHOLOGY 1285

Table 1
Syndromes Obtained Empirically in Two or More Studies

Source of data
Mental health workers
Case Direct
Syndrome histories reports Teachers Parents

Broad band
Overcontrolled 1BG, 14B 2SB 5, 7, 18, 22, 24B 2B, 3BG, 10, 21B
Undercontrolled 1BG, 14B, 20B 8B, 19, 2SB 5, 7, 18, 22, 24B 2B, 3BG, 10, 13, 21B
Pathological Detachment 1BG, 14B, 20B 22
Learning problems 7 19B
Narrow band
Academic Disability 6B 4, 12, 21B
Aggressive 1BG, IS, 16BG 6B, 17B, 23, 27 9 2B, 3BG, 4, 11, 12, 21B
Anxious 1C 6B, 26 9 11,36
Delinquent IB, 15, 16BG 1SB 2B, 3BG, 4, 11, 12, 21B
Depressed 1G, 16BG 23 2B, 3G, 4
Hyperactive 1G, 15, 16G 17B, 27 9 2B, 3BG, 4, 11, 12, 21B
Immature 6B 3B, 22B
Obsessive-Compulsive 1BG 2B, 3B
Schizoid 1BG, IS, 16G 17B, 23, 26, 27 2B, 3BG, 12
Sexual Problems IB 6B, 27 3G
Sleep Problems 4, 12, 21B
Social Withdrawal IS 23 9 2B, 3BG, 21B
Somatic Complaints 1BG 2B, 3BG, 4, 11, 21B
Uncommunicative 23 2B, 3B

Note. Each syndrome name is intended to reflect the common elements in syndromes that appear similar
but may have been given different names in different studies. See text for explanation of broad- and narrow-
band groupings. The numbers in the body of the table refer to the following studies, with B beside a number
indicating that the syndrome was found for boys, G indicating that it was found for girls, and no letter in-
dicating that the sexes were not analyzed separately: 1 = Achenbach (1966), 2 = Achenbach (1978),
3 = Achenbach & Edelbrock (in press), 4 = Arnold & Smeltzer (1974), 5 = Behar & Stringfield (1974)
6 = Borgatta & Fanshel (1965), 7 = Clarfield (1974), 8 = Collins, Maxwell, & Cameron (1962), 9 =
Conners (1969), 10 = Conners (1970), 11 = Conners (1973), 12 = Dreger et al. (1964), 13 = Ferguson,
Partyka, & Lester (1974), 14 = Himmelweit (cited in Eysenck, 1953), 15 = Jenkins (1966), 16 = Jenkins
& Glickman (1946), 17 = Kobayashi, Mizushima, & Shinohara (1967), 18 = Kohn & Rosman (1972),
19 = Lessing & Zagorin (1971), 20 = Lorr & Jenkins (1953), 21 = Miller (1967), 22 = Quay, Morse, &
Cutler (1966), 23 = Raff, Cromwell, & Davis (Note 4), 24 =• Ross, Lacey, & Parton (1965), 25 = Schaefer
& Millman (1973a), 26 = Spivack & Spotts (Note 3), 27 = Spivack & Spotts (1967).

Morse, and Cutler, and Achenbach, and it Sleep Problems were each found in three. An
was interpreted as a general abnormality Academic Disability syndrome was also found
factor. in four narrow-band analyses, and an anal-
Among the narrow-band syndromes there ogous syndrome, Learning Problems, was
is persuasive evidence for the generality of found in one broad-band analysis and in a
the Aggressive, Delinquent, Hyperactive, and second-order analysis of one of the narrow-
Schizoid syndromes, each of which was found band analyses (Miller, 1967). However, the
in 10-14 studies. There is also good evidence redundancy of items comprising this type of
for the Anxious, Depressed, Social With- syndrome leaves some doubt about whether
drawal, and Somatic Complaints syndromes, it is better viewed as a syndrome of be-
which were each found in six studies. Of the havior problems or as a single problem of
remaining syndromes, Sexual Problems was poor school performance. One version in-
found in four studies, and Immature, Ob- cluded math problems, numbers problems,
sessive-Compulsive, Uncommunicative, and and concepts problems (Clarfield, 1974); an-
1286 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

other included spells poorly, reads poorly, and syndromes were produced by ,6 of the 10 that
makes jailing grades in arithmetic (Dreger et employed present-absent response scales, 3
al., 1964); and a third included spells poorly, of the 6 that employed 3-step scales, and 7
reads poorly, and writes poorly (Miller, of the 9 that employed multistep scales. Even
1967). the largest discrepancy in proportions, that
between the 3-step and the multistep scales,
Methodological Differences and the did not approach significance.
Narrow-Band - Broad-Band It is difficult to assess the influence of
Distinction other differences in the studies, such as
subject characteristics and methods of analy-
Of the many methodological differences in sis, because not enough studies fall into
the first-order analyses summarized in Table clear-cut categories for comparison. The
1, the number of items analyzed had the effects of age and sex of subjects on the syn-
clearest effect on the narrow-band - broad- dromes would be of particular interest, but
band dichotomy. Narrow-band syndromes only a few studies have included parallel
were produced by only 1 of the 7 studies analyses of subjects grouped by age or sex
that employed 50 or fewer items, 7 of the (Achenbach, 1966, 1978; Achenbach & Edel-
11 that employed 51-100 items, and 8 of the brock, in press; Arnold & Smeltzer, 1974;
8 that employed more than 100. The differ- Borgatta & Fanshel, 1965; Jenkins & Click-
ence between the proportions for the studies man, 1946). These have generally revealed
employing SO or fewer items and those em- differences related to age or sex, but not
ploying more than 100 items was significant enough comparisons of this type have yet
at p = .002 (Fisher exact test). However, been made to determine whether certain
classification of studies according to number syndromes are peculiar to particular groups.
of items analyzed was somewhat confounded There is, however, already plenty of evidence
with type of rater, since 4 of the 6 studies of that the frequency of specific problems varies
teacher ratings employed SO or fewer items, with the age, sex, race, and socioeconomic
and the other two employed S1-100 items. status of the child, and in the case of teacher
Only 1 of the 6 teacher studies produced ratings, with the individual teacher, race of
narrow-band syndromes, compared with IS of teacher, and school (Eaves, 1975; Schultz,
the 20 studies employing other raters (p = Salvia, & Feinn, 1973, 1974; Shechtman,
.036). Yet deletion of teacher ratings from 1970, 1971; Touliatos & Lindholm, 1975).
the comparison between studies using 50 or Because most of the studies summarized in
fewer items and those using more than 100 Table 1 lumped together children of both
items still left significantly fewer studies sexes or diverse ages, the syndromes emerg-
producing narrow-band syndromes in the ing with regularity in various studies are
former group (p — .012). The failure to ob- likely to represent a conservative estimate of
tain teacher ratings on lengthy checklists the degree of differentiation obtainable if
makes it impossible to determine whether children were grouped by age and sex. More
teachers would produce more differentiated precise identification of age, sex, and situa-
syndromes if given enough items, but the tional differences in syndromes should cer-
lack of opportunity for teachers to observe tainly command high priority in future stud-
certain problem behaviors (e.g., sleeping and ies.
eating problems, bed-wetting) probably lim- Among the various statistical methods em-
its the degree of differentiation that can be ployed, the most popular have been principal
expected from them. components and principal factor analysis with
Unlike the number of items, the number varimax rotation. Since these methods have
of categories in the response scales does not produced a variety of broad- and narrow-
appear to have influenced the narrow-band - band factors, they do not by themselves
broad-band dichotomy. Among the studies appear to exercise significant constraints on
that described their response scales and re- the degree of differentiation obtained. How-
ported first-order analyses, narrow-band ever, unrotated solutions have not produced
 CHILD PSYCHOPATHOLOGY 1287

more than three strong syndromes (Achen- summing the raw scores of the items in a
bach, 1966; Collins et al., 1962; Himmel- syndrome, although in some studies these
weit, cited in Eysenck, 1953). The three scores were then standardized. Wherever pos-
studies using cluster analyses of items each sible, we have averaged correlations across
produced four or five clusters but were so syndromes within a study, using the z trans-
similar in approach that it is difficult to draw formation.
conclusions about the effects of clustering Table 2 shows that test-retest reliabilities
methods per se (Jenkins, 1966; Jenkins & for ratings by parents and teachers are quite
Glickman, 1946; Kobayashi et al., 1967). adequate for narrow-band as well as broad-
The more formalized methods of variable and band syndromes, although no test-retest reli-
subject cluster analysis that have since been abilities are available for ratings by mental
developed may well produce results quite health workers. Satisfactory stability is also
different from the relatively informal methods indicated by studies of teacher ratings over
employed in these studies. periods ranging from li months (Miller,
1972) to 6 months (Ross et al., 1965) and
Reliability and Stability of of parent ratings over 3 months (Miller et al.,
1972). Somewhat lower stabilities were re-
Syndrome Ratings
ported for child care workers' 6-month re-
One of the most pervasive complaints ratings of boys in residential treatment
about traditional adult taxonomies is that (Schaefer & Millman, 1973a) and parents'
they are unreliable. The meager existing evi- 6-, 15-, and 18-month reratings of children
dence on similar taxonomies of childhood dis- seen in child guidance clinics (Achenbach,
orders also indicates mediocre reliability, 1978, in press; Achenbach & Edelbrock, in
with Freeman (1971) reporting 59% agree- press). However, these stabilities were still
ment and Rutter et al. (1975) reporting substantial, considering that the subjects
67% agreement among child psychiatrists underwent treatment designed to change their
who used standardized case history materials behavior and that the 15- and 18-month
to classify children in terms of broad diag- reratings were made at the onset and after
nostic categories. The contribution of incon- the termination of clinic contacts. The
sistencies within diagnosticians to unreliabil- lengthiest follow-up showed statistically sig-
ity is indicated by Freeman's finding that nificant stability in parents' ratings over a
when his psychiatrists were given the same 5-year period (Gersten et al., 197,6).
materials to judge again after a 3-month Interrater reliabilities have generally been
interval, their second diagnosis agreed with of the same order of magnitude for teachers,
their first in only 72% of the cases. parents, and child care workers, and for
The translation of empirically derived syn- broad- and narrow-band syndromes. Since
dromes into diagnostic categories is taken up any observer in a position to observe a rep-
below, but first we consider various types of resentative cross section of a child's behavior
consistency in the scoring of syndromes. in natural situations is also likely to influ-
Table 2 presents an overview of findings on ence the behavior observed, very high agree-
syndromes that met our criterion of at least ment among such observers is probably an
five items with loadings or average inter- unrealistic goal. As can be seen from the
correlations of .30 or higher. The findings are bottom row of Table 2, agreement between
grouped in terms of short-term (1 month or raters who saw the subjects in distinctly
less) test-retest reliability, short-term (li-6 different situations was the lowest of all.
months) and long-term (15 months to 5 Only a few of the studies summarized in
years) stability, interrater agreement where Table 2 reported comparisons of the sample
two raters saw the subjects in approximately means of ratings obtained on particular syn-
the same situation, and interrater agreement dromes. Three studies showed slight de-
where two raters saw the subjects in different creases in most ratings of nonclinical sam-
situations. The correlations are generally be- ples made 1-2 weeks apart (Achenbach,
tween factor scores or scores computed by 1978; Achenbach & Edelbrock, in press;
1288 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

Table 2
Reliability and Stability of Syndrome Ratings
i
Correlation
Study Type of rater BB NB
Test-retest reliability (1 week to 1 month)
Achenbach (1978) P .93 .88
Achenbach & Edelbrock (in press) P .86 .87
Clarfield (1974) T .92"
Conners (1969) T .82
Cowen et al. (1973) T .83
Evans (1975) T .85
Short-term test-retest stability (If months to 6 months)
Achenbach (in press) P .77 .72
Behar & Stringfield (1974) T .83
Kohn & Rosman (1972) T .72
Miller (1972) T .84
Miller, Hampe, Barrett, & Noble (1972) P .87 .81
Ross, Lacey, & Parton (1965) T .89
Schaefer & Millman (1973a) MH .59
Long-term test-retest stability (15 months to 5 years)
Achenbach (1978) P .68 .60
Achenbach & Edelbrock (in press) P .49 .56
Gersten, Langner, Eisenberg, Simcha-Fagan,
& McCarthy (1976) P .39
Interrater reliability
Achenbach (1978) P .76 .72
Achenbach & Edelbrock (in press) P .70 .65
Behar & Stringfield (1974) T .78
Kohn & Rosman (1972) T .83"
Millman & Davis (1975) MH .43
Peterson (1961) T .76
Peterson, Becker, Shoemaker, Luria, &
Hellmer (1961) T .76
Peterson et al. (1961) P .50
Quay, Sprague, Shulman, & Miller (1966) P .73
Schaefer & Millman (1973a) MH .591-
Schaefer & Millman (1973b) MH .86°
Spivack & Levine (1964) MH .84d
Different raters seeing subjects in different situations
Kohn & Rosman (1973a) T, Te .46
Peterson et al. (1961) P, T .33
Quay (1964) MH« .24
Quay (1966) T, CW .29
Quay & Quay (1965) T' .45
Quay & Quay (1965) T« .51
Brown, Achenbach, & Minichiello (Note 6) T, Th .19
Quay, Sprague, Shulman, & Miller (1966) M, T .37
Quay, Sprague, Shulman, & Miller (1966) F, T .27
Note. Correlations are fs unless otherwise indicated. NB = narrow-band syndromes; BB = broad-band
syndromes; P = parents; T = teachers; MH = mental health workers; Te = testers; CW = cottage
workers; M = mothers; F = fathers.
0
Total score r.
b
Spearman-Brown r.
"Spearman-Brown Mdn.
* Day-night corrections officers.
' Seventh grade teachers.
8
Eighth grade teachers.
h
Regular teachers, hospital teachers.
 CHILD PSYCHOPATHOLOGY 1289

Evans, 1975). A fourth study showed similar data for referred and nonreferred children
decreases over a 3-month interval (Miller et combined, the correlations between parents
al., 1972). This suggests a possible tendency averaged .60; between parents and clinicians,
to report less pathology for normal children .37; between parents and teachers and be-
on a second trial following fairly soon after tween clinicians, .35; and between clinicians
the first. In 1-month reratings of placebo- and teachers, .24. When data were analyzed
treated disturbed children, Conners (1969) separately for referred and nonreferred chil-
found teachers to report fewer problems on dren, agreement between parents was also
three syndromes and more problems on two. higher for both groups than was agreement
Over longer periods, Schaefer and Millman between any other combination of observers.
(1973b) found decreases for most syndromes It thus appears that neither professional
scored at a 12-month interval for children in training, nor detachment from family dynam-
residential treatment, and Gersten et al. ics, nor the relatively standardized situations
(1976) found significant decreases on six in which children are seen by teachers and
syndromes and increases on five in parents' clinicians increase interobserver agreement
ratings of general population samples S years for either referred or nonreferred children
apart. Ross et al. (1965) found a significant over what can be obtained from parents. The
increase in aggressive problems and insignifi- higher agreement of clinicians and teachers
cant increases on other scales scored by with parents than with each other also sup-
teachers of untreated boys at a 6-month ports the value of parent reports.
interval. In the only direct comparisons of Besides showing higher levels of agreement,
different types of raters, no significant dif- parents are likely to provide more complete
ferences were found in syndromes scored pictures of their children's behavior than are
from ratings by mothers and fathers (Achen- other observers, according to findings by
bach, 1978; Achenbach & Edelbrock, in Novick, Rosenfeld, Bloch, and Dawson
press), but preschool teachers reported fewer (1966). These investigators compared the
problems than teacher aides did, although no number of deviant behavior items reported
statistical test was cited (Behar & String- for clinic children by each parent, the child's
field, 1974). The evidence on agreement in teacher, a home observer, a school observer,
levels of scores is thus rather meager, even and a social worker who conducted a 2-3-
though decisions about the classification of hour intake interview with each family. Of all
individuals are likely to rest more directly valid items reported by at least one source,
on the size of their scores than on group mothers reported 63%, fathers 55%, teachers
correlations between the scores and other 22%, home observers 14%, school observers
variables. 12%, and social workers IT%. The impact
Because children's behavior is differentially of parents' reports on clinical judgments is
affected by different settings and observers, also apt to be substantial; McCoy (1976)
obtaining high agreement among diverse ob- found that clinical trainees' ratings of chil-
servers is perhaps less important than deter- dren's adjustment and need for treatment
mining which sources of observations reveal were influenced much more by parents' re-
stabilities that are in turn related to etiology, ports than by filmed segments of the chil-
prognosis, and the effectiveness of possible dren's playroom behavior, even when there
treatment approaches. It is clear that we were major contradictions between what the
need far more systematic comparisons of parents reported and what the films showed.
the value of different types of data as guides The greater confidence invested in parents'
to prognosis and the prescription of treat- reports may not be entirely misplaced, how-
ment. However, a study by Miller (1964) ever, because Lobitz and Johnson (1975)
provided at least some suggestive data on found that clinic-referred children did not
the relative value of various informants. Mil- differ significantly from nonreferred children
ler obtained Q sorts on clinic-referred chil- when observed in a standardized clinic en-
dren and their nonreferred siblings by their vironment but did show significantly more
parents, teachers, a psychologist who saw deviant behavior when observed at home.
them in a family interview, and a psychia- Taken together, these findings indicate that
trist who saw them in a play session. With parents are typically the most important
1290 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

source of data on child behavior problems and similar but much shorter AML. Both instru-
that their reports should be systematically ments yielded factors corresponding to the
integrated into classification procedures broad-band Undercontrolled and Overcon-
whenever possible. trolled syndromes, plus a Learning Problem
factor, although only one item represented
Validity of Syndrome Ratings this factor on the AML. When teachers rated
The question of how to validate empiri- TRF items on 3-point scales and AML items
cally obtained syndromes raises a host of on 5-point scales for primary graders re-
conceptual issues. The lack of well-estab- ferred to the PMHP, the cross-instrument
lished diagnostic categories is what motivates correlations were .71 for the Undercontrolled,
most of the efforts in the first place, but this .26 for the Overcontrolled, and .48 for the
lack also makes it difficult to select criteria Learning Problem scores. When teachers used
against which to validate the results of such S-point scales to rate all their pupils on both
efforts. Even if we possessed better estab- instruments, the cross-instrument correlations
lished categories of the DSM type for child- rose to .81, .62, and .64, respectively. Posi-
hood disorders, they would be questionable tive correlations among nearly all combina-
as validity criteria, for little relationship has tions of scores reflected the presence of a
been found between DSM-type classifications general problem behavior dimension in the
of adult patients and the treatments they teachers' ratings. In fact, the correlation of
receive (Bannister, Salmon, & Leiberman, .63 between the Overcontrolled cluster of the
1964). In the absence of an accepted system AML and the Undercontrolled cluster of the
of constructs and criterion measures, at- TRF in the second sample actually exceeded
tempts to validate syndromes have generally the correlation of .62 between the Undercon-
involved "bootstrapping" of one sort or an- trolled clusters of the two instruments.
other; that is, investigators have attempted Although their ratio of subjects to items
to "lift themselves by their own bootstraps" fell below our criterion for establishment of
by establishing relationships among measures, syndromes, Lessing and Zagorin (1971) fac-
all of which are known to be imperfect. Al- tor analyzed and intercorrelated the factors
though bootstrapping seldom yields very obtained from the 58-item Peterson (1961)
satisfying results in the short run, it is Problem Checklist and the SS-item Wichita
probably an inescapable stage in the quest Guidance Center Checklist filled out by moth-
for better taxonomies of behavior disorders. ers of 102 clinic-referred 10- to 12-year-olds.
The efforts at validation can be divided into Of the four factors extracted from the PPC,
those that have assessed relations either the largest two had congruence coefficients
between syndromes scored with different in- ranging from .80 to .88 with the two factors
struments in a particular population or with obtained previously for children rated by
the same instrument in different populations, teachers in Grades 5-7 (Peterson, 1961;
those that have sought to establish criterion- Quay & Quay, 19,65), and from .73 to .88
referenced validity with respect to indepen- with factors in the 3-factor solutions for
dent diagnostic categorizations, and those eighth graders and preadolescent delinquents
that have assessed external correlates of syn- (Quay, 1964; Quay & Quay, 1965). Factor
dromes other than diagnostic categorizations. scores on the Conduct Problem factor ob-
Where these findings are relevant to ques- tained from the PPC by Lessing and Zagorin
tions of validity in ratings, we include them, correlated .79 with a Conduct Problem factor
even if they are from studies that did not they obtained from the WGCC, but the other
meet our criteria for initial establishment of three WGCC factors had no clear counter-
syndromes. parts in the PPC, and the highest remaining
correlation between factor scores was .45.
Cross-Instrument and Cross-Population Three factors extracted from the Institute
Relations for Juvenile Research Checklist filled out by
Cowen et al. (1973) reported relations psychiatrists also had no clear counterparts
between ratings on their TRF and their in the two parent checklists, since the highest
 CHILD PSYCHOPATHOLOGY 1291

intercorrelation between factor scores was correlated .81 and .62 with scores on Con-
.48 for the Conduct Problem factor of the ners's (1973) Conduct Problem and Anxiety
PPC and a Delinquent Conduct factor of the factors, respectively. Scores on those of Ach-
IJRC. enbach's narrow-band scales that correspond
Two other studies have reported relations to Conners's Conduct, Anxiety, Hyperactive,
between versions of Peterson's checklist and Psychosomatic, and Antisocial factors had
syndromes scored from other instruments. In correlations with the Conners scores ranging
one, teachers of disturbed 7-14-year-old boys from .39 to .78, with a mean of .62. In the
completed the Quay-Peterson (Note 5) BPC same study, the Conners (1969) Teacher
and the Devereux Elementary School Be- Rating Scale was filled out by each child's
havior Rating Scale (DESB; Proger, Mann, regular classroom teacher and by a teacher
Green, Bayuk, & Burger, 197S). Even who instructed the child while he was hos-
though the syndromes scored from the DESB pitalized for evaluation. The highest correla-
did not meet our criterion of at least five tions of the teacher and parent instruments
items with loadings .30 or higher, scores on were for hyperactivity, where the correlation
the BPC Conduct Problem factor correlated was .57 between ratings by parents and class-
.81 with scores on two DESB scales, entitled room teachers and .49 between ratings by
Classroom Disturbance and Disrespect-Defi- classroom and hospital teachers.
ance. Conduct Problem also correlated .53 Three studies have reported coefficients of
with the DESB Irrelevant Responsiveness congruence between factors obtained on ver-
scale. The next highest correlation of any sions of the BPC for various populations.
of the four BPC scores and the DESB was Quay, Morse, and Cutler (1966) obtained
.50 between the BPC Immaturity score and congruence coefficients ranging from .59 to
the Inattentive-Withdrawn score of the .94 (Mdn = .&5) for the Conduct and Per-
DESB. sonality factors and from .44 to .89 (Mdn =
In a test of the relationships between syn- .59) for the Immaturity factor, derived from
dromes across time and raters, as well as samples of emotionally disturbed pupils, nor-
across instruments, Kohn and Rosman mal eighth graders, adolescent delinquents,
(1972) obtained first grade teachers' ratings and preadolescent delinquents. In the second
on Peterson's (1961) Problem Checklist for study, Quay (1966) reported congruence co-
children who had been rated 12 and 18 efficients ranging from .57 to .73 for the
months earlier by preschool teachers on the three largest factors obtained from ratings
Apathy-Withdrawal and Anger-Defiance syn- of case histories of preadolescent and ado-
dromes of the combined Kohn Social Compe- lescent delinquents. In the remaining study,
tence Scale and Symptom Checklist. At both teachers' and mothers' ratings of 34 BPC
the 12- and 18-month intervals, the correla- items were factor analyzed for only 34 learn-
tions were .28 between the PPC Personality ing-disabled children, but the three largest
Problem score and the Kohn Apathy-With- factors had congruence coefficients ranging
drawal score, and .40 between the PPC Con- from .82 to .89 with the corresponding fac-
duct Problem score and the Kohn Anger-'De- tors obtained in the Quay studies (Paraske-
fiance score. Much higher correlations of .76 vopoulos & McCarthy, 1970).
and .80 have been obtained between teach- It is clear from the above that the broad-
ers' concurrent ratings of preschoolers on band undercontrolled syndrome is highly
Kohn's two factors and the corresponding robust and consistently reappears when the
two factors of Behar's Preschool Behavior same or similar instruments are used by
Questionnaire (Behar, 1977). different raters to rate the same or different
In a study by Brown, Achenbach, and populations. The overcontrolled syndrome is
Minichiello (Note 6), Conners's Parent's somewhat less robust, and its high correla-
Questionnaire and Achenbach's CBCL were tion with the undercontrolled syndrome in
filled out by parents of boys referred for the Cowen et al. (1973) study suggests that
evaluation of hyperactivity. Achenbach's Cowen's version of it may reflect general
(1978) Externalizing and Internalizing scores school maladjustment more than a pattern of
1292 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

behavior distinctly different from his ag- referenced validity thus indicates consider-
gressive-outgoing syndrome. However, other able power for the undercontrolled syndrome,
versions of the overcontrolled syndrome de- but less for the overcontrolled syndrome, es-
serve considerable confidence. pecially Cowen's version of it.

Criterion-Referenced Validity Nondiagnostic Correlates oj Syndromes

Due perhaps to the lack of well-differenti- The most common form of bootstrapping
ated and reliable diagnostic categories, the has been to seek correlates of syndromes that
few existing studies relating empirically ob- would increase the interpretability or useful-
tained syndromes to independent diagnostic ness of the syndromes in the absence of defin-
categorizations have each used only a small itive validity criteria. In several studies, this
number of broad-band categories. Ross et al. has primarily entailed demonstrating that
(1965) compared the Aggressive, Withdrawn, groups identified as disturbed had signifi-
and Prosocial Behavior scores obtained on the cantly higher behavior problem scores than
PASS filled out by teachers of boys who had did normal comparison groups (Achenbach,
been independently classified by a clinical 1978; Achenbach & Edelbrock, in press;
psychologist as aggressive, withdrawn, or well Behar & Stringfield, 1974; Cowen et al.,
adjusted. Using as a cutoff point the eighty- 1973; Ferguson et al., 1974; Kohn & Ros-
fifth percentile of scores obtained on each man, 1973b; McCarthy & Paraskevopoulos,
scale by a random sample of boys, Ross 1969). More pertinent to the interpretation
found 94% of the aggressive boys to be above of syndromes are comparisons of children
the cutoff on the Aggressive scale, 61% of who were differentiated according to the syn-
the withdrawn boys to be above the cutoff on dromes. The broad-band Overcontrolled and
the Withdrawn scale, and 33% of the well- Undercontrolled syndromes have provided the
adjusted boys to be above the cutoff on the basis for most of these comparisons.
Prosocial scale. None of the well-adjusted Beginning with his 1966 study, Achen-
boys were above the cutoff point on the clini- bach's versions of the Over- and Undercon-
cal scales, but 22% of the aggressive boys trolled syndromes have been used to classify
were above the cutoff point on the With- children as internalizers if 60% or more of
drawn scale, and 11% of the Withdrawn their reported problems came from the In-
boys were above the cutoff on the Aggressive ternalizing (Overcontrolled) syndrome, and
scale. The scales thus demonstrated substan- as externalizers if .60% or more of their
tial, though far from perfect, convergent and problems came from the Externalizing (Un-
discriminant validity with respect to the dercontrolled) syndrome. Classification of
clinical psychologist's categorization. each sex according to the Internalizing and
Cowen et al. (1973) classified children Externalizing syndromes obtained for that
referred to the PMHP as manifesting pri- sex showed that externalizers outnumbered
marily acting-out, withdrawn, or learning internalizers by about two to one among boys,
problems. Comparisons of scores on the AML whereas this ratio was reversed for girls
showed the acting-out group to have higher (Achenbach, 1966). Within-sex comparisons
scores than either of the other groups on the have generally shown that internalizers were
A (aggressive-outgoing) scale, whereas the adapting better, as reflected in school per-
learning problem group had the highest score formance, scores on standardized tests, teach-
on the L (learning problems) scale, which ers' and peers' ratings, premorbid social
consisted of only one item. However, the M problems, and performance on experimental
(moody-internalized) scale lacked discrimi- measures of impulsivity, delay of gratifica-
nant power; the acting-out and learning tion, and foresight (Achenbach, 1966; Ach-
problems groups both obtained higher scores enbach & Lewis, 1971; Rolf, 1972; Rolf &
on it than did the withdrawn group. Like the Garmezy, 1974; Weintraub, 1973). Inter-
cross-instrument and cross-population com- nalizers have also been found to display more
parisons, the meager evidence on criterion- self-image disparity (Katz, Zigler, & Zalk,
 CHILD PSYCHOPATHOLOGY 1293

1975) and to remain longer and improve by the narrow-band syndromes. In contrast,
more in psychotherapy (Achenbach & Lewis, children categorized by the Aggressive syn-
1971). In addition, parents of internalizers drome were lower in SES, had parents with
have been found to differ from those of more social problems, and had lower IQs
externalizers in being more strict with their than did children categorized by the other
child and more concerned about the child's narrow-band syndromes.
problems and in having fewer marital sepa- In addition to Achenbach's version of the
rations, fewer social problems, less overall over- and undercontrolled syndromes, the
pathology, and different Minnesota Multi- versions of these syndromes emerging from
phasic Personality Inventory (MMPI) pro- teachers' ratings in the PMHP have been
files (Achenbach, 1966; Achenbach & Lewis, used to classify children in various ways and
1971; Anderson, 1969; Weintraub, 1973; to compare them with children whose pri-
Butcher, Note 7). mary problem was in school learning. It has
A follow-up of disturbed adolescents into been found that children manifesting the
adulthood (mean age = 38) has shown that Overcontrolled syndrome improved more than
even when internalizers and externalizers the other two groups during their contact
were matched on many demographic varia- with the project (Lorion & Cowen, 1976;
bles, they differed in the following ways Lorion, Cowen, & Caldwell, 1974), whereas
(Hafner, Quast, & Shea, 1975): Across both children manifesting the Undercontrolled
sexes, internalizers performed better in syndrome were significantly better known but
school, completed more grades, more fre- less liked by their teachers than were the
quently finished high school, had more other two groups (Cowen, Lorion, & Wilson,
friends, and had higher mental health rat- 1976).
ings than externalizers did. Within-sex com- Jenkins (1966) also examined correlates of
parisons showed that Internalizing males had the Overcontrolled-Undercontrolled dichot-
better job stability, higher adult socioeco- omy by comparing background characteristics
nomic status, fewer incarcerations, fewer of children classified by his Shy-Seclusive
marriages, and fewer divorces than External- and Overanxious-Neurotic syndromes with
izing males, whereas fewer Internalizing fe- children classified by his Hyperactive-Dis-
males married very early, were divorced, and tractible, Undomesticated, and Socialized De-
were placed in mental hospitals than were linquent syndromes. Consistent with com-
externalizing females. There were also a parisons based on the Internalizing-External-
number of significant differences in the izing dichotomy, Jenkins found that fewer
MMPI profiles of internalizers and external- children manifesting the first two syndromes
izers of both sexes. In contrast with the were from broken homes and had mothers
relatively poor adult outcomes of external- hostile to them than children manifesting
izers found by Hafner et al. among hetero- the other three syndromes. Statisically sig-
geneous groups of disturbed adolescents, Roff, nificant differences in biographical character-
Knight, and Wertheim (1976) found that istics were also found in comparisons of Shy-
among males admitted to Veterans Admin- Seclusive children with Overanxious-Neurotic
istration hospitals with a diagnosis of schizo- children and in comparisons of children
phrenia, those classified as internalizers on grouped by the other three syndromes, but
the basis of their child guidance clinic rec- because there was no report of the total
ords tended to have poorer outcomes than number of variables assessed, it is not clear
those classified as externalizers. whether these differences exceeded chance
Although fine-grain statistical comparisons expectations.
have not been made among the narrow-band In summary, the many differences found
syndromes identified by Achenbach (1966), between children manifesting the over- and
children of both sexes who were categorized undercontrolled syndromes indicate that
by his Obsessions, Compulsions, and Phobias these are very different types of children
syndrome had the highest mean IQs and so- living in very different families: Undercon-
cioeconomic status of all groups categorized trolled children and their families are in
1294 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

more open conflict with other people, are less the role of syndromes in classifying individu-
socially competent, and are less appropriate als. One approach is to regard a syndrome as
candidates for traditional mental health ser- representing a type of individual, in the
vices. The one exception to the picture of sense of a personality or character type that
poorer adaptation by undercontrolled sub- endures beyond the precipitating events that
jects was found in the Roff et al. (1976) bring the individual to professional attention.
study, where all subjects had to be initially A second approach is to regard a syndrome
competent enough to be accepted by the mili- as representing a reaction type, the form of
tary and then had to be disturbed enough to which may be determined as much by spe-
be hospitalized and diagnosed schizophrenic. cific precipitating stresses as by enduring
It is not clear whether these selective factors characteristics of the individual. A third
were responsible for the finding by Roff et al. approach is to regard a syndrome as nothing
that Overcontrolled subjects had worse out- more than a collection of specific behaviors
comes, but the obtained relationship may that happen to be statistically associated be-
well be a joint function of the pattern of cause the environmental contingencies sup-
problems (Undercontrolled vs. Overcon- porting them are statistically associated but
trolled) and the presence of a psychotic dis- that do not represent either a type of indi-
order. Since classification by Achenbach's vidual or a structural reaction type. A fourth
(1966) narrow-band Schizoid syndrome has approach is to regard syndromes as reflecting
been found to include both Over- and Under- dimensions or traits, such that individuals
controlled individuals, it may be that be- are best described in terms of their scores on
cause Overcontrolled individuals who suffer all syndromes rather than by categorization
from schizoid conditions create fewer prob- according to their resemblance to a particu-
lems for other people, they would be more lar syndrome.
likely to be accepted by the military than Whichever view of syndromes is adopted,
would Undercontrolled individuals who suffer the problem of discriminating among indi-
from equally severe schizoid conditions. If viduals must be addressed before empirically
this were the case, more severely disturbed derived syndromes can be translated into
Overcontrolled than Undercontrolled indi- classification systems for research or clinical
viduals could have qualified for the Roff et purposes. The different viewpoints may differ
al. sample and, in turn, could have produced in the discrimination procedures they dic-
a distribution of outcomes relatively favor- tate, whether the procedure is assignment to
able to the undercontrolled. In any case, it mutually exclusive generic categories, classi-
appears that classification by the broad-band fication by a single behavior problem, or
syndromes is related to numerous other char- analysis of profiles of scores obtained on
acteristics in groups that are heterogeneous multiple syndromes. However, if individuals
with respect to traditional diagnostic cate- are successively grouped according to the dic-
gories and that such classification may also tates of each theoretical approach, these
discriminate among individuals who are groupings can be compared with respect to
homogeneous with respect to broad tradi- their mutual agreement and to how the indi-
tional categories such as schizophrenia but viduals assigned to different categories are
who differ in outcome. found to differ in other ways. Computing
correlations between syndrome scores and
From Syndromes to Categories other characteristics is not an adequate sub-
stitute, because correlations by themselves
Even though most of the efforts reviewed do not provide a basis for decisions about
here have been motivated by the need for an individual cases.
improved classification system, there has been To date, only the broad-band Over- and
remarkably little translation of statistical Undercontrolled syndromes have been em-
associations among problems into procedures ployed on a significant scale to group chil-
for discriminating among individuals. In dren who could then be compared on other
theory, there are at least four ways to view characteristics. These comparisons have re-
 CHILD PSYCHOPATHOLOGY 1295

vealed many differences that provide a fairly the children in question. The multitude of
coherent picture of children classified by the variables that may each make small but sig-
two syndromes, but much more work is nificant contributions also calls for sophisti-
needed to identify etiological factors and the cated strategies of data reduction and analy-
differential effects of various treatment ap- sis. However, unless the products of such
proaches on the children categorized by the work are translated into concepts that can be
syndromes. Furthermore, the exclusion of used by the average practitioner, they are
substantial proportions of children from unlikely to benefit children in need of mental
these groupings and the considerable hetero- health services.
geneity of children included in the groupings
both argue for using narrower band syn- Cluster Analysis of Profiles
dromes in future attempts at categorization.
Effective utilization of narrow-band syn- One strategy for translating narrow-band'
dromes raises a number of methodological syndromes into differentiated categories of
issues, however. First, sample sizes must be children is to cluster analyze profiles of
larger than in a dichotomous classification in scores. Considering the lack of standardiza-
order to ensure enough subjects in each tion of cluster-analytic methods and the fact
narrow-band category for purposes of com- that most were developed for use with data
parison. Second, much more attention must unrelated to child psychopathology, it is not
be paid to the distribution of subjects on surprising that they have rarely been applied
such variables as age, sex, socioeconomic to the classification of children's behavior
status, and severity of disturbance because disorders. To our knowledge, only a handful
the narrow-band syndromes may characterize of researchers have applied formal cluster
more restricted groups than the broad-band analyses to empirically derived behavior
syndromes do (cf. Achenbach, 1966). Third, problem groupings in order to form cate-
differentiated assessments of competencies as gories of children who could then be com-
well as problems are needed to take account pared on characteristics not included in the
of the different kinds of adaptation possible cluster analyses. Three examples are pre-
for different kinds of children in various en- sented here as illustrations of a possible ap-
vironments. Fourth, use must be made of proach to more comprehensive utilization of
methods of categorization designed to capi- empirically obtained syndromes, but they
talize on whatever differences among chil- should not necessarily be regarded as defini-
dren can be reliably measured and can in- tive prescriptions for what is needed.
crease discriminative power with respect to In one research program, hierarchical
differences in etiology, prognosis, and appro- cluster analyses that grouped subjects whose
priate treatments. Thus, because behavior profiles of scores were most similar in terms
ratings are inevitably crude and because of Euclidean distance were applied to scores
most behavior disorders are unlikely to on 11 dimensions identified by factor analy-
yield the kinds of precise and positive cri- sis of teachers' ratings on the DESB (Spi-
teria obtainable for organic diseases, it is im- vack, Swift, & Prewitt, 1971). Separate
portant to determine how other character- cluster analyses of children at each level
istics of children, such as developmental from kindergarten through sixth grade
level and various competencies, can add to yielded six clusters that replicated well and
the power of classification systems. classified over 85^ of the subjects. Children
In sum, we need a programmatic approach classified by these clusters were found to
to the differentiation of disturbed children in differ significantly in IQ, school achievement,
relation to etiology, developmental level, parents' education, race, and sex.
adaptive patterns, prognosis, and differential In another research program, maximum
treatment effectiveness, as opposed to a shot- (complete) linkage cluster analysis using
gun approach utilizing whatever subjects and Euclidean distance was applied to profiles of
observers are at hand with no provision for scores on 18 behavior problem dimensions
linking obtained syndromes to the needs of previously identified through factor analysis
1296 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

of behavior checklists filled out by interview- Conclusions

ers from mothers' reports (Langner et al., in
press). The nine profile types derived were Despite great diversity in instruments,
given the following labels: Sociable, Competi- subjects, raters, and statistical methods, em-
tive-Independent, Aggressive, Dependent, pirical efforts have produced a number of
Moderate Backward Isolate, Severe Back- broad- and narrow-band syndromes of child
ward Isolate, Delusional, Self- and Other behavior problems with considerable con-
Destructive, and Organic. Five of the profile sistency. The reliability and stability of re-
types had significantly different frequencies peated ratings by the same observers are
among children differing in ethnic back- generally quite adequate for these syndromes,
ground (Langner, Gersten, & Eisenberg, although agreement among different observ-
1974). There was also a significant associa- ers is lower, especially among observers hav-
tion between the classification of children ing different roles with respect to the sub-
according to profile types derived from jects and viewing them in different situations.
parent interviews 5 years apart (Langner, Because observers and situations inevitably
Note 8). Applying cluster analysis in a simi- affect children's behavior, it is probably more
lar way to data on children whose families profitable to determine which observers'
were on welfare, six profile types were found, ratings are most predictive of other impor-
most of which were similar to profile types tant characteristics than to strive for high
found in the previous sample, which had levels of agreement among diverse observers.
excluded welfare families (Eisenberg, Ger- Now that some consistencies have been es-
sten, Langner, McCarthy, & Simcha-Fagan, tablished, it is clear that certain reorienta-
1976). These types were significantly associ- tions are necessary if research in this area is
ated with ethnic background, as well as with ever to benefit disturbed children. At a
school performance, trouble with the police, methodological level, it appears that the time
and referral for mental health services. has come for standardization of instruments
In the third research program, centroid and methods of analysis across studies. It is
cluster analysis using correlation as the simi- no longer useful for investigators to devise
larity measure was applied to the behavior their own rating instruments, score them for
problem scales of the editions of the Child whatever subjects are at hand, and analyze
Behavior Profile that have been standardized them with whatever programs their computer
for children aged 6-16 (Edelbrock & Achen- centers happen to offer. There are several
bach, Note 9). By independently cluster instruments for which a considerable body
analyzing two samples of 200 clinic cases for of data exists on substantial samples. Among
each of the age and sex groups, six profile those for which only broad-band syndromes
types were found that replicated well across have been obtained, the most prominent are
samples for the 6—11 -year-olds of both sexes, the BPC (Quay & Peterson, Note S) for use
five profile types were found that replicated by a wide variety of raters, the TRF (Clar-
for the 12-16-year-old boys, and seven pro- field, 1974) for use by elementary school
file types were found that replicated for the teachers, and the Kohn Symptom Checklist
12-16-year-old girls. These profile types and Social Competence Scale (Kohn & Ros-
showed statistically significant stability in 6- man, 1972) for use by preschool teachers.
month and 18-month follow-ups of children Among those that reflect more differentiated
treated in guidance clinics. Furthermore, syndromes are the School Behavior Check-
there were significant interactions between list (Miller, 1972) and Teacher Rating
type of profile and outcome at follow-up, Scale (Conners, 1969) for elementary school
with some profile types showing marked de- teachers, the CBCL (Achenbach, 1978) and
clines in reported problems, whereas others Parent's Questionnaire (Conners, 1970,
did not. These findings indicate that children 1973) for parent ratings, and the DCB (Spi-
differing in initial profile type differ markedly vack & Spotts, Note 3) for child care work-
in their prognoses under current mental ers. If none of these instruments is totally
health services. satisfactory, a study's contribution will nev-
 CHILD PSYCHOPATHOLOGY 1297

ertheless be far greater if it at least includes In order for any research-based taxonomy
one of them along with whatever new instru- to become part of the service system, its
ment the investigator is inspired to develop virtues cannot be limited to quantitatively
so that the findings can be integrated with demonstrated reliability and validity. It must
previous work in the field. also provide categories, concepts, and corre-
Along with the need for greater uniformity lates of demonstrated value to practitioners
of instrumentation and methods of analysis, of a variety of professions and persuasions.
there is a need for greater differentiation in Yet, because service traditions are not readily
the samples studied. Because so few sys- altered by research endeavors, researchers
tematic comparisons have been made between are likely to have an impact only if they
samples differing in such obvious character- actively demonstrate the value of their pro-
istics as age and sex, it is likely that patterns ducts within the context of everyday ser-
of problems existing among children of one vices. This will require a concerted effort by
sex or particular age groups are obscured by researchers to persuade practitioners to try
analyses performed on heterogeneous samples. their products in return for whatever the
Thus, to obtain more precision in the descrip- researchers can provide in the way of feed-
tion of different kinds of children, it is essen- back, useful clinical tools, and a clear demon-
tial that standardized methods for deriving stration of improved services to clients. If
syndromes be applied to samples that differ researchers succeed in putting their products
systematically on demographic variables, into more clinically usable forms and work-
In addition to these methodological reori- ing with practitioners to implement them, the
entations, a fundamental conceptual reorien- result should be a more programmatic ap-
tation is needed to move from the empirical proach to research that is better matched to
derivation of syndromes toward the classifica- the practical realities of our service system.
tion of children in ways that will be useful It is both to meet the needs of this system
to researchers and practitioners alike. This and to reform it that future efforts at the
requires explicit translation of statistical as- development of classification systems for
sociations among items into groupings of indi- child psychopathology should be directed.
viduals who can then be compared on other
important characteristics. The few efforts in
this area have primarily involved classifica- Reference Notes
tion of children according to only two or 1. Group for the Advancement of Psychiatry. Psy-
three broad-band syndromes, although for- chopathological disorders in childhood: Theo-
mal cluster analysis of profiles of scores is retical considerations and a proposed classification
also beginning to be used. Beyond the con- (Report No. 62). New York: Author, 1966.
ceptual issues involved in moving from the 2. Schaefer, E. S. Major replicated dimensions of
identification of syndromes to the categoriza- adjustment and achievement: Cross-cultural,
cross-sectional, and longitudinal research. Paper
tion of individuals, however, loom larger presented at the meeting of the American Edu-
issues concerning the varied purposes for cational Research Association, Washington, D.C.,
classification, the appallingly primitive state April 1975.
of mental health services to children, and the 3. Spivack, G., & Spotts, J. Childhood symptoma-
gulf between the "two cultures" of research- tology: Further data defining the meaning of
the Devereux Child Behavior (DCB) Rating
ers and practitioners. It is clear that a better Scale factors. Devon, Pa.: Devereux Foundation
system for classifying child psychopathology Institute, 1966.
is a prerequisite not only for improving re- 4. Raff, C. S., Cromwell, R. L., & Davis, D. W.
search on etiology, epidemiology, prognosis, Analysis of the acting-out and withdrawn-psy-
and treatment but also for improving com- cotic constructs in emotionally disturbed children.
munication among people responsible for dis- Paper presented at the meeting of the Midwestern
Psychological Association, Chicago, May 1968.
turbed children, for improving the training
5. Quay, H. C., & Peterson, D. R. Manual for the
of practitioners, and for promoting more Behavior Problem Checklist. Unpublished manu-
rational social policies concerning children script, University of Illinois, 1967; Revised edi-
(cf. Hobbs, 197Sa, 197Sb). tion, University of Miami, 197S.
1298 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

6. Brown, G. L., Achenbach, T. M., & Minichiello, Behar, L. B. The Preschool Behavior Questionnaire.
M. D. Parent and teacher ratings of hyperactive Journal of Abnormal Child Psychology, 1977, 5,
children. Manuscript in preparation, 1978. 265-295.
7. Butcher, J. N. MMP1 characteristics of internal- Behar, L. B., & Stringfield, S. A behavior rating
izing and externalizing boys and their parents. scale for the preschool child. Developmental Psy-
Paper presented at the First Conference on chology, 1974, 10, 601-610.
Recent Developments in the Use of the MMPI, Borgatta, E. F., & Fanshel, D. Behavioral character-
Minneapolis, March 1966. istics of children known to psychiatric outpatient
8. Langner, T. S. Personal communication, Novem- clinics. New York: Child Welfare League of
ber 22, 1976. America, 1965.
9. Edelbrock, C. S., & Achenbach, T. M. Child Be- Clarfield, S. P. The development of a teacher re-
ferral form for identifying early school maladap-
havior Profile patterns of children referred for
clinical services. Paper presented at the 86th An- tation. American Journal of Community Psychol-
ogy, 1974, 2, 199-210.
nual Convention of the American Psychological
Association, August 1978. Collins, L. F., Maxwell, A. E., & Cameron, K. A
factor analysis of some child psychiatric clinic
data. Journal of Mental Science, 1962, 108, 275-
References 285.
Conners, C. K. A teacher rating scale for use in drug
Achenbach, T. M. The classification of children's studies with children. American Journal of Psy-
psychiatric symptoms: A factor-analytic study. chiatry, 1969, 126, 884-888.
Psychological Monographs, 1966, 80(1, Whole No. Conners, C. K. Symptom patterns in hyperkinetic,
615). neurotic, and normal children. Child Develop-
Achenbach, T. M. The Child Behavior Profile: I. ment, 1970, 4, 667-682.
Boys aged 6-11. Journal of Consulting and Clini- Conners, C. K. Rating scales for use in drug studies
cal Psychology, 1978, 46, 478-488. with children. Psychopharmacology Bulletin:
Achenbach, T. M. The Child Behavior Profile: An Pharmacotherapy with Children (Department of
empirically based system for assessing children's Health, Education, and Welfare, Health Services
behavioral problems and competencies. Interna- and Mental Health Administration Publication
tional Journal of Mental Health, in press. No. 73-9002). Washington, D.C.: U.S. Govern-
Achenbach, T. M., & Edelbrock, C. S. The Child ment Printing Office, 1973.
Behavior Profile: II. Boys aged 12-16 and girls Cowen, E. L., Dorr, D., Clarfield, S., Kreling, B.,
aged 6-11 and 12-16. Journal of Consulting and McWilliams, S. A., Pokracki, F., Pratt, D. M.,
Clinical Psychology, in press. Terrell, D., & Wilson, A. The AML: A quick-
Achenbach, T. M., & Lewis, M. A proposed model screening device for early identification of school
for clinical research and its application to en- maladaptation. American Journal of Community
copresis and enuresis. Journal of the American Psychology, 1973, 1, 12-35.
Academy of Child Psychiatry, 1971, 10, 535-5S4. Cowen, E. L., Lorion, R. P., & Wilson, A. B. Know-
Ackerson, L. Children's behavior problems: Vol. 2. ing, liking, and judged problem severity in rela-
Relative importance and interrelations among tion to referral and outcome measures in a school-
traits. Chicago: University of Chicago Press, 1942. based intervention program. Journal of Consult-
American Psychiatric Association. Diagnostic and ing and Clinical Psychology, 1976, 44, 317-329.
statistical manual, mental disorders (1st ed.). Dreger, R. M., Lewis, P. M., Rich, T. A., Miller,
Washington, D.C.: Author, 1952. K. S., Reid, M. P., Overlade, D. C., Taffel, C., &
American Psychiatric Association. Diagnostic and Flemming, E. L. Behavioral Classification Pro-
statistical manual of mental disorders (2nd ed.). ject. Journal of Consulting Psychology, 1964, 28,
Washington, D.C.: Author, 1968. 1-13.
Anderson, L. M. Personality characteristics of par- Eaves, R. C. Teacher race, student race, and the
ents of neurotic, aggressive, and normal preado- Behavior Problem Checklist. Journal of Abnormal
lescent boys. Journal of Consulting and Clinical Child Psychology, 1975, 3, 1-9.
Psychology, 1969, 33, 575-581. Eisenberg, J. G., Gersten, J. C., Langner, T. S., Mc-
Arnold, L. E., & Smeltzer, D. J. Behavior checklist Carthy, E. D., & Simcha-Fagan, 0. A behavioral
factor analysis for children and adolescents. classification of welfare children from survey data.
Archives of General Psychiatry, 1974, 30, 799- American Journal of Orthopsychiatry, 1976, 46,
804. 447-463.
Baker, R. P., & Dreger, R. M. The Preschool Be-
Evans, W. R. The Behavior Problem Checklist:
havioral Classification Project: An initial report.
Data from an inner city population. Psychology in
Journal of Abnormal Child Psychology, 1973, 1,
88-120. the Schools, 1975, 12, 301-303.
Bannister, D., Salmon, P., & Leiberman, D. M. Eysenck, H. J. The structure of human personality.
Diagnosis-treatment relationships in psychiatry: London: Methuen, 1953.
A statistical analysis. British Journal of Psychi- Fanshel, D., Hylton, L., & Borgatta, E. F. A study
atry, 1964, 110, 726-732. of behavior disorders of children in residential
 CHILD PSYCHOPATHOLOGY 1299

treatment centers. Journal of Psychological Stud- public-school children. American Journal of Men-
ies, 1963, 14, 1-23. tal Deficiency, 1976, 81, 135-146.
Ferguson, L. R., Partyka, L. B., & Lester, B. M. Langner, T, S., Gersten, J. C., & Eisenberg, J. G.
Patterns of parent perception differentiating Approaches to measurement and definition in the
clinic from nonclinic children. Journal of Abnor- epidemiology of behavior disorders: Ethnic back-
mal Child Psychology, 1974, 2, 169-181. ground and child behavior. International Journal
Freeman, M. A reliability study of psychiatric diag- of Health Services, 1974, 4, 483-501.
nosis in childhood and adolescence. Journal of Langner, T. S., Gersten, J. C., Eisenberg, J. G.,
Child Psychology and Psychiatry, 1971, 12, 43-S4. Greene, E. L., & Herson, J. H. Children under
Gersten, J. C., Langner, T. S., Eisenberg, J. G., stress: Family and social factors in the behavior
Simcha-Fagan, O., & McCarthy, E. D. Stability of urban children and adolescents. New York:
and change in types of behavioral disturbance of Columbia University Press, in press.
children and adolescents. Journal of Abnormal Lessing, E. E., Black, M., Barbera, L., & Seibert, F.
Child Psychology, 1976, 4, 111-127. Dimensions of adolescent psychopathology and
Hafner, A. J., Quast, W., & Shea, M. J. The adult their prognostic significance for treatment out-
adjustment of one thousand psychiatric and come. Genetic Psychology Monographs, 1976, 93,
pediatric patients: Initial findings from a twenty- 155-168.
five year follow-up. In R. D. Wirt, G. Winokur, Lessing, E. E., & Zagorin, S. W. Dimensions of
& M. Roff (Eds.), Life history research in psycho- psychopathology in middle childhood as evaluated
pathology (Vol. 4). Minneapolis: University of by three symptom checklists. Educational and
Minnesota Press, 1975. Psychological Measurement, 1971, 31, 175-198.
Hewitt, L. E., & Jenkins, R. L. Fundamental pat- Lobitz, G. K., & Johnson, S. M. Normal versus
terns of maladjustment: The dynamics of their
deviant children: A multimethod comparison.
origin. A statistical analysis based upon five hun- Journal of Abnormal Child Psychology, 1975, 3,
dred case records of children examined at the
Michigan Child Guidance Institute. Springfield: 353-374.
State of Illinois, 1946. Lorion, R. P., & Cowen, E. L. Comparison of two
Hobbs, N. The futures of children. San Francisco: outcome groups in a school-based mental health
Jossey-Bass, 1975. (a) project. American Journal of Community Psy-
Hobbs, N. Issues in the classification of children. chology, 1976, 4, 65-73.
San Francisco: Jossey-Bass, 1975. (b) Lorion, R. P., Cowen, E. L., & Caldwell, R. A.
Jenkins, R. L. Psychiatric syndromes in children and Problem types of children referred to a school-
their relation to family background. American based mental health program. Journal of Con-
Journal of Orthopsychiatry, 1966, 36, 450-457. sulting and Clinical Psychology, 1974, 42, 491-
Jenkins, R. L., & Glickman, S. Common syndromes 496.
in child psychiatry: I. Deviant behavior traits. II. Lorion, R. P., Cowen, E. L., & Caldwell, R. A.
The schizoid child. American Journal of Ortho- Normative and parametric analyses of school
psychiatry, 1946, 16, 244-261. maladjustment. American Journal of Community
Katz, P. A., Zigler, E., & Zalk, S. R. Children's self- Psychology, 1975, 3, 291-301.
image disparity: The effects of age, maladjust- Lorr, M., & Jenkins, R. L. Patterns of maladjust-
ment, and action-thought orientation. Develop- ment in children. Journal of Clinical Psychology,
mental Psychology, 1975, 11, 546-550. 1953, 9, 16-19.
Kobayashi, S., Mizushima, K., & Shinohara, M. McCarthy, J. M., & Paraskevopoulos, J. Behavior
Clinical groupings of problem children based on patterns of learning disabled, emotionally dis-
symptoms and behavior. International Journal of turbed, and average children. Exceptional Chil-
Social Psychiatry, 1967, 13, 206-215. dren, 1969, 36, 69-74.
Kohn, M., & Rosman, B. L. A social competence McCoy, S. A. Clinical judgments of normal child-
scale and symptom checklist for the preschool hood behavior. Journal of Consulting and Clinical
child: Factor dimensions, their cross-instrument Psychology, 1976, S, 710-714.
generality, and longitudinal persistence. Develop- Miller, L. C. Q-sort agreement among observers of
mental Psychology, 1972, 6, 430-444. children. American Journal of Orthopsychiatry,
Kohn, M., & Rosman, B. L. Cross-situational and 1964, 34, 71-75.
longitudinal stability of social-emotional func- Miller, L. C. Louisville Behavior Check List for
tioning in young children. Child Development, males, 6-12 years of age. Psychological Reports,
1973, 44, 721-727. (a) 1967, 21, 885-896.
Kohn, M., & Rosman, B. L. A two-factor model of Miller, L. C. School Behavior Checklist: An inven-
emotional disturbance in the young child: Valid- tory of deviant behavior for elementary school
ity and screening efficiency. Journal of Child children. Journal of Consulting and Clinical Psy-
Psychology and Psychiatry, 1973, 14, 31-56. (b) chology, 1972, 38, 134-144.
Kraepelin, E. Compendium der Psychiatric. Leipzig: Miller, L. C., Hampe, E., Barrett, C. L., & Noble, H.
Abel, 1883. Test-retest reliability of parent ratings of chil-
Lambert, N. M., & Nicoll, R. C. Dimensions of dren's deviant behavior. Psychological Reports,
adaptive behavior of retarded and nonretarded 1972, 31, 249-250.
1300 THOMAS M. ACHENBACH AND CRAIG S. EDELBROCK

Millman, H. L., & Davis, J. K. Assessing behavioral Rolf, J. E., & Garmezy, N. The school performance
change: A reliability study of the Devereux Child of children vulnerable to behavior pathology. In
Behavior Rating Scale. Devereux Forum, 1975, D. F. Ricks, A. Thomas, & M. Roff (Eds.), Life
10, 6-12. history research in psychopathology (Vol. 3).
Novick, J., Rosenfeld, E., Bloch, D. A., & Dawson, Minneapolis: University of Minnesota Press, 1974.
D. Ascertaining deviant behaviors in children. Rosen, B. M., Bahn, A. K., & Kramer, M. Demo-
Journal of Consulting Psychology, 1966, 30, 230- graphic and diagnostic characteristics of psychi-
238. atric clinic outpatients in the U.S.A., 1961.
O'Donnell, J. P., & Cress, J. N. Dimensions of be- American Journal of Orthopsychiatry, 1964, 34,
havior problems among Oglala Sioux adolescents. 455-468.
Journal of Abnormal Child Psychology, 1975, 3, Ross, A. O., Lacey, H. M., & Parton, D. A. The
163-169. development of a behavior checklist for boys.
Paraskevopoulos, J., & McCarthy, J. M. Behavior Child Development, 1965, 36, 10U-1027.
patterns of children with special learning disabili- Rutter, M. A children's behaviour questionnaire for
ties. Psychology in the Schools, 1970, 7, 42-46. completion by teachers: Preliminary findings.
Patterson, G. R. An empirical approach to the classi- Journal of Child Psychology and Psychiatry,
fication of disturbed children. Journal of Clinical 1967, 8, 1-11.
Psychology, 1964, 20, 326-337. Rutter, M., Shaffer, D., & Shepherd, M. A multi-
Peterson, D. R. Behavior problems of middle child- axial classification of child psychiatric disorders:
hood. Journal of Consulting Psychology, 1961, An evaluation of a proposal. Geneva, Switzer-
25, 205-209. land: World Health Organization, 1975.
Peterson, D. R., Becker, W. C., Shoemaker, D. J., Sandifer, M. G., Pettus, C., & Quadc, D. A study of
Luria, Z., & Hellmer, L. A. Child behavior prob- psychiatric diagnosis. Journal of Nervous and
lems and parental attitudes. Child Development, Mental Disease, 1964, 139, 350-356.
1961, 32, 151-162. Schaefer, C. E., & Millman, H. L, A factor ana-
Prior, M., Boulton, D., Gajzago, C., & Perry, D. lytic and reliability study of the Devereux Child
The classification of childhood psychoses by nu- Behavior Rating Scales. Journal of Abnormal
merical taxonomy. Journal of Child Psychology Child Psychology, 1973, 1, 241-247. (a)
and Psychiatry, 1975, 16, 321-330. Schaefer, C. E., & Millman, H. L. The use of be-
Proger, B. B., Mann, L., Green, P. A., Bayuk, R. havior ratings in assessing the effect of residential
J., & Burger, R. M. Discriminators of clinically treatment with latency age boys. Child Psychi-
defined emotional maladjustment: Predictive va- atry and Human Development, 1973, 3, 157-164.
lidity of the Behavior Problem Checklist and (b)
Devereux Scales. Journal of Abnormal Child Psy- Schultz, E. W., Salvia, J. A., & Feinn, J, Deviant
chology, 1975, 3, 71-82. behaviors in rural elementary school children.
Quay, H. C. Personality dimensions in delinquent Journal of Abnormal Child Psychology, 1973, 1,
males as inferred from the factor analysis of 378-389.
behavior ratings. Journal of Research in Crime Schultz, E. W., Salvia, J. A., & Feinn, J. Preva-
and Delinquency, 1964, 1, 33-37. lence of behavioral symptoms in rural elementary
Quay, H. C. Personality patterns in pre-adolescent school children. Journal of Abnormal Child Psy-
delinquent boys. Educational and Psychological chology, 1974, 2, 17-24.
Measurement, 1966, 26, 99-110. Shechtman, A. Age patterns in children's psychi-
Quay, H. C., Morse, W. C., & Cutler, R. L. Person- atric symptoms. Child Development, 1970, 41,
ality patterns of pupils in special classes for the 683-693.
emotionally disturbed. Exceptional Children, Shechtman, A. Symptoms observed in normal and
1966, 32, 297-301. disturbed black children. Journal of Clinical Psy-
Quay, H. C., & Quay, L. C. Behavior problems in chology, 1971, 27, 445-447.
early adolescence. Child Development, 1965, 36, Spitzer, R. L., & Fleiss, J. L. A re-analysis of the
215-220. reliability of psychiatric diagnosis. British Jour-
Quay, H. C., Sprague, R. L., Shulman, H. S., & nal of Psychiatry, 1974, 125, 341-347.
Miller, A. L. Some correlates of personality dis- Spivack, G., & Levine, M. The Devereux Child
order and conduct disorder in a child guidance Behavior Rating Scales: A study of symptom
clinic sample. Psychology in the Schools, 1966, 3, behaviors in latency age atypical children. Ameri-
44^7. can Journal of Mental Deficiency, 1964, 68, 700-
Roff, J. D., Knight, R., & Wertheim, E. Disturbed 717.
preschizophrenics: Childhood symptoms in rela- Spivack, G., & Spotts, J. The Devereux Child Be-
tion to adult outcome. Journal of Nervous and havior Scale: Symptom behaviors in latency age
Mental Disease, 1976, 162, 274-281. children. American Journal of Mental Deficiency,
Rolf, J. E. The social and academic competence of 1965, 69, 839-853.
children vulnerable to schizophrenia and other Spivack, G., & Spotts, J. Adolescent symptomatol-
behavior pathologies. Journal of Abnormal Psy- ogy. American Journal of Mental Deficiency,
chology, 1972, 80, 225-243. 1967, 72, 74-95.
 CHILD PSYCHOPATHOLOGY 1301

Spivack, G., & Swift, M. S. The Devereux Ele- list. Journal of Abnormal Child Psychology, 1975,
mentary School Behavior Rating Scale: A study 3, 115-126.
of the nature and organization o>f achievement Weintraub, S. A. Self-control as a correlate of an
related disturbed classroom behavior. Journal of internalizing-externalizing symptom dimension.
Special Education, 1966, 1, 71-90. Journal of Abnormal Child Psychology, 1973, 1,
Spivack, G., Swift, M., & Prewitt, J. Syndromes of 292-307.
disturbed classroom behavior: A behavioral diag- Wolff, S. Dimensions and clusters of symptoms in
nostic system for elementary school. Journal of disturbed children. British Journal of Psychi-
Special Education, 1971, S, 269-292. atry, 1971, 118, 421-427.
Tarter, R. E., Templer, D. I., & Hardy, C. Reli- Wysocki, B. A., & Wysocki, A. C. Behavior symp-
ability of the psychiatric diagnosis. Diseases of toms as a basis for a new diagnostic classification
the Nervous System, 1975, 36, 30-31. of problem children. Journal of Clinical Psychol-
Touliatos, J., & Lindholm, B. W. Relationships of ogy, 1970, 26, 41-45.
children's grade in school, sex, and social class to
teachers' ratings on the Behavior Problem Check- Received August 1, 1977 •