KNOWLEDGE: REPRESENTATION AND MANIPULATION

CHAPTER OUTLINE
Mental Representation of Knowledge
Mental Manipulation of Images
Synthesizing Images and Propositions
Spatial Cognition and Cognitive Maps

KNOWLEDGE REPRESENTATION
• The form for what you know in your mind
about things, ideas, events, and so on, in
the outside world.

• Introspectionist Approach
• We can ask people to describe their own knowledge
representations and knowledge-representation
processes: What do they see in their minds when they
think of the Statue of Liberty, for example?
• Unfortunately, none of us has conscious access to our
own knowledge-representation processes and self-
reported information about these processes is highly
unreliable (Pinker, 1985).

• Rationalist Approach
• We try to deduce logically how people represent
knowledge.
• In classic epistemology—the study of the nature, origins,
and limits of human knowledge—philosophers
distinguished between two kinds of knowledge
structures.
1. Declarative knowledge – refers to facts that can be
stated, such as the date of your birth, the name of
your bestfriend, or the way a rabbit looks.

2. Procedural knowledge – refers to knowledge of
procedures that can be implemented. Examples are the
steps involved in tying your shoelaces, adding a column
of numbers, or driving a car.
The distinction is between knowing that and knowing
how (Ryle, 1949).

• Two main sources of empirical data on knowledge
representation:
• Standard laboratory experiment
• Neuropsychological studies

• In experimental work, researchers indirectly study
knowledge representation because they cannot look into
people’s minds directly.
• They observe how people handle various cognitive
tasks that require the manipulation of mentally
represented knowledge.

• In neuropsychological studies, researchers typically
use one of two methods:
• They observe how the normal brain responds to
various cognitive tasks involving knowledge
representation.
• They observe the links between various deficits in
knowledge representation and associated pathologies
in the brain.

• In the following sections, we explore some of the
theories researchers have proposed to explain how we
represent and store knowledge in our minds:
• First, we consider what the difference is between
images and words when they are used to represent
ideas in the outside world.
• Then we learn about mental images and the idea that
we store some of or knowledge in the form of images.

• In the following sections, we explore some of the
theories researchers have proposed to explain how we
represent and store knowledge in our minds:
• Next, we explore the idea that knowledge is stored in
the form of both words and images (dual-code theory).
• Finally, we consider an alternative—propositional
theory—which suggests that we actually use an
abstract form of knowledge encoding that makes use
of neither words nor mental images.

• Knowledge can be represented in different ways in your
mind: It can be stored as
• Mental picture
• Words
• Abstract propositions

COMMUNICATING KNOWLEDGE: PICTURES VS. WORDS
• External representation – ideas in the outside world,
such as a book.
• Some ideas are better and more easily represented in
pictures, whereas others are better represented in
words.
• Example: “What is the shape of the egg?”
“What is justice?”

The picture is
relatively analogous
(similar) to the real-
world object it
represents. The
picture shows
concrete attributes,
such as shape and
relative size.

• Symbolic representation – the relationship between the
word and what it represents is simply arbitrary (based on
random choice or personal whim, rather than any reason
or system.)
• Capture some kinds of information but not other kinds of
information.
• In forming words, the sounds or letters also must be
sequenced according to rules (e.g, “c-a-t,” not “a-c-t’ or “t-
a-c”).
• In forming sentences, the words also must be sequenced
according to rules.
• “The cat is under the table.”
• “Table under cat the is.”

• Pictures, aptly capture concrete and spatial information in
a manner analogous to whatever they represent. They
convey all features simultaneously.
• In general, any rules for creating or understanding
pictures pertain to the analogous relationship between the
picture and what it represents.
• Words, handily capture abstract and categorical
information in a manner that is symbolic of whatever the
words represent.
• Representations in words usually convey information
sequentially.

PICTURES IN YOUR MIND: MENTAL IMAGERY
• Internal representation
• How do we represent what we know
in our minds.
• Imagery – is the mental
representation of things that are not
currently seen or sensed by the
sense organs.

• In our minds we often have images for objects,
events, and settings.
• Mental imagery even can represent things that you
have never experienced.
• Mental images even may represent things that do
not exist at all outside the mind of the person
creating the image.

• Imagery may involve mental representations in
any of the sensory modalities, such as hearing,
smell, or taste.
• Imagine the sound of fire alarm, your favorite
song, or your nation’s anthem. Now, imagine the
smell of a rose, of fried bacon, or of an onion.
Finally, imagine the taste of a lemon, pickle, or
your favorite candy.

• Most research on imagery has focused on visual imagery,
such as representations of objects or settings that are not
presently visible to the eyes.
• Most of us are more aware of visual imagery than of other
forms of imagery.
• We use visual image to solve problems and to answer
questions involving objects.
• Which is darker red—a cherry or an apple?
• How many windows are there in your house?
• How do you get from your home to your first class of the
day?

• Using guided-imagery techniques for controlling pain and for
strengthening immune responses and otherwise promoting
health.
• Imagine being at a beautiful beach and feeling very
comfortable, letting your pain fade into the background.
• Imagine the cells of your immune system successfully
destroying all the bad bacteria in your body.

• Such techniques are also helpful in
overcoming psychological problems,
such as phobias and other anxiety
disorders.
• Design engineers, biochemists,
physicists, and many other scientists
and technologists use imagery to think
about various structures and processes
and to solve problems in their chosen
field.

• Research also indicates that the use of
mental images can help to improve
memory.
• Mental imagery also is used in other
fields such as occupational therapy.
Using these techniques, patients with
brain damage train themselves to
complete complex tasks.

• In what form do we
represent images in our
minds?
• All images of everything we
ever sense may be stored
as exact copies of physical
images.
• But realistically, to store
every observed physical
image in the brain seems
impossible.
• Amazingly, learning can
indeed take place just by
using mental images.

DUAL-CODE THEORY: IMAGES AND SYMBOLS
• Dual-code theory – we use both pictorial and
verbal codes for representing information in our
minds.
• According to Paivio, mental images are analog
codes.
• Analog codes – resemble the objects they are
representing.
• Example: trees and rivers might be represented
by analog codes
movement of the hands on an analog
clock are analogous to the passage of time.
 The mental images we form in our minds
are analogous to the physical stimuli we
observe.

• In contrast, our mental representations for
words chiefly are represented in a symbolic
code.
• Symbolic code is a form of knowledge
representation that has been chosen
arbitrarily to stand for something that does
not perceptually resemble what is being
represented.

• A symbol may be anything that is arbitrarily
designated to stand for something other than
itself.
• We recognize that the numeral “9” is a
symbol for the concept of “nineness.” It
represents a quantity of nine of
something.
• Concepts like justice and peace are best
represented symbolically.

• Paivio, consistent with his dual-code theory, noted that
verbal information seems to be processed differently than
pictorial information.
• Participants more easily recalled the pictures when
they were allowed to do so in any order. But they more
readily recalled the sequence in which the words were
presented than the sequence for the pictures, which
suggests the possibility of two different systems for
recall of words versus pictures.

• Analogical and Symbolic Representation of Cats
• To get an intuitive sense of how you may use each of the two kinds of
representations, think about how you mentally represent all the facts
you know about cats. Use your mental definition of the word cat and all
the inferences you may draw from your mental image of a cat. Which
kind of representations is more helpful for answering the following
questions:
• Is a cat’s tail long enough to reach the tip of the cat’s nose if the cat is
stretching to full length?
• Do cats like to eat fish?
• Are the back legs and the front legs of a cat exactly the same size and
shape?
• Are cats mammals?
• Which is wider—a cat’s nose or a cat’s eye?
• Which kind of mental representations were most valuable for
answering each of these qustions?

• Dual Coding
• Look at the list of words your friends and family members
recalled in the demonstration in Chapter 6. Add up the total
number of recollections for every other word (i.e., book,
window, box, hat, etc.—the words in odd-numbered
positions in the list). Now add up the total number of
recollections for the other words (i.e., peace, run, harmony,
voice, etc.—the words in even-numbered positions on the
list).

• Dual Coding
• Most people will recall more words from the first set than
from the second set. This is because the first set is made
up of words that are concrete, or those words that are
easily visualized. The second set of words is made up of
words that are abstract, or not easily visualized.
• This is a demonstration of the dual-coding hypothesis (or
its more contemporary version, the functional-equivalence
hypothesis).

STORING KNOWLEDGE AS ABSTRACT CONCEPTS:
PROPOSITIONAL THEORY
• Propositional theory suggests that we do not store mental
representations in the form of images or mere words.
• We may experience our mental representations as images,
but these images are epiphenomena—secondary and
derivative phenomena that occur as a result of other more
basic cognitive processes.
• According to propositional theory, our mental representations
(sometimes called “mentalese”) more closely resemble the
abstract form of a proposition.

STORING KNOWLEDGE AS ABSTRACT CONCEPTS:
PROPOSITIONAL THEORY
• A proposition is the
meaning underlying
a particular
relationship among
concepts.

WHAT IS A PROPOSITION?
• UNDER ( CAT, TABLE) - logical expression
• [Relationship between elements] ([Subject element], [Object
element])
• Verbal representation:
• “The table is above the cat”
• “The cat is beneath the table.”
• “Above the cat is the table.”
• Logicians have devised a shorthand means, called “predicate
calculus,” of expressing the underlying meaning of a
relationship. It attempts to strip away the various superficial
differences in the ways we describe the deeper meaning of a
proposition.

USING PROPOSITIONS
• We may use propositions to represent any kind of
relationship, including actions, attributes, spatial positions,
class membership, or almost any other conceivable
relationship. The possibility for combining propositions
into complex propositional representational relationships
makes the use of such representations highly flexible and
widely applicable.

PROPOSITIONAL REPRESENTATIONS OF
UNDERLYING MEANINGS
• Type of Relationship
• actions
• Representation in Words
• A mouse bit a cat.
• Propositional Representation
• Bite [action](mouse[agent of
action], cat [object]
• Imaginal Representation

UNDERLYING MEANINGS
• attributes
• Mice are furry.
• [external surface characteristic]
(furry [attribute], mouse [object]

UNDERLYING MEANINGS
• Spatial positions
• A cat is under the table.
• [vertically higher position] (table,
cat)

UNDERLYING MEANINGS
• Class or category membership
• A cat is an animal.
• [categorical membership] (animal
[category], cat [member])

LIMITATIONS OF MENTAL IMAGE
• Mental Images
• Quickly glance at this figure and
then cover it with your hand.
Imagine the figure you just saw.
Does it contain a
parallelogram?

CAN MENTAL IMAGES BE AMBIGUOUS?
(a) Look closely at the rabbit, then cover it with your hand and recreate it
in your mind. Can you see a different animal in this image just by
mentally shifting your perspective?
(b) What animal do you observe in this figure? Create a mental image of
this figure, and try to imagine the front end of this animal as the back
end of another animal and the tail end of this animal as the front end
of another animal.
(c) Observe the animal in this figure, and create a mental image of the
animal; cover the figure, and try to reinterpret your mental image as
a different kind of animal (both animals probably are facing in the
same direction).

AMBIGUOUS FIGURE
• Can be interpreted in more than one way.
• Often used in studies of perception.
• Without looking back at the figure, can you
determine the alternative interpretation of Figure 7.6
(a)?

AMBIGUOUS FIGURE
The alternative interpretation of the rabbit is a
duck. In this interpretation, the rabbit’s ears are the
duck’s bill.

LIMITATIONS OF PROPOSITIONAL THEORY
• The mental reinterpretation of ambiguous figures involves two
manipulations:
1. Mental realignment of the reference frame. This
realignment would involve a shift in the positional
orientations of the figures on the mental page or screen
on which the image is displayed.
2. Mental reconstrual (reinterpretation) of parts of the
figure.

• Across experiments, 20% to 83% of participants were able to
reinterpret ambiguous figures, using one or more of the
following hints:
1. Implicit reference-frame hint. Participants were shown
another ambiguous figure involving realignment of the
reference frame; ex. A hawk’s head/a goose’s tail, and a
hawk’s tail/a goose’s head.
2. Explicit reference-frame hint. Participants were asked
to modify the reference frame by considering either “the
back of the head of the animal they had already seen as
the front of the head of some other animal” (conceptual
hint); “the front of the thing you were seeing as the back
something else” (abstract hint).

• Across experiments, 20% to 83% of participants were able to
reinterpret ambiguous figures, using one or more of the
following hints:
3. Attentional hint. Participants were directed to attend to
regions of the figure were realignments or reconstruals were
to occur.
4. Construals from “good” parts. Participants were asked
to construe an image from parts determined to be “good”
(according to both objective [geometrical] and empirical
[inter-rater agreement] criteria), rather than parts to be “bad”
(according to similar criteria.)

• Additionally, some spontaneous reinterpretation of mental
images for ambiguous figure may occur. This is particularly
likely for images of figures that may be reinterpreted without
realigning the reference frame.
• Figure (c), which may be a whole snail or an elephant’s head,
or possibly even a bird, a helmet, a leaf, or a seashell.

THE INFLUENCE OF SEMANTIC LABELS
• Visual images can be distorted through verbal
information.
• Participants were asked to view figures that were labeled.
When they recalled the images, they were distorted in the
direction of the meaning of the images.
• Semantic (verbal) information (e.g., labels for figures)
tends to distort recall of visual images in the direction of
the meaning of the images.
• Recall differs based on the differing labels given for the
figures.

FUNCTIONAL-EQUIVALENCE HYPOTHESIS
• Although visual imagery is not identical to visual perception, it is
functionally equivalent to it.
• Functionally equivalent things are strongly analogous to each
other—they can accomplish the same goals.
• This view essentially suggests that we use images rather than
propositions in knowledge representations for concrete objects
that can be pictured in the mind.

PRINCIPLES OF VISUAL IMAGERY
• According to the functional-equivalence
hypothesis, we represent and use visual imagery in
a way that is functionally equivalent (strongly
analogous) to that for physical percepts.
• Ronald Finke has suggested several principles of
visual imagery that may be used to guide research
and theory development.

PRINCIPLES OF VISUAL IMAGERY
1. Our mental transformations of images and our
mental movements across images correspond to
those of physical objects and percepts.
2. The spatial relations among elements of a visual
image are analogous to those relations in actual
physical space.
3. Mental images can be used to generate
information that was not explicitly stored during
encoding.
4. The construction of mental images is analogous
to the construction of visually perceptible figures.
5. Visual imagery is functionally equivalent to visual
perception in terms of the processes of the
visual system used for each.

NEUROSCIENCE AND FUNCTIONAL EQUIVALENCE
• Schizophrenia provides an interesting example of the
similarities between perception and imagery.
• Auditory hallucinations are experiences of “hearing” that
occur in the absence of actual auditory stimuli.
• Evidence from other researchers reveals that during
auditory hallucinations there is abnormal activation of the
auditory cortex.
• In sum, it is believed that auditory hallucinations occur at
least in part because of malfunctions of the auditory
imaging system and problematic perception processes.
• These results suggest that there is indeed functional
equivalence between what our senses perceive and what
we create in our minds.

MENTAL ROTATION
• Involves rotationally transforming an object’s visual
mental image.

MENTAL ROTATION
• For which of these pairs of figures
does the figure on the right show
an accurate rotation of the figure on
the left?
• The forms were rotated from 0 to
180 degrees.
• Picture plane (2-D space
clockwise or counterclockwise)
• Depth (3-D space)
• Distracter form

MENTAL ROTATION • There was no significant
difference between rotations in
the picture plane and rotations in
depth.
• To rotate objects at larger angles
of rotation takes longer.
• Mental rotation may be an
automatic process.
• Such automatic processes may
be a sign of more effective
visuospatial skills because
increased speed is associated
with increased accuracy in spatial
memory.
• Aging affects some aspects of

INTELLIGENCE AND MENTAL ROTATION
• Primary Mental Abilities test of Louise and Thelma
Thurstone (1962) requires mental rotation of two-
dimensionally pictured objects in the picture plane.
• It has identified the mental representations and
cognitive processes that underlie adaptations to the
environment and thus, ultimately, that constitute
human intelligence.

NEUROSCIENCE AND MENTAL ROTATION
• Mental rotation also indicates that the
motor cortex (areas in the posterior
frontal cortex) is activated during this
task. The areas associated with hand
movement were particularly active
during the mental rotation task.
• Areas of the cerebral cortex have
representations that resemble the 2-D
spatial arrangements of visual
receptors in the retina of the eye.
These mappings may be construed as
relatively depictive of the visual arrays
in the real world.
• The same brain areas involved in
perception also are involved in mental
rotation tasks.

GENDER AND MENTAL ROTATION
• In young children, there is no gender differences
either in performance or in neurological activation.
• There seem differences in the activation of the
parietal regions between men and women. There is
less parietal activation for women than for men
completing the same mental rotation task.
• In women, spatial tasks involve both sides of the
brain, whereas in men, the right side dominates this
function.
• The differences in brain activation may mean that
men and women use different strategies to solve
mental rotation problems.
• Women have a proportionally greater amount of
gray matter in the parietal lobe than do men, which
is associated with a performance disadvantage for
mental rotation tasks for the women (as they need
increased effort to complete the tasks).

ZOOMING IN ON MENTAL IMAGES: IMAGE SCALING
• We use mental images the same way we
use our actual perceptions.
• When you look at a building from afar,
you won’t be able to see as many
details as when you are close by, and
you may not be able to see things as
clearly.
• In general, seeing details of large objects
is easier than seeing such details of
small ones. We respond more quickly to
questions about large objects we
observe than to questions about small
ones we observe.

IMAGE SCALING
• Find a large bookcase (floor to ceiling, if possible; if not,
observe the contents of a large refrigerator with an open
door). Stand as close to the bookcase as you can while still
keeping all of it in view. Now, read the smallest writing on
the smallest book in the bookcase. Without changing your
gaze, can you still see all of the bookcase? Can you read
the title of the book farthest from the book on which you are
focusing your perception? Depending on what you want to
see (a detail like a book title or the whole shelf), you may
have to zoom in and out of what you see. When you look at
a small detail, it will be hard to perceive the whole shelf,
and vice versa. The same is true for mental images.

IMAGE SCANNING
• Look at the rabbit and the fly in figure 7.10. Close your eyes and
picture them both in your mind. Now in your imagination, look only
at the fly and determine the exact shape of the fly’s head. Do you
notice yourself having to take time to zoom in to “see” the detailed
features of the fly? If you are like most people, you are able to
zoom in on your mental images to give the features or objects a
larger portion of your mental screen, much as you might physically
move toward an object you wanted to observe more closely.
• Now, look at the rabbit and the elephant and picture them both in
your mind. Next, close your eyes and look at the elephant.
Imagine walking toward the elephant, watching it as it gets closer
to you. Do you find that there comes a point when you can no
longer see the rabbit or even all of the elephant? If you are like
most people, you will find that the image of the elephant will
appear to overflow the size of your image space. To “see” the
whole elephant, you probably have to mentally zoom out again.

MENTAL SCANNING: AN
IMAGINARY ISLAND

REPRESENTATIONAL NEGLECT
• A person asked to imagine a scene and then
describe it ignores half of the imagined scene.

JOHNSON-LAIRD’S MENTAL MODELS
• Mental models are knowledge structures that
individuals construct to understand and explain their
experiences.

LEFT BRAIN OR RIGHT BRAIN: WHERE IS
INFORMATION MANIPULATED?
• The right hemisphere appears to
represent and manipulate
visuospatial knowledge in a manner
similar to perception.
• Also represents knowledge in a
manner that is analogous to our
physical environment.
• The left hemisphere appears to be
more proficient in representing and
manipulating verbal and other
symbol-based knowledge.
• Also has the ability to manipulate
imaginal components and symbols
and to generate entirely new
information.

TWO KINDS OF IMAGES: VISUAL VERSUS SPATIAL
• Visual imagery refers to the use of images that
represent visual characteristics such as colors and
shapes.
• Spatial imagery refers to images that represent
spatial features such as depth dimensions,
distances, and orientations.

SPATIAL COGNITION
• Deals with the acquisition, organization, and use of
knowledge about objects and actions in two- and
three-dimensional space.

COGNITIVE MAPS
• Are internal representations of our physical
environment, particularly centering on spatial
relationships.

COGNITIVE MAPS
• Rats
• Edward Tolman
• He argued for the importance of the mental
representations that give rise to behavior.
• The ability of rats to learn a maze.

COGNITIVE MAPS
• Bees
• Nobel Prize-winning German scientist
• Studied the behavior of bees when they return to
their hive after having located a source of nectar.

COGNITIVE MAPS
• Pigeons
• Noted for their excellent cognitive maps.
• These birds are known for their ability to return to
their home from distant locations.

COGNITIVE MAPS
• Hippocampus is involved in the formation
of cognitive maps in humans as well.
• Left hippocampus plays a pivotal role in
map formation.
• Also crucial for the perception of
landmarks within the environment.
• Right hippocampus is involved in
sensitivity to global features of the
environment.

COGNITIVE MAPS
• Humans seem to use three types of
knowledge when forming and using cognitive
maps:
1. Landmark knowledge is information
about particular features at a location and
which may be based on both imaginal and
propositional representations.
2. Route-road knowledge involves specific
pathways for moving from one location to
another. It may be based on both
procedural knowledge and declarative
knowledge.
3. Survey knowledge involves estimated
distances between landmarks, much as
they might appear on survey maps. It may
be represented imaginally or
propositionally.

RULES OF THUMB FOR USING MENTAL MAPS:
HEURISTICS
• When we use landmark,
route-road, and survey
knowledge, we sometimes
use rules of thumb that
influence our estimations of
distance. These rules of
thumb are cognitive strategies
termed heuristics.
• Allow people to solve
problems and make
judgments quickly and
efficiently, but they are also
prone to errors.

RULES OF THUMB FOR USING MENTAL MAPS:
HEURISTICS
• Right-angle bias: people tend to think of intersections (street crossings)
as forming 90-degree angles more often than the intersections really do.
• Symmetry heuristic: people tend to think of shapes (states or countries)
as being more symmetrical than they really are.
• Rotation heuristic: when representing figures and boundaries that are
slightly slanted (oblique), people tend to distort the images as being either
more vertical or more horizontal than they really are.
• Alignment heuristic: people tend to represent landmarks and boundaries
that are slightly out of alignment by distorting their mental images to be
better aligned than they really are.
• Relative-position heuristic: the relative positions of particular landmarks
and boundaries is distorted in mental images in ways that more accurately
reflect people’s conceptual knowledge about the contexts in which the
landmarks and boundaries are located, rather than reflecting the actual
spatial configuration.

• Sometimes, to get between places, we need to imagine the
route we will need to traverse.
• Mental imagery provides a key basis for this adaptation.
• Thus, our imagery abilities are potential keys to our survival
and to what makes us intelligent in our everyday lives.

1. What are some practical applications of having two
codes for knowledge representation? Give an
example applied to your own experiences, such as
applications to studying for examinations. (10 pts)
2. Create a cognitive map of your house from your
school campus using the three types of knowledge.
(10 pts)
a. Landmark
b. Route-road
c. survey

1. What is a concept?
2. What is category?
3. What is the difference between prototypes and
examplars?
4. What is the theory-based view of meaning?
5. What are the components of a semantic network?
6. What is a schema?
7. Why do we need scripts?
8. What is procedural knowledge?
9. What are the different kinds of nondeclarative
knowledge?
10. What are the two types of priming?
11. What is the ACT-R model?
12. How is procedural knowledge represented in the ACT-R
model?
13. What is parallel processing?
14. How does a connectionist network represent
knowledge?
15. What is domain specificity?

KNOWLEDGE: REPRESENTATION AND MANIPULATION

More Related Content

What's hot (20)

Viewers also liked (13)

Similar to KNOWLEDGE: REPRESENTATION AND MANIPULATION (20)

More from Maria Angela Leabres-Diopol (20)

KNOWLEDGE: REPRESENTATION AND MANIPULATION