Conversation: Language & Vision

By
On May 14, 2011 at 9:46 p.m., I posted the first draft of what will eventually become the third story of the “Making and Empathy” chapter in the book Realizing Empathy: An Inquiry Into the Meaning of Making surrounding my experience with poster design. This is an edited version of the conversation that followed, regrouped and rearranged for clarity and relevance.

 

anson: I have always pondered whether it is possible for those born blind, deaf, and mute, to think or dream of abstract concepts that they have never encountered.

Whenever I have to process complex thoughts, I hear a voice inside my head, speaking a language with grammar that helps me understand and sort things out. How about babies? Having yet to acquire a language, how do they think properly? Do they just act on their instincts and feelings? What about grown-ups who do not have the ability to put thoughts together into sentences with proper grammar?

Some say that language is the key to our ability to process abstract thought and hence develop intelligence. I think there are many who are mentally and physically disabled, but can still think and understand things like other people. Language seems to be able to boost our ability to organize thoughts and abstract ideas, but it seems like we, humans, have a much more basic way of perceiving, feeling, and understanding the world around us, a fundamental layer of communication beneath our language that everyone has the innate ability to access. I am obviously speaking of what I do not understand, but maybe someone who does can shed light on these issues.

slim: I don’t know, either. But it occurs to me that there may be a set of perceptual triggers that encapsulate the fundamental and primitive qualities of perception, probably pre-language with the potential to be widely shared. Why couldn’t we imagine an interaction paradigm based exclusively on those triggers? After that is established, one could layer the symbolic and gestural semantics on top of it as needed.

joonkoo: These questions are very much related to the origin of knowledge, and the nature vs. nurture debate. I’m a blank slate when it comes to language, but I can point you to a few studies in the domain of vision and number processing. Just be aware that I may be over-generalizing.

The human visual cortex29   is organized in a category-selective manner. For example, the lateral part of the occipital cortex is activated when a person is viewing living things in general. On the other hand, the medial part of the cortex is activated when viewing non-living things. This category-specific organization can be driven by experience over development but it can also be somewhat hard-wired. One study looked at the patterns of neural activity in congenitally30 blind subjects, and they showed the same kind of neural activation patterns in response to these categories of objects even when they were presented auditorily. This study suggests that our visual experience is not necessarily the only critical factor that gives rise to the functional organization of our brain — at least in that context.

slim: When you say living vs. non-living, is a plant living or non-living? Is this related to how autistic people behave differently in relation to non-living vs. living things?

joonkoo: I don’t recall exactly how they categorized living vs. non-living in their study, but one thing I do think is true is that living vs. non-living is probably just one of many ways that things in the nature can naturally divide into, probably confounded with many other ways of categorizing things. For example, it may well be natural vs. man-made things that the brain really cares about. To me, the precise categorization of these things aren’t really important. What’s more interesting is that the visual cortex does not necessarily require visual input for its functional organization.

slim: If the visual cortex doesn’t require visual input for its function, it sounds like that would be a rather remarkable statement when it comes to our categorization of cortices into visual vs. others, no? Am I understanding this correctly?

joonkoo: Not exactly. Here’s another way to think about it. In normal development, the visual cortex is designed to process visual sensory information — based on the anatomical fact. But it’s used differently when it lacks visual input for any unexpected reason. What’s interesting is that even if the visual cortex is putatively31 doing something different in these congenitally blind people, there seems to be a set of universal principles that govern the functional organization of the visual cortex.

When these participants hear a living thing, for example, they have to bring up some mental image of that thing, which is probably not visual imagery, yet their visual cortex works the same way as it does on a participant.

slim: Oh, whoa .

So what you’re saying is that when blind people hear something, it triggers a mental image in their head, which uses the visual cortex, although the imagery they bring up is not visual?

joonkoo: Yes, my guess is that it’s probably a mixture of auditory and other multimodal imagery. But yes, their visual cortex works similarly to that of other subjects considered to be normal.

I guess this can be said as a form of plasticity. But I think this is much more profound than plasticity within a domain or modality (e.g., after losing a finger, the motor cortex that has been associated with that finger is now used for other fingers).

slim: When you say plasticity, I’m guessing it is a situation where a certain part of your body takes on a different role when what it was originally associated with is no longer available?

joonkoo: Yes. Evidence for brain plasticity is very cool.

To Anson’s point, however, this isn’t to say that the experience of abstract or symbolic thought is unimportant. Perhaps a more relevant story comes from a study that investigates number sense in native Amazonians,32 who lack the words for numbers. Through the use of numeric symbols, we have little problem expressing arbitrary quantity. On the other hand, Amazonians have only one, two, and many. Given this, they are pretty good at approximate arithmetic, even with numbers far beyond their naming range, but their performance on exact arithmetic tasks was poor. In fact, they failed to understand that n + 1 is an immediate successor of n.

anson: Would a relevant topic be why the Golden Ratio33 is universally pleasing to the eyes? It seems to indicate that there’s something common to human perception.

joonkoo: Yes, the Golden Ratio is interesting! In fact, there seem to be a lot of links between the biological system and math. One thing that I am more familiar with is the Power Law34 and γ, the Euler constant.35

Many of the psychophysical models are based on this constant and the natural log, and I would love to understand this more as well.

The definition of γ seems to be quite similar to neuronal firing patterns (e.g., long-term potentiation), and I speculate that all these fancy mathematics such as  γ, π, the Golden Ratio, may be driven by some of our intrinsic biological properties. I’m talking too much about things that I don’t fully understand. This should be a question for a computational biologist.

———-

29 The back area of the brain concerned with vision makes up the entire occipital lobe and the posterior parts of the temporal and parietal lobes. The visual cortex, also called the striate cortex, is on the medial side of the occipital lobe and is surrounded by the secondary visual area. This area is sensitive to the position and orientation of edges, the direction and speed of movement of objects in the visual field, and stereoscopic depth, brightness, and color; these aspects combine to produce visual perception. It is at this level that the impulses from the separate eyes meet at common cortical neurons, or nerve cells, so that when the discharges in single cortical neurons are recorded, it is usual to find that they respond to light falling in one or the other eye. It is probable that when the retinal messages have reached this level of the central nervous system, and not before, the human subject becomes aware of the visual stimulus, since destruction of the area causes absolute blindness in man. (Encyclopædia Britannica Online: 1 2)

30 Existing or dating from one’s birth, belonging to one from birth, born with one. (OED Online)

31 That is commonly believed to be such; reputed, supposed; imagined; postulated, hypothetical. (OED Online)

32 CNRS and INSERM researchers (Pierre Pica, Cathy Lemer, Véronique Izard and Stanislas Dehaene) studied the example of the Mundurucus Indians from Brazilian Amazonia, whose vocabulary includes number words only up to four or five. Tests performed over several months among this population show that the Mundurucus cannot readily perform “simple” mathematical operations with exact quantities, but their ability to use approximate numbers is comparable to our own.

This research, published in the October 15, 2004, issue of the journal Science, suggests that the human species’ capacity for approximate arithmetic is independent of language, whereas precise computation seems to be part of the technological inventions that vary largely from one population to the next. (“Cognition and Arithmetic Capability”)

33 Also known as the golden section, golden mean, or divine proportion, in mathematics, the irrational number (1 + √5)/2, often denoted by the Greek letters τ or ϕ, and approximately equal to 1.618. (Encyclopædia Britannica Online)

34 Van Mersbergen, Audrey M., “Rhetorical Prototypes in Architecture: Measuring the Acropolis with a Philosophical Polemic”, Communication Quarterly,
Vol. 46 No. 2, 1998, pp 194–213. A relationship between two quantities such that the magnitude of one is proportional to a fixed power of the magnitude of the other. (OED Online)

35 The constant that is the limit of the sum 1 + ½ + … + 1/ n − log n as n tends to infinity, approximately equal to 0.577215665 (it is not yet known whether this number is rational or irrational). (OED Online)