Duke University Alumni Magazine

TRICKING THE EYE OR TRAPPING A REFLEX ? REVISITING VISION BY DENNIS MEREDITH Optical iconoclasts: Purves, right, and one of his colleagues, Beau Lotto, discovered a new way of looking at visual illusions Experiments with the enigma of optical illusions have led neurobiologist Dale Purves and his colleagues to a controversial new theory of how our brains perceive the visual world.

itting in a small, darkened room, Dale Purves stares at a computer screen that displays the image of two blocks stacked one on top of the other (Figure 4). It's obvious to him, as it is to every other human on Earth, that the surfaces of the two blocks are different shades of gray. As the experiment requires, he carefully adjusts the shading on two separate test patches on the screen until he is satisfied that they match the apparent shading difference between the two blocks.

But Purves, and everyone else, is dead wrong in their perception. Actually, the surfaces of the two blocks are identical shades. It's a nearly unbelievable fact made startlingly obvious by merely covering the figure's center section, including the line joining them and the light and dark gradients. Ironically, the perceptual "wrongs" in this experiment, and many others exploring such visual illusions, have produced a remarkable scientific "right" --a radical new theory of how humans perceive the visual world.

Figure 1: In the figure on the left, the eye sees two fictitious bands: one somewhat lighter than the rest of the surface, where the shading begins across the curved surface of the edge of the cube (blue arrow); and one darker, where the shading ends on the side of the cube (red arrow). The bands don't really exist in this computer-generated image. In this illusion, called Mach bands, Purves and colleagues argue that we see the bands based on millions of years of visual experience with real reflective surfaces. The photograph of an actual cube on the right shows such highlights (blue arrow) and lowlights (red arrow) that are typically part of the scenes, and therefore incorporated into our perception.

Purves and his colleagues have developed a "wholly empirical" theory of vision, which contends that the gelatinous neural machinery by which your brain processes visual information is operating very differently than you --and the vast majority of brain researchers --believe. Their theory challenges the cherished belief that our visual brains are logically organized, analytical machines that provide a neural blueprint for interpreting images. He and his collaborators--Beau Lotto, Shuro Nundy, Amita Shimpi, Mark Williams, and Zhiyong Yang--have honed their concept over five years of meticulous experiments. The result of their exacting work: a theory that vision is basically a reflex no different than the knee-jerk response produced when the doctor taps your knee with a rubber hammer. Certainly, these "vision reflexes" are more complex, says Purves, but like the knee-jerk, your visual sensations are encoded in already established visual circuitry that is simply waiting to be triggered.

Humans evolved this counterintuitive way of generating visual perceptions for much the same reason as any other reflex, holds the theory. The knee-jerk circuitry, for example, evolved because it keeps us from falling down when we unexpectedly stumble on something. Similarly, vision reflexes evolved because there is simply no other way, except by a massive collection of such reflexes, to understand the world that gives rise to visual stimuli.

"The basic problem in vision is that the meaning of a light stimulus is inevitably ambiguous," says Purves. "The light that reaches the eye is always a product of both the quality of the object's surface and the quality of its illumination. So, it is impossible to know whether an object looks the shade or color that it does because of how it reflects light, or because of the nature of the light initially falling on it--or some combination of the two."

There is only one way animals, including humans, can sort out the meaning of such ambiguous stimuli, say Purves and his colleagues. And that is gradually to develop circuitry that enables them to respond, not according to properties of the light falling on the retina as such, but according to the sources in the physical world that have generated that type of stimulus in the past. Thus, our vision developed under the unrelenting pressure of survival-of-the-fittest. Our knuckle-walking ancestors whose brain wiring made it hard for them to distinguish shadows from holes in the ground--both of which can be generated by the same light patterns--failed to survive. On the other hand, those whose visual perceptions generated behaviors that happened to work lived to reproduce, passing on their visual circuitry to offspring for further evolutionary refinement.

Says Purves, "The visual part of the brain -- and presumably the rest of the brain as well --works like a computer that doesn't really 'understand' the rules of checkers. Nevertheless, it can develop a good game by simply changing its play according to the moves that worked well in a given situation in the past. We've had millions of years as a species, and a fair number of years as individuals, to perfect the neural networks triggered by visual stimuli. This very large amount of experience is what has made us so good at visualizing the world we live in, even though we never really see what's there."

Figure 2: The four angles presented here, believe it or not, are each exactly 90 degrees in the plane of the paper. They look very different because, in this computer rendering, the three-dimensional sources of the identical angles would have turned out to be objects that are, indeed, different. The visual system sees the same physical stimulus differently because of what the stimuli would typically have been.

The careful, soft-spoken scientist allows that his theory may ultimately prove as illusory as the visual illusions he studies. Despite his experience and many honors--as chairman of Duke's department of neurobiology and holder of the George B. Geller Professorship in Neurobiology--Purves is the first to admit that many believe that his idea about how vision works is completely wrong. Nonetheless, he is determined to see the theory receive a fair hearing. And, he is just as determined that, if proven right, the theory should yield its full measure of benefit to humankind. Those benefits could be enormous, ranging from theoretical models for powerful new generations of brain-like computers to a more rational and productive understanding of the how the brain generally processes information.

"If we're wrong, okay, that's the way science works--by developing theories and testing them," he says. "But I don't think we are. Every visual puzzle that we have tried as a challenge to the theory--and some have been around for centuries--has turned out to be explainable in these terms, giving us more and more confidence. If we're right, it means that we and others will have to rethink the direction and ultimately the value of much of the current research that consists basically of figuring out what's connected to what in the visual system. The expectation of most vision scientists is that when we understand enough about the connectivity of the system, the basis of perception will just sort of pop out as a consequence. Given the nature of what we actually see, that is wishful thinking."

The first leap of thinking that scientists must make in accepting this theory of vision, says Purves, is to discard some long-held assumptions about the light that illuminates the world around us and cascades onto our retinas during every waking moment. "Many current explanations of vision are basically putting a Band-Aid on concepts that need major surgery," he asserts. "Everyone knows that visual stimuli are in some sense ambiguous, but not many want to follow this ambiguity to its logical conclusion. Maybe this reluctance is because things certainly don't seem ambiguous as you look around. You certainly recognize yourself when you look in the mirror; books on a shelf look like books, trees look like trees, and so on. But if you think more deeply about light, you realize that without several million years of species experience and a few decades of personal experience, visual stimuli would be totally meaningless. A photon carries no information about its history, and there's no way direct way to disentangle what has actually given rise to the light falling on the retina. As a result, the visual world is infinitely ambiguous." Thus, he concludes, visual experience is not merely the best teacher; it's the only teacher.

And experiment is the only way to prove this theory. So, Purves and his colleagues have supported their ideas with results from a multitude of clever experiments. They, as well as paid subjects, have spent long hours in darkened rooms staring at a variety of visual stimuli, reporting precisely what they see. These experiments, sponsored by the National Institutes of Health, have shown that despite Abraham Lincoln's assertion, it is quite possible to "fool all of the people all of the time." The experiments reveal that what we see represents not the qualities of objects in the world or the qualities of the light that reaches the eye, but quite literally what that stimulus has typically turned out to be in the past.

One such stimulus is the image of blocks discussed at the beginning of this story. The experiments using the blocks are part of a series aimed at showing that perceived brightness is not based on the amount of light that actually reaches the eye from surfaces in a scene. Instead, the perception is but a consequence of a particular light pattern in the stimulus triggering a reflex response that corresponds to what the scene's elements have usually turned out to be in the past. Conventional vision theory attempts to explain the misperception of the blocks as differently shaded by attributing it to a sort of neurological "cross-talk" between light-sensors in the retina, or elsewhere in the visual system, as the observer views the scene. Specifically, this standard theory holds that signals from eye sensors fixed on the dark band "leak" signals to adjacent sensors, influencing those sensors to tell the brain that the adjacent surface is darker. The textbook explanation is thus that the block illusion is simply a side-effect of retinal connections.

Poppycock, says Purves. He and his colleagues argue that the visual system uses its millions of years of inherited experience to trigger a neural activity pattern that arises because the perception of a darker band in the image's center section would usually mean the top block is better lit. Similarly, the lighter band would usually mean the bottom block is less well lit. Because the apparent joint between the two surfaces has usually turned out to signify two differently reflective surfaces in different amounts of light, this is what we see. Thus, reality doesn't matter to the visual system. It insists that the blocks "must" be two different shades, even though they are not.

Figure 3: In this illusion, the light from the "brown" Chiclet-like object in the middle of the upper face of the cube is identical to the "orange" Chiclet in the middle of the shaded face. For comparison, see the outline version of the scene in which the other colored Chiclets and the cube's shadowing have been omitted. To demonstrate what your eyes will not believe, try cutting out the outline version, snipping out the brown squares, and superimposing the outline over the multicolored square. The Duke neurobiologists have argued that the colors we see are also generated empirically according to what the spectra reaching the eye have typically signified.

Purves and his colleagues have also experimented with color perception to glean support for their theory--in some cases using near-unbelievable illusions. For example, using the colorful block (Figure 3), Purves and Beau Lotto have shown dramatically that color perception is also based on experience. The block shows how changing the empirical meaning of a scene not only changes the brightness people perceive in the scene, but also the colors. Although it is nearly impossible for people to believe, the "orange" square in the middle of the cube's shadowed face is identical to the "brown" square on the upper surface. The squares are shown without the confounding shadows in the accompanying key diagram. (If you don't believe that the colors are identical, try cutting out the diagram, removing the brown squares, and laying the key diagram over the image.)

Such "unrealistic" color perceptions are difficult to account for in terms of conventional color theory, and have long been a source of controversy. Purves and Lotto argue that the different colors seen in this circumstance arise from the same empirical strategy of vision that explains the perception of the differently bright surfaces in the shaded block scene.

In the multicolored block, the colors reflected from the two surfaces would in many circumstances look identical. However, when the surfaces are part of a scene apparently illuminated by light of different intensities, as in the figure, the identical surfaces trigger different color perceptions. Again, these color perceptions arise, not because they represent reality, but because millions of years of accumulated vision reflexes dictate what the stimuli would usually have signified in such a scene.

In yet another experiment based on a well-known visual illusion of so-called "Mach bands," Purves, Lotto, and Mark Williams have shown that seeing is not believing, but rather believing is seeing. In the left-hand image (Figure 1), the viewer invariably perceives a faint but clearly discernible lighter band at the beginning of the gradient of shading and

a darker band at the end. In reality, no such bands exist in this computer-generated image. Rather, the image was rendered as a smoothly changing gradient of shading from light to dark. However, the brain insists on seeing the illusion of bands because of experience with real objects. That experience has taught that the reflectivity of materials means that curved surfaces typically show highlights and lowlights, as revealed in the right-hand photograph of a real block.

Besides exploring the perception of brightness and color, the group has studied how perceptions of geometry are also subject to experience wired into the visual system by natural selection. In the image of angled sticks (Figure 2), observers invariably perceive that the absolute size of the four angles is different. In reality, each of these objects is actually a right angle. (Don't believe it? Try measuring the angles with a protractor.) The perception of the angles, say Purves and colleagues Nundy, Lotto, and Shimpi, is biased by what each of the identical angles would have typically turned out to be during interactions with the sources of such stimuli in three-dimensional space.

The Duke scientists are continuing to fit together the puzzle pieces to complete the picture of their theory of vision by performing further experiments on perception of color, motion, and three-dimensionality. But even as they construct their theory, they also find themselves faced with understanding the very human puzzle of why many fellow researchers do not readily accept it.

"The idea of empirical influences on vision has been around for a long time but, to a large extent, has been ignored by people doing neurobiology," says Purves. "The reason is fairly obvious: So far, neurobiologists have made great progress using conventional techniques of anatomy and physiology to map the brain's visual circuitry. Two people have already received Nobel Prizes for this kind of work, and it has proven enormously valuable. If you're having that kind of success, you really don't need to think about whether you've got the right framework. The peculiar phenomena we have been concerned with can easily be pushed aside as anomalies that don't have much importance. After all, what could be more sensible than the idea that understanding the details of brain circuitry will eventually explain everything about how we see? The problem is that, after fifty years of work, neurobiologists still cannot explain a single visual perception, no matter how simple, in terms of visual circuitry. That tells you pretty clearly that something is wrong."

Figure 4: Dale Purves, like everyone else, sees in this scene that the surface of the top block is darker than the bottom block. However, covering up the center section, including the shaded gradients that join the blocks, reveals that the two surfaces are actually identical. Purves and his collaborators have used this kind of visual illusion to argue that the basic strategy of visual perception is to generate perceptions empirically based on experience of what visual stimuli have typically turned out to be.

Unfortunately for the effort to truly understand how we see, such reductive techniques based on mapping circuitry have led to a hodge-podge of ideas about vision, Purves says. Conventional vision theory postulates that the brain uses some still-mysterious representational logic to analyze the reality underlying the visual world, much as we would analyze a picture on a computer screen in rational terms. However, such theories are at a loss to explain the illusions and other ambiguities that form the cornerstone of the theory that Purves and his group have been formulating.

"When scientists have considered the contribution of empirical factors to vision, they've generally used it only as an 'add-on' to existing analytical theories--basically as a way the visual system tweaks the perception of images to handle ambiguities or complexities by so-called 'top-down' influences," says Purves. "We're committed to the view that the empirical contribution, far from being an add-on, is really the whole shebang. You can pretty much explain the full scope of what we actually see in this wholly empirical way, but not, as far as we can tell, in any other."

Potentially even more important than just explaining vision is the possibility that the scientists' empirical theory could be extended to explain how the brain works more generally. "It's a remarkable fact that the cerebral cortex, which is the seat of all higher brain function including vision, has pretty much the same structure throughout," he says. "All areas of the cortex look almost identical and all have much the same cortical circuitry. Thus, one implication of what we are finding in vision is that the empirical associations based on the success or failure of experience may provide a basis for understanding the function of other parts of the brain."

Despite the theory's scientific promise, some observers fear that it might chip away at the pedestal upon which we have placed ourselves, as in Shakespeare's declaration, "What a piece of work is man." Not at all, insists Purves. "People are obsessed with the idea that we human beings are not only different from other animals, but that our brains are different from our spinal cords, which clearly operate in a largely reflexive manner. The fact that our brain is not really so different from the spinal cord, except in degree, in no way demeans our humanity. In fact, our brains may be much easier to understand as engines of reflex association than as computers that analyze a scene and somehow tell an internal observer what it all means. In any case, our brains are still the most intricate and biologically effective structure in the known universe."