Verified by 
Neuroscience
Of Spider-Man Movies and Other 3D Thrillers
How the brain processes images in three dimensions
Posted July 24, 2012
I just got home after seeing The Amazing Spider-Man, and I have to admit, the childlike joy that floods my synapses during such an experience is hard to beat. I love the growing number of 3D films that are available and the thrilling entertainment they provide. But ever since I wrote Brain Sense, even simple recreational activities set me to wondering how the brain produces what the mind perceives.
To address that question, I’ve recently had the privilege of corresponding with Barry Sandrew, founder and CTO/CCO of Legend3D, a leading innovator in 2D-to-3D conversion technology. Dr. Sandrew has an extensive background in neuroscience and a roster of 3D conversions for blockbuster hits including Transformers: Dark of the Moon, Hugo, Shrek and the upcoming theatrical re-release of Top Gun. With 14 VFX patents and 23 years of feature film and TV experience, he has a wealth of expertise to further explain how the brain (and audiences) absorb 3D content and why 3D can enhance the movie-going experience.
In response to my general queries, Dr. Sandrew provided a set of questions and answers that were so enlightening, I’d like to share them here, pretty much in their entirety:
Q: How does the brain process a 3D image?
A. As binocular beings, we have two eyes each containing a retina made up of a sheet of specialized cells that are activated by light of various wavelengths. The lens in each of our eyes focuses reflected light from images in the outside world onto its corresponding retina. Because our eyes are approximately 63mm or 2.5 inches apart, the light reflected off an object in front of us hits each retina with a slight horizontal offset. We call this slight horizontal difference between our left and right eyes “disparity,” and it’s disparity that makes things appear 3D to us.
The best way to experience disparity is to put your thumb in front of your eyes and alternately close one eye and then the other. Your thumb will appear to jump from side to side because the two separate perspectives of your thumb that each eye receives are displaced horizontally.
When you open both eyes, the separate images from your left and right eye reach your visual cortex, where they are directed to highly specialized neurons that are tuned to critical parameters of disparity. As a consequence, the images from your left and right eye are fused into one uniquely different image that exhibits the appearance of depth and volume. From there, neuronal processing is spread in a hierarchical manner to associative and other parts of the brain.
Because disparity gets bigger the closer something comes toward us, disparity is also an accurate indicator of both speed and direction. An object that is close to our face has a greater disparity and consequently has more three-dimensional volume than an object several feet away. If an object is far enough away (between 100 and 200 yards) it will appear to have no three-dimensional volume at all.
Q: What does it mean to “see in 3D”?
A: Most of us take 3D vision for granted. We don’t give it a second thought. We receive three-dimensional information about the world around us using double-eye or binocular cues, but we don’t often appreciate the large amount of three-dimensional information that is derived from only two dimensions. When the image of one thing overlaps another, we know that the overlapping object is closer to us. Shadows can give the impression of contour and depth, even from a 2D image. Texture, brightness, and color saturation are all affected by distance in 2D, and consequently they can offer additional three-dimensional cues.
One of the most important two dimensional cues is parallax. The best way to describe parallax is to use the old car window example. When you are in a car looking out the window, trees in the foreground are whizzing by while forests in the distance are moving comparatively quite slowly. This gives us cues about the relative position in space of the trees in the foreground and those in the background.
The actual separation of items in space and their volume can only occur when we have two functioning eyes that are sufficiently aligned to register images correctly on our retinas. The distance between our eyes has likely evolved over time and is appropriate for our particular species. If our eyes were closer together, things would appear bigger and the three-dimensional volume of space would be reduced. If our eyes were farther apart things would look smaller and the three-dimensional volume of space would be much greater. Our binocular vision allows us to do close manual work like making weapons and harvesting food, while also providing us with knowledge of potential dangers around us to better detect predators. That said, the survival value of binocular vision is extremely significant to the longevity of our species.
Q: Why do we have to wear glasses when watching a 3D movie?
A: In a 3D movie two sets of images are projected on the movie screen, a left image and a right image. In order to have the audience experience a movie in 3D, the left image must be directed to the left eye of each member of the audience and only the left eye, and the right image must be directed to the right eye of each member of the audience and only the right eye. There are different ways of accomplishing this and they all involve glasses.
Experiencing 3D in a movie theater is totally different than the way we experience 3D in everyday life, as in the example above describing how disparity works using your thumb. When you put your thumb in front of your face, you are both focusing on the thumb and converging or turning your eyes inward on the same thumb. In a 3D movie theater focusing and converging become distinctly separate functions. The only place in a movie theater where things are in focus is at the screen. There is nothing in front of the screen or behind the screen. As a result, we focus on the surface of the screen, but it’s the disparity between the left and right images that our brain uses to effectively converge behind, in front of, or on the screen.
For More Information:
See Part 3 of Dr. Sandrew's interview at 3D Vision and the Brain.
Visit Dr. Sandrew’s blog, Reflections on the Growth of 3D: “People Who Hate 3D Movies Should Get Their Eyes Examined” and “How Hugo Gave One Neuroscientist the Gift of Stereo Vision”
