Disney Research Pioneers New Frontiers Using Virtual Reality

Scientists from Walt Disney Company’s research division are pioneering innovative ways to optimize hand-eye coordination while using a virtual reality (VR) interface to catch a real tennis ball. According to Disney Research, the dynamic interaction with tangible physical objects in virtual reality dramatically enhances the VR experience. These findings have many potential applications for entertainment, gaming, sports training, as well as, the future development of therapeutic interventions and education-based virtual reality learning tools.

The Disney Research team of Günter Niemeyer and Matthew Pan presented their first-of-its-kind research, "Catching a Ball in Virtual Reality," this week at the IEEE Virtual Reality 2017 conference in Los Angeles. In a statement to Disney Research, senior research scientist, Günter Niemeyer, said:

"Catching and feeling the real ball in your hand makes VR much richer, more believable, more exciting, more interactive, more dynamic, more real. Catching a ball requires many coordinated skills learned from early childhood, including strong hand-eye coordination."

Niemeyer received his M.S. and Ph.D. from the Massachusetts Institute of Technology (MIT) in the areas of adaptive robot control and bilateral teleoperation. His research at Disney continues to examine human-robotic interactions and proprioception in virtual reality environments. (The cerebellum plays a central role in coordinating the unconscious aspects of proprioception.)

In their latest experiment using dynamic VR to guide participants to successfully catch a ball, Niemeyer and Matthew Pan, Disney Research lab associate and a Ph.D. student at the University of British Columbia, have shown (for the first time) that the hand-eye coordination required to catch a real ball is possible in virtual reality.

During this experiment, Pan and Niemeyer examined three different VR ball-catching situations. Interestingly, study participants had a 95 percent success rate catching the ball in all three VR circumstances. In fact, the ability of the VR system to predict the flight of a tennis ball and assist the participant in visualizing the ball's trajectory gave the catcher a practicing advantage not available in the real world. In a statement that touches on the brain mechanics involved with catching a ball, Pan explained the high success rate:

"The most apparent explanation is that, without information about the ball's location, the catcher must rely on the identified target point, changing the task from one requiring higher brain functions to estimate trajectory to a simpler, visually guided pointing task."

Notably, the predictive assistance of VR visualizations enhanced the ball catcher's senses but also helped fine-tune an effective strategy for catching the ball within milliseconds of actually making contact with the target, as you can see in the video from the Disney Research lab below:

Your Cerebellum Is the Seat of Proprioception and Hand-Eye Coordination Required to Catch a Ball

The new Disney Research doesn't dive deeply into the neuroscience of their advances in virtual reality ball catching. That said, I've spent decades researching and writing about the role that the cerebellum (Latin for "little brain") plays in optimizing sports performance and human potential both on and off the court.

Based on my fascination with the cerebellum, I was excited to look at the "Catching a Ball in Virtual Reality" experiment through the lens of cerebellar proprioception, hand-eye coordination, and VOR. (Cerebellar is the sister word to cerebral and means "relating to or located in the cerebellum.")

The vestibulo-ocular reflex (VOR) is an automatic reflex that produces an eye movement in the direction opposite to head movement when tracking a target, or catching a ball. The VOR of the cerebellum allows someone to focus and track any moving object in sport and life. Without the VOR system of the cerebellum, you couldn’t play hand-eye coordination sports or catch a real ball in virtual reality. Also, having an "eye for the ball" is a reflection of a robust VOR system.

As I mentioned earlier, your cerebellum oversees the unconscious proprioception necessary to constantly monitor your body’s position in space. It also orchestrates the precise timing of fine-tuned muscle movements by comparing the speed of an incoming object to the position of your body, limbs, hands, etc. This extends to using a catcher’s mitt, tennis racket, baseball bat or any sports equipment to hit a ball or hockey puck.

From an evolutionary perspective, the "blindsight" capacity of the cerebellum is believed to be a leftover from our reptilian roots. In modern times, blindsight and a robust VOR allows a well-trained athlete to track a moving ball or hockey puck at lightning-fast speeds . . . just as a lizard uses blindsight to catch a fly with its tongue in the blink of an eye.

Practice enhances an athlete's cerebellar abilities across the board. Years of practice, practice, practice is how Babe Ruth perfected the cerebellar skills needed to hit a speeding curveball out of the park. Just as, decades of practice enables Andy Roddick to serve a 155-mph ace without even seeing the tennis ball above his head while hitting the sweet spot of his racket.

The Cerebellum Allows Athletes to "See Without Seeing" & "Know Without Knowing"

Wayne Gretzky famously said "A good hockey player plays where the puck is. A great hockey player plays where the puck is going to be." This insight touches on the cerebellar power of an athlete to avoid "overthinking" while trusting his or her gut instincts.

My late father, Richard Bergland, was my primary tennis coach as I was growing up and during my adolescence. He was also a neuroscientist, neurosurgeon, and former tennis champion who incorporated his fascination with the cerebellum to “know without knowing” into his tennis coaching. My dad's methods always reminded me of Obi-Wan Kenobi trying to teach Luke Skywalker how to use a lightsaber with the blast shield of his helmet down or to trust The Force enough to turn off the guidance computer in his X-wing starfighter.

When I first watched the Disney Research video (above) of the study participant catching a tennis ball wearing a virtual reality "shield" that blocked his actual vision, I had flashbacks to 1977 when I was an 11-year-old aspiring tennis player who dreamed of becoming the next Björn Borg. And who was also obsessed with how Luke Skywalker used The Force in ways that optimized his intuition and blindsight.

As Obi-Wan Kenobi says in the famous Star Wars scene below, "This time, let go of your conscious self, Luke. And act on instinct." (For the record: I credit the archetypal lessons of Joseph Campbell's Hero's Journey and The Power of Myth knowingly woven into the Star Wars movie narratives by George Lucas for inspiring me to break a Guinness World Record and explore 'extraordinary worlds' as an ultra-endurance athlete. And, then, return home to the 'ordinary world' to write about the lessons I learned on my adventures for a mainstream general audience in The Athlete's Way: Sweat and the Biology of Bliss.)

Unwittingly, my dad's coaching advice always echoed that of Obi-Wan Kenobi in my mind's eye. As an example, my father used to make me practice my tennis serve endlessly while wearing a blindfold. Practicing my serve with the equivalent of Luke's blast shield down, strengthened my cerebellar abilities to know where the ball was without using my eyes.

Anecdotally, I can attest that the muscle memory held in my cerebellum from these "blindsight" drills is indelibly encoded into my cerebellar neural circuitry. Although I only play tennis once every few years these days... after just a few minutes of warming up, I can successfully serve a tennis ball with my eyes closed—even though it's been decades since I practiced this blindfolded tennis drill regularly.

As a neuroscience-based tennis coach, my father would say esoteric and potentially unrelatable things that often went over my head. Such as, "Chris, think about hammering and forging the muscle memory held in the Purkinje cells of your cerebellum with every stroke." Luckily, my dad had the intellectual humility to explain what he meant without making me feel like a "dumb jock" when I'd ask, with a confounded look on my face, "What on Earth are you talking about?!?"

Nevertheless, in an ongoing attempt to make my father's academic and cerebral language more relatable, I've always imagined ways to apply his neuroscientific advice by using cinematic visualizations or pretending I was the protagonist walking in the same shoes as my matinee idols.

"Of this, I am absolutely certain. Becoming a neurosurgeon was a direct consequence of my eye for the ball." — Richard Bergland, M.D.

As a neurosurgeon, my father credited the cerebellar skills he mastered on the tennis court as a transferrable skill that he brought with him to the operating room. He'd often say, "Of this, I am absolutely certain. Becoming a neurosurgeon was a direct consequence of my eye for the ball."

To my father, becoming a brain surgeon was akin to being a well-trained athlete or a master sculptor. He considered neurosurgery to be much more cerebellar than cerebral. In fact, my dad said that if he thought too much about what he was doing in the O.R., it would take him out of his "flow channel" and being "in the zone." This made it more difficult to experience superfluidity and maintain grace under pressure.

“Unclamping” the rigid control of executive functions in his prefrontal cortex is a visualization and terminology my father borrowed from William James' "Gospel of Relaxation" and shared with me as a rookie athlete to help me learn how to avoid "paralysis by analysis" on and off the court.

My dad knew from life experience that “overthinking” caused him to tense up and choke during both match points and high-pressured neurosurgical procedures. But, through decades of practice, he taught himself how to wholeheartedly trust the cerebellar "knowing without knowing" skills held in his cerebellum. In doing so, he went on to become chief of neurosurgery at Harvard Medical School's BIDMC, and other teaching hospitals.

From my pop culture perspective, I always translated my father's academic vernacular into a personalized explanatory style, such as tapping into The Force being all about unclamping my prefrontal executive functions and seating myself in the cerebellum. In my imagination, Obi Wan's words of wisdom to Luke mirrored my dad's advice to trust the muscle memory that I'd 'hammered and forged' into my cerebellum via endless practice, practice, practice.

Daily practice optimizes the functional connectivity of your cerebellum and allows for peak performance without having to overly engage cerebral neural circuitry or think too much. Of course, anytime someone starts to overthink on the playing field or tennis court, he or she is much more likely to choke, fumble, and drop the ball. (I wrote about this in a Psychology Today blog post, "Why Does Overthinking Cause Athletes to Choke?")

"Imagineering" Future Applications of Virtual Reality Technology

For decades, virtual reality has been used in flight simulators to teach pilots (such as my sister, Sandy Bergland, who applied the cerebellar lessons of our father to become a Boeing 777 pilot for FedEx) how to problem solve and have grace under pressure in the heat of the moment during worst-case-scenario emergencies.

Again, the underestimated value of VR technology is that it automatically engages the proprioceptive aspects of the cerebellum which allows someone to function on "autopilot" using well-trained implicit memory while simultaneously freeing up the cerebral working memory to problem solve effectively.

Clearly, Walt Disney's penchant for combining creativity and innovation is thriving at Disney Research. In terms of "imagineering" (a term the father of Disney coined to represent a blend of imagination and engineering) future uses of VR technologies that could genuinely improve people's lives, the possibilities seem limitless.

"As virtual reality systems become increasingly common, the idea that the user experience can be enriched by enabling dynamic interaction with real objects is gaining interest," Markus Gross, Director of Disney Research Zürich, said in a recent statement about catching a ball in VR. "This early work by our team is tantalizing and suggests that bridging the virtual and real worlds is not only possible, but offers many new opportunities and benefits."

Hybrid Interfaces Combining VR and Real-World Elements Could Be Used to Treat a Broad Range of Cerebellar Disorders

Last week, in a completely different area of cutting-edge neuroscience research, Stanford University scientists stumbled on a previously unknown function of the cerebellum using another state-of-the-art technology called "two-photon calcium imaging." The Stanford researchers discovered (for the first time) that granule cells in the cerebellum encode and predict rewards. These findings corroborate a groundswell of other research in the past two years suggesting that "unconscious" neuronal activity in the cerebellum may play a role in driving addictive drug-seeking behaviors.

I'm optimistic that the latest empirical evidence from Stanford linking the cerebellum with reward-seeking behavior and the Disney Research melding virtual reality cerebellar tasks with real-world objects could someday be "imagineered" to create personalized drug interventions that engage cerebellar responses to drug cues in a safe and controlled VR environment.

Also, blending real-world elements that trigger post-traumatic stress disorder (PTSD) in a safe and sound virtual reality setting could be a way to help combat veterans, many of whom have PTSD that is linked to cerebellar damage created by micro-blasts in the war zone. (I wrote about these findings in Psychology Today blog post, "Cerebellum Damage May Be the Root of PTSD in Combat Veterans.")

Another potential application of melding VR and the real world to improve well-being in cerebellum-related deficits is autism spectrum disorders (ASD) which have been shown to have a definitive cerebellar element. For example, combining eye-tracking technology and vestibulo-ocular reflex exercises in a VR setting that somehow incorporates the real world, could help children with ASD practice making predictive eye contact in a variety of controlled social situations. Similarly, virtual reality technologies could be developed to help patients with ataxia after a cerebellar stroke or brain tumor.

In terms of positive psychology and everyday uses of VR in the future, advanced virtual reality tools similar to those used to coach an athlete could be tailored to coach someone on specific techniques needed to ace a job interview by mastering proprioceptive aspects of human interaction held in VOR and body language.

On the potential dark side of this new technology, there is always the possibility that relying too much on VR advances could create a type of 'Future Shock' or disconnection from face-to-face human interactions. This type of futuristic dystopia was recently explored in the "Playtest" episode of Black Mirror. In this episode, the protagonist (Wyatt Russell) gets trapped in a virtual reality game that short-circuits. The game creators are unable to turn off the computer which makes it impossible for him to escape the virtual world full of his deepest fears and nightmares created by the VR game specialists.

In closing, it's worth noting that in the final scene of the original Star Wars film (A New Hope, Episode IV) Luke hears the words of Obi-Wan Kenobi saying "Use the force, Luke. Let go!" and switches of his VR-based computer targeting system. Using his intuition and not relying on a VR computer ultimately allows Luke to evade Darth Vader and blow up the Death Star. In some ways, this scene could be viewed with hindsight as a prophetic reminder that, in a 21st-century digital era, we should be cautious about becoming too dependent on computer technologies and strive to keep our cerebellum functions strong through real-world life experiences, such as playing sports, daily physical activities, and social engagement.

Obviously, many things that can be coached and mastered in virtual reality can (and should) also be practiced and fine-tuned without any VR screentime whatsoever. That being said, these are exciting times for pioneering research in both the worlds of virtual reality and neuroscience. Hopefully, upcoming neuroscientific and VR discoveries will be applied in a marriage of virtual reality with 'everyday reality' that improves daily lives and fortifies the best in all of us.