Hearing in the brain
Several years ago, I met with neuroscientists in Cambridge, MA. In between meetings, I stopped at the wonderful Harvard Coop bookstore in Harvard Square and found an astonishing book about photographs taken through an electron microscope. One image featured the organ of Corti in the cochlea—the cells and membranes of our inner hearing system. I was astounded at the symmetry and the beauty of the structures. The image looked like a work of art and I wondered what it would look like on film. I imagined floating through the ear, gliding past the middle ear, floating through the inner ear, and zipping up to the brain and back. And with all the membranes moving, driven by actual computational neuroscience, Hollywood-grade CGI, and a rich musical score, I became excited. I decided right then to make the film. I would name it Though a Portal to the Mind.
As I learned more about hearing, I understood that it was not just about translating motion out in the world (it can detect changes within one-billionth in density of air particles) into neural sensations, but also about detecting patterns among those changes. I wondered how much the symmetry of the structures might contribute to recognizes the patterns in music (see Patterns in a Bach Prelude and Fugue), or the cognition of any pattern, such as metaphor. And what if Rupert, who hallucinates aurally as well as visually in Light Sonata, has neurons in his hearing system that leak signals into his visual system? His hallucinations could be astounding and beautiful—or overwhelming and terrifying.
I organized my film around a thought experiment: what if we were tasked with designing hearing? Where would we start? What challenges would we face? I launched a series of thought experiments, bright in color, each illustrating how we might tackle a specific challenge, and I alternate them with how our hearing solves the challenge much more elegantly.
In this first clip, we see the challenge of how to transfer vibrations in air particles into meaningful ripples in the fluids within the cochlea.