The Tongue Display Unit (TDU) at UPMC: seeing with the tongue?

Courtesy of staff at University of Pittsburgh’s Medical Center (UPMC), in September 2013 I was invited to the Sensory Substitution Lab to do something that I have wanted to do for years: have a hands-on experience with a device that I have written about. WICAB’s ‘BrainPort’ is a ‘sensory substitution’ device that translates a visual feed from a camera into impulses on the tongue. This was designed for those with blindness or severe vision impairment, and was featured in US and UK news covering veterans returning from Afghanistan and Iraq. There is a lot of promise in this technology, and the BrainPort is the legacy of pioneering research in Tactile Visual Sensory Substitution (TVSS) systems by Paul Bach-Y-Rita.

I write about this in a chapter (‘Seeing with the tongue’) in my forthcoming book on blindness Seeing With the Hands: Blindness, Vision, and Touch after Descartes [see this post], but also a chapter ‘A Methodological Molyneux Question: Sensory Substitution, Plasticity and the Unification of Perceptual Theory’, with Mazviita Chirimuuta, in the collection Perception and Its Modalities (Oxford UP, 2014) [link to version on].

A research subject assessing the different shapes through the TDU, and reaching to them
A research subject assessing the different shapes through the TDU, and reaching to them

I was the guest of Amy Nau and her team, who have had funding at the Sensory Substitution Lab to test research subjects from the police and military, and who have become suddenly blind due to injury. Their research (e.g. see this paper) uses the BrainPort to assess the identification of three dimensional shapes (as the above photograph shows). I had a go with the same equipment and test setup (see photo below, with my fingers).

The photograph shows the test setup, with the camera’s view displayed on the laptop screen, and the darkened area to maximize visual contrast for the camera

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Blog at

Up ↑