Friday, October 5, 2007

Alan Rath, sensory substitution and Terry Pratchett

Alan Rath has been building interactive robots and video sculptures for at least 20 years. Many of his sculptures deal with surveillance and observation, although he takes a lighthearted approach to the subject. Many of his pieces appear to be watching the viewer and some have sensors that activate when a viewer comes near. "One by one the artworks come alive as I walk into the studio in the morning." [1]

A recent interview of Rath at Berkley made me realize how similar the concepts I have begun to research are to his ideas. [2] Rath calls the inspiration for his work, "images in the back of my eyeballs." He is touching on the characteristics between the senses and the conscious mind that interprets our senses, which comes up with ideas and intuitions, and make decisions based on our senses. He goes on to say, "We live more in language than in physical space," referring to the symbolism of words, the fact that we create simulacra of our world so that we can make simple decisions like eating breakfast. (Which is a daunting task when you start to think about everything that is involved from gathering, packaging and storing the food, to preparing it and storing it in the refrigerator and preparing it the morning of your breakfast. When you add the bio-mechanical movements of the arms, hands, and individual fingers, legs, feet, and toes that are required to perform each of these activities, its a wonder we are able to eat at all, let alone get out of bed in the morning.)

The human mind, ever vigilant in its pursuit of simplifying the universe around it, so that we can get up in the morning, manifests its most important evolutionary trait: the skillful categorization and quantification of the universe around it. Add a symbolic representation to each of these actions in a hierarchical of scale, and suddenly the dauntingly impossible becomes becomes possible think about. 'Open the fridge door' can be combined with 'move the hand so it grasps the egg,' into 'get an egg.' This in turn can be combined with 'turn on burner', and 'place pan on stove,' into 'cook eggs.' Language is the expression of the mind's desire to create symbols, so that it doesn't have to continuously think about everything.

Writer Terry Pratchett also uses the metaphor of the darkness behind the eyes in Thief of Time. In his novel, non-corporeal beings of non-human origin (which, incidentally have nearly limitless senses) try to occupy human bodies, they are suddenly struck with the simplicity of the five senses. Furthermore, they are absolutely confounded by the little voice, that continuously tries to interject itself into every thought.
“It is essential for humans to use the personal pronoun. It divides the universe into two parts. The darkness behind the eyes, where the little voice is, and everything else. It is…a horrible feeling. It is like…being questioned all the time.” [3]
When describing the world, we can only use our senses and inferences based on the senses. Using Plato's Allegory of the Cave, its very interesting to think that we are limited to our senses, and that because it's the only thing we know, it is the only way we can describe the world around us. [5] We use words (as symbols) to label something, like 'the big red car.' Visually, anyone without a sight disability can visualize the car, and have a very good idea what it looks like. Most everyone without an auditory disability knows what it sounds like. Most everyone knows what the smooth shiny painted surface of a car feels like.

But what is a car, that can't easily be described by the senses? (Perhaps a useful way to look at this question is what else does a car symbolize that can't easily be described be the senses?) What if we had a sense that saw heat changes as a vehicle moves through an area? What if we could see the CO2 expelled by a car while driving by? What if we could see the driver's bank account draining as he drives a large red suburban, guzzling gas? What if we could see the increases in happiness of a the children inside increase as they all go to an amusement park? Can these be considered "senses?"

Sensory substitution is intended to assist visually or hearing disabled people by using the remaining senses. In the case of sight, visual images are translated to touch. Tactile devices are placed either on the tongue since it is so sensitive or on the neck or back of the hand. A high enough resolution can actually provide people (or more accurately the brains of people) who used to see with enough information to actually understand what they are "seeing." Much of the research is focused on the blind, but there are studies using it for the deaf as well.

In subjects who lose a sense, the brain already 'knows' how to hear and understand sound, these devices simply retrain the brain to use a different sense (a transduction of senses.) But the brain actually ends up utilizing the already developed part of the brain after practice with the device. It is my understanding that the brain is very flexible, and can adapt very well, and very quickly. But is it really this fluid? what does this say about our senses? What if we grew up with different senses? Would the brain grow to simply adapt them if they were a part of evolution? Does it even matter? I'm beginning to think not. recently, I read about someone who is embedding magnets into the skin of a few willing participants. After a little practice, these individuals can learn to detect magnetic fields very easily. [4]

So what is this saying about our senses? Our senses are mutable. Consider this: Human senses are simply reacting to the energy of electromagnetic fields. Vision, for example, is a reaction to photons - pure energy - reflecting off of objects. Photons don't bounce off objects like billiard balls. Matter itself is mostly space. The distance between particles that make up atoms are so immense, that matter is really 90%+ nothing. If photons acted like billiard balls, they'd mostly just pass through everything. Vision would be useless, there would be nothing to see. [7]

Rather, they reflect like waves off of the fields created by the atoms in objects. [6] From a certain scientific point of view, nothing really exists as matter, and our senses are simply detecting energy fields. Smell reacts to combinations of fields that make up molecules. Touch detects the repelling force these fields exert on our skin. Hearing detects the motion - kinetic energy - of air particles, which are themselves comprised of energy. Our mind is doing an incredible thing, reconstructing all this information about complex fields around us, into a form that we are able to understand, describe, document, share and remember.

I don't suggest stepping in front of any trucks, thinking that a bit of mostly empty space won't hurt you. The energy contained in that empty space creates a strong enough field to scatter your body's fields in every direction. But I do think we should consider what it means if everything is made of energy. How 'real' are objects? We never really do touch anything - the fields surrounding the atoms in abjects never let us actually touch them. We never really actually see anything - just the reflection of photons off of objects. All we are doing is creating our interpretation of the world around us in our mind from the stimulus that our senses detect. In a way, everything is just in our heads.

1 Bing, Alison. "Alan Rath: Meta Mechanics." Sculpture (Washington, D.C.) v. 25 no. 7 (September 2006) p. 34-9
2 Alan Rath, Interview at Berkley. http://bampfa.berkeley.edu/media/rath.mp4 Viewed 09-26-07.
3 The little voice might be described as the conscience, or possibly the ego. We all have it. For those of you who insist you are without either one, I may lose you here, but see if you can keep up. Pratchett, Terry.
Thief of Time.
4 Body Hacking. Make Magazine? I'll need to try to find this. For now: http://www.makezine.com/blog/archive/2007/04/body_hacking_if_you_cant.html
http://www.ambiguous.org/quinn/bodyhacking.html
http://conferences.oreillynet.com/cs/et2007/view/e_sess/10352
5 For a quick synopsis, see: Wikipedia.
Allegory of the Cave. http://en.wikipedia.org/wiki/Allegory_of_the_cave. Visited 10-04-07
6 Fermilab.
All About Light. http://www.fnal.gov/pub/inquiring/more/light/
7 The gold foil experiment showed that the atom was comprised of mostly empty space with a concentrated positive center. http://en.wikipedia.org/wiki/Atomic_theory Visited 10-07-07

Also see:
Alan Rath's website
Make Magazine

No comments:

Post a Comment