NEW YORK - Gil Weinberg is pressed for time. He knows exactly what innovation he wants to introduce into his electronic Beatbug percussion devices before their debut in Israel, but he isn't sure he'll be able to do it on time. It's understandable. After all, no one has yet managed to transfer a sound from place to place by means of a hand movement that resembles the motion of casting a fishing rod. Not that the Beatbugs Weinberg developed four years ago at the Massachusetts Institute of Technology (MIT) media lab can't "catch" sounds - but these capabilities, which were satisfactory before, no longer do it for him. He sees what kids are doing with them and now he feels he just has to add some more sensors to all the Beatbugs. He has to take them to the next generation.
As of now, all that someone need do if he wants to transfer the sounds he's created to another Beatbug, is to aim the front part toward the "target" Beatbug and then press hard on its back. Within a fraction of a second, the tune that up to that moment came from the first Beatbug, begins to play from the other musician's Beatbug. That musician can then add elements of his own to the melody, press hard and pass it on again.
The youngsters who were asked to test the Beatbug didn't just press on them in order to send sounds from one to the other, says Weinberg, his blue eyes flashing with excitement. They added dramatic hand motions, as if the music was an invisible ball they were throwing and catching. "From interviews that we did with them, we found that the kids would like it if the movements they made would produce music. They developed it into interpersonal communication, and the computer didn't know about all these movements. It was so gesture-connected that we said it would be a shame not to sense this, too."
The music, Weinberg learned from awed observation of the children, can even be more substantial than he ever imagined. For the youngsters who held his Beatbugs in their hands, the music was far from being just "a collection of ring tones, or changes in air pressure." When they aimed the Beatbugs at each other and sent one another beats and voices with dramatic waves of the arms, they could almost see the sounds and touch them. And this is what Weinberg always wanted. To touch music.
Over the past three years, since he completed the student chapter of his life after receiving his doctorate from the MIT media lab, Dr. Gil Weinberg has become something of a wandering artist. After he left Cambridge in Boston, he established the Georgia Institute of Technology Music Technology Research Group, and has headed it ever since, teaching and doing research. But unlike other scientists, Weinberg appears not only in the classroom or the lab, but also on stages all over the world.
During a packed week of events starting on Sunday (March 19), Weinberg will appear and host artist workshops at Hama'abada (The Lab) in Jerusalem. A unique experience is definitely in store for anyone who comes to see him using the unusual devices he has invented and hopes to bring into Israel without arousing any suspicion. A group of Jewish and Arab musicians will accompany him on some of the evenings. Where else can you hear a concert featuring an instrument that resembles a rolled-up sock, experience a piece involving a rapid dialogue between such instruments, connected to each other by a network, and observe a percussion workshop in which a large wooden robot and a human drummer improvise together on a huge Indian tom-tom, as if this were a normal collaboration between two veteran musicians?
For Weinberg, 39, this is the pursuit of a dream, one that will never let him sit idly and quietly by. Quiet isn't what he's after, certainly not when he's always just one sensor or finger-press away from the next surprising sound that might emerge at any moment from the unusual orchestra that accompanies him everywhere. It's not clear who's leading whom, and Weinberg likes it that way. The stage is still a little foreign to him now, though it's becoming less so as he and his ensemble have been filling performance halls around the world.
"Whenever you work with musicians or children, you see that it's possible to add something more to make it better. That's the idea with technology. You never stop developing new things, there's always something to add," says Weinberg.
On his travels, he and his collection of gizmos often raise eyebrows. In airports, they sometimes trigger a lot of beeping and questioning. "I like being in this groove of 'what the heck is he doing?'" smiles Weinberg. But the responses are always enthusiastic. The Wall Street Journal praised his work "Nerve," saying the piece "in which six older children tossed rhythms back and forth was engaging and interactive." Weinberg's instrument, called the Shaper, was presented on a prime-time CBS News report and the inventor was described as "one of the most brilliant minds in America."
Already as a teenager, Weinberg knew that he wanted to create music in a different way, though at the time he certainly couldn't envision the sorts of instruments he would eventually create. "There are two things that you know about yourself: that you were born and that you'll die. In the middle, you have to do something with yourself. There are people who are busy surviving. There are people who try to obtain more money and power. There are people who try to find meaning in religion. I decided that I would invent something of my own, that I wanted to create something, to leave something behind, and music is the most abstract art there is."
Raps on the hands
At age seven, Weinberg, a Jerusalem native, began to take piano lessons. "I played because they sent me to play. I didn't like it at all in the beginning. The only thing my teachers cared about was that I sit straight and hold my hands straight. I got hit on the hands, all these little raps ... It was a strict teaching method that put the emphasis on technique and theory before you were even allowed to think at all about being expressive, about being creative," recalls Weinberg, who as a boy made a number of attempts to write pop music.
"When I came in with my own pieces they'd say, 'What are you doing? You have no right to start doing composition, to express yourself, or to challenge yourself before you have the fundamentals.' I think this is exactly the opposite of what music should be. It's part of the reason that I'm doing what I do today. I'm trying to get children to be creative and expressive long before they have technique and theory. They can express themselves by pushing, pulling and other motions. Music is something you can invent and improvise with. I'm sure that this is a much better way into this world than focusing on technique and theory in the beginning."
Weinberg wanted to simplify the learning process and make entry into the world of sounds more accessible for kids. "Thanks to technology and computers, it can be done. I can invent these instruments, take these new gestures and program the computer to help kids deal with things like technique and theory," he explains. "For example, everyone can sense tension and relaxation in music - it's part of the reason we love music. But with my instruments you don't have to know the difference between the tonic and the dominant. With my instruments, the more your press, the more tense the harmonic progression you create. The technology allows you to connect directly to the creative, expressive side of music. Later on, if they want to know what the tonic and the dominant is, they can go learn that, too."
In high school, after the requisite Pink Floyd-Led Zeppelin-Beatles period, Weinberg started listening to jazz and has done so ever since. At Tel Aviv University, where he studied in the interdisciplinary program and focused on musicology, he was exposed to 20th-century modern music. Two things made him stop writing music. "First, I discovered that not everyone is equally talented and that I wasn't as talented as, say, [Theolonios] Monk, or John Lennon," Weinberg says. But the second reason was much more significant. "Music is an abstract thing," he notes. "But there's something about that that bothers me. You work all night, you finish the last note on a piece, you get up in the morning, press play and lean back. It's something powerful. But I was missing something concrete - I wanted to be able to see the piece, to hold it."
While still a student, Weinberg's attention was also caught by a simple musical notation computer program. "Maybe that's what did it for me," he says. "A program is something I can see and touch. I thought maybe I'd try to create music software."
Eclectic assignments
In 1991, Weinberg and his partner Yigal Barkat founded a company called Sense Multimedia, which took on a very eclectic variety of assignments in the Israeli multimedia world, which was still in its infancy at the time. Among other things, the company created a CD-Rom edition of the Carta atlas, set up The Third Ear Web site and developed one of the first karaoke software programs. After three years, in 1994, Weinberg decided to move on and began working for the Music Notes company, where he established the multimedia division and developed products on the basis of optic technology for reading music, as well as a software program for teaching piano.
But Weinberg's goal wasn't to invent a software program or an obedient robot that would create shortcuts for himself and others. What motivated him then and still does today is an endless search for a truly different musical experience.
"I wanted something that would provide an unmediated connection with a musical instrument, something you could touch, look for and discover, things with which you could create a new music," he explains. "A lot of musicians would say that computers are an excellent thing, because they are an excellent tool. That is, I have an idea in my head and the computer can help me express it or edit it. Not to disparage that, but I think it's just a waste to think of the computer as a tool. It's not nice to say that about the computer. Why should it be only a tool or a servant? If you know how to work with it, it can be a musician, a full and integral partner, inspiring, full of expression, creative. It can throw you in directions you never thought you'd go. So it's certainly not just a tool. I'm trying to get it to play me the way that I play it, and to make something new happen in this interaction between human and computer.
"I'm not trying to use the computer to make a reproduction of what I can already do," Weinberg continues, his words coming in a rush. "Some people will say: 'Why do I need to pay a saxophonist and a string section when I have MIDI sounds here that sound the same?' Well, first of all, it never really sounds just like an acoustic sound. But even if it did sound the same, why do it? Get a little more money and pay a real musician who will play it right. It's important to me not to use computers to imitate, but rather to create new things."
Anyone who frequents the creative twilight zone between technology and art knows that the perfect hothouse for transforming ideas from vision to reality is none other than the MIT media lab, where Weinberg arrived in 1997. During the six years he spent in Cambridge, this alien-looking orchestra of shapes and sounds began to take shape (in the form of sensors and wires). "It was one of the most incredible periods in my life. There are energies there of creative people collaborating with each other," he says with palpable emotion.
Once Weinberg's excitement over the wealth of resources at MIT and the brilliant minds that surrounded him gave him a little space, his advisers asked him to start thinking about what he really wanted to do there. It didn't take him long to go back to his old dream, which in Cambridge seemed attainable for the first time: to create an instrument that would enable whoever played it to feel the music, literally.
"I felt like I was missing something squeezable, like jelly," says Weinberg, describing the thoughts that led him to start developing the instrument he eventually named the Shaper. "I wanted something rich in form that you could touch, pull, feel, press deeply, and it would react."
Basically, it sounds like he wanted a cat. "Something like that, yes," Weinberg laughs, not seeming to rule out the idea, though. "Anyway, we tried all kinds of materials - from balls for tension release, to gel and sponges. We went to all kinds of stores and pharmacies and bought existing materials that we tried, but in the end, the people from my group came up with the idea of using a conductive fabric."
The transition to the unique material proved to be the breakthrough to the achievement of Weinberg's dream. His revolutionary musical instrument looks like a palm-size ball covered with a pretty, colorful fabric. It has a dense, but not too dense, texture and beneath the high-tech fabric are sophisticated sensors. From the ball extends a long cable that connects to the computer and conducts the information to it, which is processed and turned into sounds.
"After we decided to use the conductive fabric, we tried all kinds of materials and put all kinds of sensors into it," Weinberg recounts. "I tried to see how sensitive the Shaper was to what you did to it, and while doing so I wrote a program. If it just does 'do-re-mi,' then we haven't done much. So we tried to develop algorithms that also push the program and enable you to control the tension and relaxation, and let someone who doesn't have a lot of theory and technique be expressive and creative."
Once he felt that he'd gone as far as he could with the Shaper, Weinberg experimented with other unorthodox musical instruments. For example, he created a musical instrument based on a smart baby crib. In each corner of a crib packed with colorful plastic balls, Weinberg placed movement sensors that were connected to a computer. A program installed in that computer picks up movement signals from the sensors in the crib, in accordance with the amount of pressure applied, and then transmits them as sound to loudspeakers. When Weinberg put small children in the crib (often his eldest son, Yonatan), they moved around and tried to figure out what was making the music come from each corner of the crib. Within a very short time, Weinberg demonstrates with a video that depicts the development process, the child intuitively understands that he is controlling the music and not the opposite, and that there is a causal relationship between the level of energy he uses and the volume of the sound that results. In 1999, when the musical crib was fully developed, it was displayed as an exhibit at the Boston Children's Museum.
Weinberg then began developing his interactive Beatbugs, which became his most communicative project to date. He says he was interested in creating an orchestra in which the instruments would be mutually dependent, in which one musician could affect the "color" of the sounds produced by the other musicians, which is not possible with regular instruments. Weinberg performed with them in schools, labs and concert halls in Japan, America, Europe and Australia - sometimes with renowned orchestras like the Berlin Symphony and the Irish National Orchestra - and was close to selling the rights to Beatbugs to the giant Mattel toy concern, but the deal fell through in the end.
Human or alien?
Three years ago, after completing his doctorate at MIT, Weinberg and his wife and two children left New England and moved south to Atlanta. The main projects he worked on after establishing the Georgia Tech Music Technology Research Group were Brain Waves - in which biologist Oron Katz and his companion Yonat Tzur helped Weinberg achieve the sonification of the neural activity of fish - and Iltur, a system that uses algorithms for collaborative composing.
But Weinberg still wasn't satisfied. "What really bummed me out in the past two years after I was very enthusiastic, was that the sound that comes from my developments with speakers is still limited electronically. It still isn't as rich as ..." At this point he starts tapping rapidly on the coffee table in front of him, on the couch he's sitting on and on its wooden backrest, "... something acoustic that you hear. Also, when you play with the computer you don't see it answer you. You can't jump in with it at the end of a song like a guitarist or drummer and finish it off."
All of these drawbacks led him, in 2004, to create Haile: a percussionist robot that reacts to the drum strokes of a human player sitting opposite it and that is capable of improvising along with him.
"I needed something that was real, something that brought the advantages of the rich sound of real instruments," Weinberg explains. "I asked myself: What could allow me to enjoy the best of both worlds - to enjoy the rich sound and the visual interaction, and the intelligence and power of the computer. And the answer was - robots."
The Haile robot began as a mechanical arm attached to an engine, a computer and a microphone, but at a certain point in the development process, Weinberg decided it needed a personality that it could only acquire if it had an outer body to hide its mechanical insides. In conjunction with the Georgia Tech architecture school, Haile was reborn as a wooden, somewhat alien/somewhat human figure that leans over a traditional Indian drum.
Weinberg will be coming to Israel with two students - Scott Driscoll, who built Haile, and Travis Thatcher - and with Prof. Jason Freeman of Georgia Tech, who will guide the audience at The Lab and enable it to be a partner in the piece of music that will be played by means of waving light batons.
The audience will also surely notice that when Weinberg and Haile start drumming in a session that gets ever more frenetic, it is genuinely hard to say who is controlling whom. What is clear is that Weinberg is enjoying himself. And, at times, it appears that Haile is, too.W
Haaretz.com, the online edition of Haaretz Newspaper in Israel, offers real-time breaking news, opinions and analysis from Israel and the Middle East. Haaretz.com provides extensive and in-depth coverage of Israel, the Jewish World and the Middle East, including defense, diplomacy, the Arab-Israeli conflict, the peace process, Israeli politics, Jerusalem affairs, international relations, Iran, Iraq, Syria, Lebanon, the Palestinian Authority, the West Bank and the Gaza Strip, the Israeli business world and Jewish life in Israel and the Diaspora.
