Computing and Music Spawn New Possibilities

In addition to pure, unadulterated coding, the beauty of computer science is its ability to seamlessly merge with and supplement many other disciplines. Interested in art? Breakthroughs in the computer graphics and animation have opened up new avenues of art by allowing artists to simulate dozens of textures, fluid materials like water, crowds, and hair. Have a flair for science? Many of today's latest developments in scientific disciplines like nanotechnology, biochemistry, and quantum physics owe thanks to the flexibility and freedom computing offers. Computing in science, art and medicine continue to attract attention and visibly impact everyday life. But then there are the less obvious applications of computing, and these less obvious applications have stirred up quite a revolutionary tune.

One such application bridges the gap between sound and sight to create, in a word, the ultimate electronic DJ table. The ReacTable, with its round, stylishly blue-lit surface and brightly-colored pucks, is what you could call a visual instrument. The pucks include sound generators, sound filters (which alter the sounds coming from generators), controllers, and global objects. By implementing a simple user interface, musicians can physically create music by adding, removing and adjusting pucks on the surface. Puck properties such as frequency, amplitude, and dry/wet level can be adjusted by either interacting with the object or by touching the surface around the object. The infinite possibilities the ReacTable offers easily fulfill the functions of the turn table and the synthesizer combined, and then some. The ability of each puck to interact with each other in infinite ways grants ReacTable a powerful flexibility that makes it a breakthrough in the hybrid field of music and computing.

Similar projects at the interdisciplinary GVU research center as well as the Center for Music Technology (GTCMT) have also advanced the possibilities of such field-merging. While some have mixed music with technology to create new ways to interact with music, others have used sound to demonstrate the visual. Researchers at Tech continue to refine, realize and revolutionize numerous other applications of the two fields.

Star performer in a harmonious duet of robotics and music is Haile, a robot percussionist who enjoys improvising tunes next to his fellow human musicians. Rather than just recognize and spit out what he has heard, Haile is capable of following tempo changes, adjusting pitches and discerning between chaotic and melodic sounds. It can listen to music in real-time, analyze it, and come up with its own take of what it just heard. The music-appreciative robot is the result of a collaboration between the colleges of Music Technology, Mechanical Engineering, Industrial Design, the Advanced Wood Products lab and the College of Computing, a medley of the many fields Tech offers, all conducted by music enthusiast Dr. Gil Weinberg. The robot and its unique, innovative means of interacting with music is a step forward in understanding the relationship humans have with music. Haile has currently mastered the drums and the xylophone, and has been joined by fellow robot musician Shimon, who specializes in marimba.

A reverse of ReacTable's treatment of the audible and the visible is the Accessible Aquarium, where sounds stand in for the visuals. Whereas the ReacTable brings in visual elements to enhance the audible experience, Accessible Aquarium brings in sound and music to compensate for a visual experience that sight-impaired people could not otherwise enjoy. Accessible Aquarium does what its name implies: it makes the aquarium an accessible commodity for the sight-impaired by painting a soundscape based on the movement and actions of life in the aquarium. The resulting blend of sound successfully recreates that peaceful, simple experience of watching small aquatic life twitch and flutter thin fins nonchalantly while interacting with each other and responding in pleasantly predictable manners to food and glass taps. The project brings together the fields of music and assistive technology, psychology and music to create a new way of seeing.

Another project created with the hearing impaired in mind is the SWAN Project, System for Wearable Audio Navigation. A joint collaboration between Psychology's Dr. Bruce Walker and the College of Computing's Frank Dellaert, the project uses an intricate tracking system in a portable device to determine the location of the user and direction the user faces. The information is then relayed to the user through bone conduction headphones known as "bonephones." The bonephones allow sound to transmit through the head without obstructing the ears, allowing the user to maintain access to surrounding sounds. An audible interface (rather than the usual visual one) transmits information to the user through specific sounds. Specific sounds include:
    -a navigation beacon, which guides the user from a starting point to the final destination.
    -object sounds, which indicate the location and type of surrounding objects.
    -surface transition, which alert the user to changes in walking surfaces.
    -locations, which differentiate between buildings, offices, bus stops, shops.
    -annotations, short messages recorded by the user about a particular environment's specifics.

The duet of computing within the field of music and sound has given birth to breakthroughs in our interaction with audio. It has spawned new innovative ways of creating music, a better understanding of music appreciation, and the use of sound to paint visual information. For the computing student who is musically inclined, there is plenty of ground left to break. The GVU center and the GTCMT continue to compose dynamically innovative applications with sound and code.