When Jay LeBoeuf was 10, he begged his parents to buy him an electric keyboard so he could plug it into his Apple IIGS computer and play ’80s synth rock.
Now LeBoeuf, an electrical engineer with a background in classical piano and a master’s degree from Stanford’s Center for Computer Research in Music and Acoustics, wants to turn computers into virtual sound engineers.
“A big motivation for me is that computers and mobile devices right now, they’re deaf,” he says. “They have no idea what’s going on.”
In 2008, LeBoeuf, 34, founded Imagine Research, a San Francisco startup that introduced its MediaMined software this month. The software enables music and film producers to search, classify and retrieve all manner of audio files. The software currently recognizes 400 distinct sounds, including instruments, voices, and noises like a dog’s bark and the crack of a baseball bat.
LeBoeuf is working on expanding that vocabulary and plans to roll out a feature early next year that marks where in the audio file each distinct sound occurs. On an episode of, say, “American Idol,” “We can determine where the judges are speaking, when there’s music, and when there’s applause,” he says. That can be used “to do intelligent summaries of a performance or allow you to skip to sections that have music.”
Once audio files have been indexed by the software, sound engineers can browse through digital movie archives and find, say, all the gunshots and explosions. MediaMined also allows recording artists to find a wide range of bass lines or drum tracks that might complement a guitar riff for their songs.
Lucasfilm is one of six companies that signed on as early partners to help Imagine Research beta-test its software.
“We’ve indexed their internal media archives,” LeBoeuf says. “It gives their sound editors a really unique tool to do Google-like audio searches within their collection.”
Other partners include the Internet Archive, a nonprofit online library, and Gobbler, a music backup service.
Building MediaMined, a process funded in large part by a $600,000 grant from the National Science Foundation, took three years. The first six months were spent uploading and labeling roughly half a million audio files. Next, LeBoeuf wrote computer software to comb through those files and identify which sonic patterns – loudness, dynamic range and pitch, for example – are characteristic of sounds like snare drums and human laughter. Finally, LeBoeuf’s team of eight enabled the software to recognize these patterns in any digital audio archive.
LeBoeuf, who still plays music in his spare time, is surprised no one before him thought to commercialize audio search software.
“I kept pondering why Google wasn’t doing this, why Pandora employed humans to listen and label music,” he says.
The question puzzled him enough to quit what he calls a dream job as a research engineer at the audio tech company Avid Technology.
Source:http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2011/11/27/BUE91M3L7A.DTL
