Visual Microphone Turns Video into Sound

Researchers from MIT, Microsoft and Adobe have developed a "visual microphone" that can recover audio signals from vibrations of objects in a video. "When sound hits an object, it causes the object to vibrate," said Abe Davis, a Ph.D. candiate at MIT and first author of the new paper [1]. "The motion of this vibration creates a very subtle visual signal that's usually invisible to the naked eye. People didn't realize that this information was there." Using just a high-speed camera that records thousands of frames per second (fps), the system was able to reconstruct music from the leaves of a potted plant, and speech from a potato chip bag. Furthermore, the team has shown that sound can also be recovered using regular consumer cameras by taking advantage of the "rolling-shutter" effect. Visit "The Visual Microphone" project webpage  for more details. [1] Davis, Abe, Michael Rubinstein, Neal Wadhwa, Gautham J. Mysore, Frédo Durand, and William T. Freeman. "The visual microphone: passive recovery of sound from video." ACM Transactions on Graphics (TOG) 33, no. 4 (2014): 79.