Abe Davis: New video technology that reveals an object's hidden properties

[Denis]

Hey guys, i don't know if you've watched this video and the related topic before, and i couldn't find it by using the search engine so i've decided to share it here with you. It is about the software developed by computer scientist Abe Davis and his team that allows one to detect sounds from silent and still videos.
Thought it might make a good article on SOTT website.

Subtle motion happens around us all the time, including tiny vibrations caused by sound. New technology shows that we can pick up on these vibrations and actually re-create sound and conversations just from a video of a seemingly still object. But now Abe Davis takes it one step further: Watch him demo software that lets anyone interact with these hidden properties, just from a simple video.

http://www.ted.com/talks/abe_davis_new_video_technology_that_reveals_an_object_s_hidden_properties#

edit: added words for clarity

 

[trendsetter37]

Wow. I am kind of speechless . Thanks for sharing this Denis that was really cool!

 

[Deleted member 9511]

Imagine taking all the movements of thousandths-of-a-pixel bits of an image of the entire universe, and putting all those vibrations together to recover the sound that was causing all of those vibrations. What would we hear?

This garden universe vibrates complete.
Some we get a sound so sweet.
Vibrations reach on up to become light,
And then thru gamma, out of sight.
Between the eyes and ears there lay,
The sounds of colour and the light of a sigh.
And to hear the sun, what a thing to believe.
But it's all around if we could but perceive.
To know ultra-violet, infra-red and X-rays,
Beauty to find in so many ways.
Two notes of the chord, that's our fluoroscope.
But to reach the chord is our lifes hope.
And to name the chord is important to some.
So they give a word, and the word is OM.

"The Word" by The Moody Blues

 

[Palinurus]

Yes, it's quite stunning and I recall having seen this before, so I did a search. Turns out the first reports about this tour de force are from early August 2014.

A few examples:

_http://www.techtimes.com/articles/12211/20140805/eavesdropping-via-potato-chips-yes-it-can-happen.htm
_http://www.newscientist.com/article/dn25999-caught-on-tape-cameras-turn-video-into-sound.html#.VW9G7Ubysxk
_http://io9.com/scientists-recover-speech-from-silent-video-of-a-vibrat-1617158704
_http://www.washingtonpost.com/news/speaking-of-science/wp/2014/08/04/mit-researchers-can-listen-to-your-conversation-by-watching-your-potato-chip-bag/
_http://rt.com/usa/177940-mit-researchers-eavesdropping-chips-bag/

Thanks for sharing anyway. It bears repeating. ;)

 

[MusicMan]

Hi Denis, my outdated technology won't let me play it, but I understand the principle:
it's the same as the spies who will beam a light ray (or infrared beam) off your window and pick up the reflected beam, to receive the modulated waves of the voices and other sounds inside.
It's old technology, I first saw it demonstrated in 1965.

With the old filament lamps, voices or sounds would vibrate the filament, and this could be picked up as well, from reflections off your walls.
You might note that reflections off your walls from your computer screen will also be detectable and decipherable with the right technology, which picks up the CRT beam as it rasters across.

I saw an episode of mythbusters in which they were trying to get voice/sounds off an 'ancient' fired clay urn, they weren't very successful, but I'm sure it could be done, it's the same principle that was used in the original phonograph, 'His Master's Voice' sort of thing.

 

[Ðekel]

This is pretty amazing !

In a way it is a kind of technological synesthesia. The algorithm enables hearing of visual movement.

We're able to "hear" visuals, as well as extend our sense of touch in a way that is visual (in the case of the seeing the person's pulse and magnifying it which could only have been done via touch before).

 

[Denis]

Hi Denis, my outdated technology won't let me play it

Hello MusicMan, can you see the video now?

http://<iframe width="560" height="...Po" frameborder="0" allowfullscreen></iframe>

but I understand the principle:
it's the same as the spies who will beam a light ray (or infrared beam) off your window and pick up the reflected beam, to receive the modulated waves of the voices and other sounds inside.
It's old technology, I first saw it demonstrated in 1965.

With the old filament lamps, voices or sounds would vibrate the filament, and this could be picked up as well, from reflections off your walls.
You might note that reflections off your walls from your computer screen will also be detectable and decipherable with the right technology, which picks up the CRT beam as it rasters across.

I saw an episode of mythbusters in which they were trying to get voice/sounds off an 'ancient' fired clay urn, they weren't very successful, but I'm sure it could be done, it's the same principle that was used in the original phonograph, 'His Master's Voice' sort of thing.

Thank you for your comment. I think this is all amazing technology! :)

 

[Andrian]

Amasing thanks for sharing, but yeah i would agree with MusicMan i think it's not something new, i remember watching the movie Eagle Eye(made in 2008) and there was a scene when a group o people have entered in a sound proof room to not be heard by the "evil" artificial intelligence supercomputer and the computer basically has figured out what they were talking about by watching through the glass window and analasing the vibrations caused by their voices in a glass of water, the science is awsome ;D.

 

[Denis]

Amasing thanks for sharing, but yeah i would agree with MusicMan i think it's not something new, i remember watching the movie Eagle Eye(made in 2008) and there was a scene when a group o people have entered in a sound proof room to not be heard by the "evil" artificial intelligence supercomputer and the computer basically has figured out what they were talking about by watching through the glass window and analasing the vibrations caused by their voices in a glass of water, the science is awsome ;D.

Hi Andre',

I also watched that movie, albeit only once and it was in 2009, but i don't remember that specific scene - maybe due to the fact that 6 years have passed since then. Or perhaps my hippocampus doesn't serve me so well as it used to. No time like today to refresh my memory. ;)
Although, the fact that the technology was displayed in the movie doesn't automatically prove its existence in real life, at the time of its release (2008). Then again, who knows?

 

[Denis]

Yes, it's quite stunning and I recall having seen this before, so I did a search. Turns out the first reports about this tour de force are from early August 2014.

A few examples:

_http://www.techtimes.com/articles/12211/20140805/eavesdropping-via-potato-chips-yes-it-can-happen.htm
_http://www.newscientist.com/article/dn25999-caught-on-tape-cameras-turn-video-into-sound.html#.VW9G7Ubysxk
_http://io9.com/scientists-recover-speech-from-silent-video-of-a-vibrat-1617158704
_http://www.washingtonpost.com/news/speaking-of-science/wp/2014/08/04/mit-researchers-can-listen-to-your-conversation-by-watching-your-potato-chip-bag/
_http://rt.com/usa/177940-mit-researchers-eavesdropping-chips-bag/

Thanks for sharing anyway. It bears repeating. ;)

Cool, thanks for sharing Palinurus. When i said that i couldn't find it, i meant that i couldn't find it on the forum's search engine, not on the internet in general. Sorry for not being clear enough in my initial post.