Kevin Vance - Tell me if this has already been thought of: Start with a video…

Entries | Archive | Friends | Friends' Friends | User Info

02:17 pm

Thursday, March 13th, 2008
Previous Entry Share Next Entry
Tell me if this has already been thought of:

Start with a video camera, and add:
  1. A laser rangefinder, stereo lenses, or other depth perceptor; and
  2. An accelerometer
Then for each video frame, you also have:
  1. A depth map; and
  2. Camera acceleration data
Load these data directly into your favorite 3D graphics program, and you get bluescreening, tracking, rotoscoping, etc. for free. There is going to be sensor drift from the accelerometer, but it wouldn't matter for short takes, and you could probably rectify that with traditional motion estimation from the video and depth frames. I'm not sure what the state of the art is in depth perception, but you should only need rough ideas of what is occluded by what.

Current Mood caffeinated
Link )Reply )

Comments
[User Picture]From: dariusk
2008-03-13 08:38 pm (UTC)
As a former machine vision guy: harder than it looks.
(Reply) (Thread)
[User Picture]From: kvance
2008-03-13 11:52 pm (UTC)
That's what I was afriad of. I took an undergrad course in computer vision, so I know the limitations of the state of the art (or at least how it was a couple years ago). But I was hoping for some kind of non-vision depth perceptor, like a scanning laser or radar.
(Reply) (Parent) (Thread)
[User Picture]From: dariusk
2008-03-14 02:26 pm (UTC)
There's a LOT of noise in those systems. Accelerometers, too--we did some game development with a non-Wii accelerometer, and wow was it basically impossible to filter for usable signal. You had dampen the thing so bad it had almost no fidelity. While I'm sure a more expensive one would have performed better, it's still hard to overcome the SNR.
(Reply) (Parent) (Thread)
[User Picture]From: gregw
2008-03-13 09:29 pm (UTC)

possibly

(Reply) (Thread)
[User Picture]From: kvance
2008-03-13 11:54 pm (UTC)

Re: possibly

Yes! I was thinking of the wiimote as a convenient sensor package, but I can't think of any way to get a depth map out of one at video speeds.
(Reply) (Parent) (Thread)
From: ex_md744
2008-03-15 03:34 am (UTC)

Re: possibly

What're you mapping anyway?
(Reply) (Parent) (Thread)
[User Picture]From: kvance
2008-03-15 04:03 pm (UTC)

Re: possibly

Just a thought experiment. If one could cobble something like this together, it would be a boon for amateur video production.
(Reply) (Parent) (Thread)