Thomas Wimmer

@wimmerthomas.bsky.social

408 Followers 139 Following

PhD Candidate at the Max Planck ETH Center for Learning Systems working on 3D Computer Vision. https://wimmerth.github.io

Improvement of multi-view consistency of generated videos through latent interpolation. In addition to the rendering of the dynamic scene f, using the rendering function g from the current viewpoint g(f)_s, we compute the latent embedding of the warped video output v_{s-1} of the previous optimization step (from a different viewpoint). We linearly interpolate the latents before passing them through the video diffusion model (VDM), which is additionally conditioned on the static scene view from the current viewpoint. The resulting output is finally decoded to a new video output v_s.
Method overview for lifting 2D dynamics into 3D. Pre-trained models are shown in blue. We detect 2D point tracks and use aligned estimated depth values to lift them into 3D. The 4D (dynamic 3D) Gaussians are initialized with the static 3D scene input.