Ji Hou — World Models & Generative AI

Bio

I am a Principal Researcher at Tencent Hy, working on world models. Previously, a Staff Research Scientist at Meta Superintelligence Labs , working on foundation models. I did my Ph.D. at TUM Visual Computing Group headed by Prof. Matthias Niessner, where I work on Computer Vision and 3D Scene Understanding. During my Ph.D., I did an internship at Facebook AI Research (FAIR) with Prof. Saining Xie and Dr. Benjamin Graham on 3D representation learning.

Publications

Google Scholar ↗

2026

Exploring MLLM-Diffusion Information Transfer with MetaCanvas

H. Lin, X. Pan, Z. Huang, J. Hou, J. Wang, W. Chen, Z. He, F. Juefei-Xu, J. Sun, Z. Fan, A. Thabet, M. Bansal, C. Wang

NeurIPS, 2026

arxiv

Hy-Nomas: Towards Free-form Interactive World Generation

Core Contributor

Technical Report, 2026

paper / project page / code / models / benchmark / demo / bibtex

HairWeaver: Photorealistic Hair Motion Synthesis with Sim-to-Real Physics Transfer Guided Video Diffusion

D. Chang, J. Hou, A. Bozic, A. Neuberger, F. Juefei-Xu, O. Maury, G. W.-C. Lin, T. Stuyck, D. Roble, M. Soleymani, S. Grabli

European Conference on Computer Vision (ECCV), 2026

arxiv / project page / bibtex

PhyGDPO: Physics-Aware Groupwise Direct Preference Optimization for Physically Consistent Text-to-Video Generation

Y. Cai, K. Li, M. Jia, J. Wang, J. Sun, F. Liang, W. Chen, F. Juefei-Xu, C. Wang, A. Thabet, X. Dai, X. Ju, A. Yuille, J. Hou

European Conference on Computer Vision (ECCV), 2026

arxiv / project page / code / bibtex

ViTok-v2: Scaling Native Resolution Auto-Encoders to 5 Billion Parameters

P. Hansen-Estruch, J. Chen, V. Ramanujan, O. Zohar, Y. Ping, A. Sinha, M. Georgopoulos, E. Schoenfeld, J. Hou, F. Juefei-Xu, S. Vishwanath, A. Thabet

International Conference on Machine Learning (ICML), 2026

arxiv / project page / code / bibtex

Image Generation with a Sphere Encoder

K. Yue, M. Jia, J. Hou, T. Goldstein

arXiv, 2026

arxiv / project page / code / bibtex

2025

Transfer between Modalities with MetaQueries

X. Pan, S. N. Shukla^†, A. Singh, Z. Zhao, S. K. Mishra, J. Wang, Z. Xu, J. Chen, K. Li, F. Juefei-Xu, J. Hou^†, S. Xie^†

arxiv, 2025 ^†Equal advising

arxiv / project page / code / bibtex

MoCha: Towards Movie-Grade Talking Character Synthesis

C. Wei, B. Sun, H. Ma, J. Hou, F. Juefei-Xu, Z. He, X. Dai, L. Zhang, K. Li, T. Hou, A. Sinha, P. Vajda, W. Chen

Neural Information Processing Systems (NeurIPS), 2025 (Spotlight Presentation)

arxiv / project page / hf page / bibtex

Token-Shuffle: Towards High-Resolution Image Generation with Autoregressive Models

X. Ma, P. Sun, H. Ma, H. Tang, C.-Y. Ma, J. Wang, K. Li, X. Dai, Y. Shi, X. Ju, Y. Hu, A. Sanakoyeu, F. Juefei-Xu, J. Hou, J. Tian, T. Xu, T. Hou, Y.-C. Liu, Z. He, Z. He, M. Feiszli, P. Zhang, P. Vajda, S. Tsai, Y. Fu

arxiv, 2025

arxiv / project page / bibtex

LinGen: Towards High-Resolution Minute-Length Text-to-Video Generation with Linear Computational Complexity

H. Wang, C.-Y. Ma, Y.-C. Liu, J. Hou, T. Xu, J. Wang, F. Juefei-Xu, Y. Luo, P. Zhang, T. Hou, P. Vajda, N. K. Jha, X. Dai

Computer Vision and Pattern Recognition (CVPR), 2025

arxiv / project page / bibtex

Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Prompts

F. Liang, H. Ma, Z. He, T. Hou, J. Hou, K. Li, X. Dai, F. Juefei-Xu, S. Azadi, A. Sinha, P. Zhang, P. Vajda, D. Marculescu

Computer Vision and Pattern Recognition (CVPR), 2025

arxiv / project page / bibtex

The Llama 4 Herd: The Beginning of A New Era of Natively Multimodal AI Innovation

Contributor

Tech Report, 2025

arxiv / blog / bibtex

2024

Movie Gen: A Cast of Media Foundation Model

Core Contributor

Technical Report, 2024

arxiv / project / bibtex

ControlRoom3D: Room Generation using Semantic Proxy Rooms

J. Schult, S. Tsai, L. Höllein, B. Wu, J. Wang, C.-Y. Ma, K. Li, X. Wang, F. Wimbauer, Z. He, P. Zhang, B. Leibe, P. Vajda, J. Hou

Computer Vision and Pattern Recognition (CVPR), 2024

arxiv / project page / video / bibtex

Cache Me if You Can: Accelerating Diffusion Models through Block Caching

F. Wimbauer, B. Wu, E. Schoenfeld, X. Dai, J. Hou, Z. He, A. Sanakoyeu, P. Zhang, S. Tsai, J. Kohler, C. Rupprecht, D. Cremers, P. Vajda, J. Wang

Computer Vision and Pattern Recognition (CVPR), 2024

arxiv / project page / bibtex

RIGA: Rotation-Invariant and Globally-Aware Descriptors for Point Cloud Registration

H. Yu, J. Hou, Z. Qin, M. Saleh, I. Shugurov, K. Wang, B. Busam, S. Ilic

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024

paper / arxiv / bibtex

2023

Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystack

X. Dai^*, J. Hou^*, C. Y. Ma^*, S. Tsai^*, J. Wang^*, R. Wang^*, P. Zhang^*, S. Vandenhende, X. Wang, A. Dubey, M. Yu, A. Kadian, F. Radenovic, D. Mahajan, K. Li, Y. Zhao, V. Petrovic, M. K. Singh, S. Motwani, Y. Wen, Y. Song, R. Sumbaly⁺, V. Ramanathan⁺, Z. He⁺, P. Vajda⁺, D. Parikh⁺

Meta AI Tech Report, 2023. * Equal contribution, alphabetical order. + Joint last authors.

arxiv / project / bibtex

NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detection

C. Xu, B. Wu, J. Hou^*, S. Tsai, R. Li, J. Wang, W. Zhan, Z. He, P. Vajda, K. Keutzer, M. Tomizuka

International Conference on Computer Vision (ICCV), 2023 * Corresponding author

paper / bibtex / project / code

Rotation-Invariant Transformer for Point Cloud Matching

H. Yu, Z. Qin, J. Hou, M. Saleh, D. Li, B. Busam, S. Ilic

Computer Vision and Pattern Recognition (CVPR), 2023

paper / bibtex / code

Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors

J. Hou, X. Dai, Z. He, A. Dai, M. Nießner

Computer Vision and Pattern Recognition (CVPR), 2023

paper / video / bibtex

2022

PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry

Y. Zhang, J. Yu, X. Huang, W. Zhou, J. Hou

European Conference on Computer Vision (ECCV), 2022

paper / video / bibtex / poster / code

2021

Pri3D: Can 3D Priors Help 2D Representation Learning?

J. Hou, S. Xie, B. Graham, A. Dai, M. Nießner

International Conference on Computer Vision (ICCV), 2021

paper / video / bibtex / project / code

Panoptic 3D Scene Reconstruction From a Single RGB Image

M. Dahnert, J. Hou, M. Nießner, A. Dai

Advances in Neural Information Processing System (NeurIPS), 2021

paper / video / bibtex / project / code

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts

J. Hou, B. Graham, M. Nießner, S. Xie

Computer Vision and Pattern Recognition (CVPR), 2021 (Oral Presentation)

paper / video / bibtex / project / code / benchmark

RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction

Y. Nie, J. Hou, X. Han, M. Nießner

Computer Vision and Pattern Recognition (CVPR), 2021

paper / video / bibtex / code

2020

RevealNet: Seeing Behind Objects in RGB-D Scans

J. Hou, A. Dai, M. Nießner

Computer Vision and Pattern Recognition (CVPR), 2020

paper / video / bibtex / project

2019

3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans

J. Hou, A. Dai, M. Nießner

Computer Vision and Pattern Recognition (CVPR), 2019 (Oral Presentation)

paper / video / bibtex / project / code

Teaching

Teaching Assistant, Seminar for 3D Machine Learning - Summer 2021
Teaching Assistant, Advanced Deep Learning for Computer Vision - Winter 2019/20
Teaching Assistant, Introdcution to Deep Learning - Summer 2018
Teaching Assistant, Deep Learning for Computer Vision - Winter 2017/18

Review Experiences

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
ACM Transactions on Multimedia Computing Communications and Applications (TOMM)
International Journal of Computer Vision (IJCV)
ISPRS Journal of Photogrammetry and Remote Sensing
IEEE Robotics and Automation Letters (RA-L)
IEEE Transactions on Image Processing (TIP)
Pattern Recognition Letters
Neurocomputing
Computers & Graphics
Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH)
Conference on Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
International Conference on Machine Learning (ICML)
Neural Information Processing Systems (NeurIPS)
International Conference on Learning Representations (ICLR)
International Conference on Robotics and Automation (ICRA)
Association for the Advancement of Artificial Intelligence (AAAI)
International Joint Conference on Artificial Intelligence (IJCAI)
ACM Multimedia (ACMMM)

Organizer

2nd GenAI Media Generation Challenge Workshop at CVPR 2025
The 4th Computer Vision for Metaverse Workshop at CVPR 2025
GenAI Media Generation Challenge Workshop at CVPR 2024
Efficient Deep Learning for Computer Vision Workshop at CVPR 2024
The 2nd Computer Vision for Metaverse Workshop at ICCV 2023