Ji Hou (侯骥)

I am a Research Scientist at Meta Generative AI. Previously, I did my Ph.D. at TUM Visual Computing Group headed by Prof. Matthias Niessner, where I work on Computer Vision and 3D Scene Understanding. During my Ph.D., I did an internship at Facebook AI Research (FAIR) with Saining Xie and Benjamin Graham on 3D representation and data-efficient learning. Before that, I obtained my master at RWTH Computer Vision Group headed by Prof. Bastian Leibe, where I studied on Computer Vision and Machine Learning.

I am interested in research and applications on 3D Computer Vision, e.g. 3D Reconstruction, VR/AR, robotics and autonomous driving etc.

  Email  /    Google Scholar  /    Github  /    Twitter  /    Linkedin

nerf_det NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detection
Chenfeng Xu, Bichen Wu, Ji Hou *, Sam Tsai, Ruilong Li, Jialiang Wang, Wei Zhan Zijian He Peter Vajda Kurt Keutzer Masayoshi Tomizuka
International Conference on Computer Vision (ICCV), 2023
paper / bibtex / project / code
* Corresponding author.

NeRF-Det is a novel method for 3D detection with posed RGB images as input. Our method makes novel use of NeRF in an end-to-end manner to explicitly estimate 3D geometry, thereby improving 3D detection performance.

RoITr Rotation-Invariant Transformer for Point Cloud Matching
Hao Yu, Zheng Qin, Ji Hou, Mahdi Saleh, Dongsheng Li, Benjamin Busam, Slobodan Ilic
Computer Vision and Pattern Recognition (CVPR), 2023
paper / bibtex / code

We introduce RoITr, a Rotation-Invariant Transformer to cope with the pose variations in the point cloud matching task. On the challenging 3DLoMatch benchmark, RoITr surpasses the existing methods by at least 13 and 5 percentage points in terms of the Inlier Ratio and the Registration Recall, respectively

mask3d Mask3D: Pre-training 2D Vision Transformers by Learning Masked 3D Priors
Ji Hou, Xiaoliang Dai, Zijian He, Angela Dai, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2023
paper / video / bibtex

We demonstrate the Mask3D is particularly effective in embedding 3D priors into the powerful 2D ViT backbone, enabling improved representation learning for various scene understanding tasks, such as semantic segmentation, instance segmentation and object detection.

pcr_cg PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry
Yu Zhang, Junle Yu, Xiaolin Huang, Wenhui Zhou, Ji Hou
European Conference on Computer Vision (ECCV), 2022
paper / video / bibtex / poster / code

We introduce PCR-CG: a novel 3D point cloud registration module explicitly embedding the color signals into geometry representation. With our designed 2D-3D projection module, the pixel features in a square region centered at correspondences perceived from images are effectively correlated with point cloud representations.

pri3d Pri3D: Can 3D Priors Help 2D Representation Learning?
Ji Hou, Saining Xie, Benjamin Graham, Angela Dai, Matthias Nießner
International Conference on Computer Vision (ICCV), 2021
paper / video / bibtex / project / code

Recent advances in 3D perception have shown impressive progress in understanding geometric structures of 3D shapes and even scenes. Inspired by these advances in geometric understanding, we aim to imbue image-based perception with representations learned under geometric constraints.

panoptic3d Panoptic 3D Scene Reconstruction From a Single RGB Image
Manuel Dahnert, Ji Hou, Matthias Nießner Angela Dai
Advances in Neural Information Processing System (NeurIPS), 2021
paper / video / bibtex / project / code

Inspired by 2D panoptic segmentation, we propose to unify the tasks of geometric reconstruction, 3D semantic segmentation, and 3D instance segmentation into the task of panoptic 3D scene reconstruction -- from a single RGB image, predicting the complete geometric reconstruction of the scene in the camera frustum of the image, along with semantic and instance segmentations.

3dsis Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts
Ji Hou, Benjamin Graham, Matthias Nießner, Saining Xie
Computer Vision and Pattern Recognition (CVPR), 2021
(Oral Presentation)
paper / video / bibtex / project / code / benchmark

Our study reveals that exhaustive labelling of 3D point clouds might be unnecessary; and remarkably, on ScanNet, even using 0.1% of point labels, we still achieve 89% (instance segmentation) and 96% (semantic segmentation) of the baseline performance that uses full annotations.

3dsis RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction
Yinyu Nie, Ji Hou, Xiaoguang Han, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2021
paper / video / bibtex / code

In this work, we introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from raw point clouds. Instead of representing scenes with regular grids, our method leverages the sparsity of point cloud data and focuses on predicting shapes that are recognized with high objectness.

revealnet RevealNet: Seeing Behind Objects in RGB-D Scans
Ji Hou, Angela Dai, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2020
paper / video / bibtex / project

This paper introduces the task of semantic instance completion: from an incomplete, RGB-D scan of a scene, we detect the individual object instances comprising the scene and jointly infer their complete object geometry.

3dsis 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
Ji Hou, Angela Dai, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2019
(Oral Presentation)
paper / video / bibtex / project / code

We introduce 3D-SIS, a novel neural network architecture for 3D semantic instance segmentation in commodity RGB-D scans. The core idea of our method is to jointly learn from both geometric and color signal, thus enabling accurate instance predictions.

Teaching Assistant, Seminar for 3D Machine Learning - Summer 2021
Teaching Assistant, Advanced Deep Learning for Computer Vision - Winter 2019/20 Teaching Assistant, Introdcution to Deep Learning - Summer 2018
Teaching Assistant, Deep Learning for Computer Vision - Winter 2017/18
Review Experiences
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
ACM Transactions on Multimedia Computing Communications and Applications (TOMM)
International Journal of Computer Vision (IJCV)
ISPRS Journal of Photogrammetry and Remote Sensing
IEEE Robotics and Automation Letters (RA-L)
Pattern Recognition Letters
Computers & Graphics
Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH)
International Conference on Learning Representations (ICLR)
Association for the Advancement of Artificial Intelligence (AAAI)
Conference on Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
Neural Information Processing Systems (NeurIPS)
International Joint Conference on Artificial Intelligence (IJCAI)
The 2nd Computer Vision for Metaverse Workshop at ICCV 2023
Visiting Map

Credits: Jon Barron