Ji Hou (侯骥)

I am a Research Scientist at Meta Reality Labs. Previously, I did my Ph.D. at TUM Visual Computing Group headed by Prof. Matthias Niessner, where I work on Computer Vision and 3D Scene Understanding. During my Ph.D., I did an internship at Facebook AI Research (FAIR) with Saining Xie and Benjamin Graham. Before that, I obtained my master at RWTH Computer Vision Group headed by Prof. Bastian Leibe, where I studied on Computer Vision and Machine Learning.

I am interested in research and applications on 3D Computer Vision, e.g. 3D Reconstruction, VR/AR, robotics and autonomous driving etc.

  Email  /    Google Scholar  /    Github  /    Twitter  /    Linkedin

Our team is hiring several interns based in Zurich or U.S. for Summer 2023 . If you have strong research background in 3D Scene Understanding, 3D Representation Learning or 3D Data-Efficient Learning, or know any such candidates, please send me an email!
pcr_cg PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometry
Yu Zhang, Junle Yu, Xiaolin Huang, Wenhui Zhou, Ji Hou
European Conference on Computer Vision (ECCV), 2022
paper / video / bibtex / poster / code

We introduce PCR-CG: a novel 3D point cloud registration module explicitly embedding the color signals into geometry representation. With our designed 2D-3D projection module, the pixel features in a square region centered at correspondences perceived from images are effectively correlated with point cloud representations.

pri3d Pri3D: Can 3D Priors Help 2D Representation Learning?
Ji Hou, Saining Xie, Benjamin Graham, Angela Dai, Matthias Nießner
International Conference on Computer Vision (ICCV), 2021
paper / video / bibtex / project / code

Recent advances in 3D perception have shown impressive progress in understanding geometric structures of 3D shapes and even scenes. Inspired by these advances in geometric understanding, we aim to imbue image-based perception with representations learned under geometric constraints.

panoptic3d Panoptic 3D Scene Reconstruction From a Single RGB Image
Manuel Dahnert, Ji Hou, Matthias Nießner Angela Dai
Advances in Neural Information Processing System (NeurIPS), 2021
paper / video / bibtex / project / code

Inspired by 2D panoptic segmentation, we propose to unify the tasks of geometric reconstruction, 3D semantic segmentation, and 3D instance segmentation into the task of panoptic 3D scene reconstruction -- from a single RGB image, predicting the complete geometric reconstruction of the scene in the camera frustum of the image, along with semantic and instance segmentations.

3dsis Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts
Ji Hou, Benjamin Graham, Matthias Nießner, Saining Xie
Computer Vision and Pattern Recognition (CVPR), 2021
(Oral Presentation)
paper / video / bibtex / project / code / benchmark

Our study reveals that exhaustive labelling of 3D point clouds might be unnecessary; and remarkably, on ScanNet, even using 0.1% of point labels, we still achieve 89% (instance segmentation) and 96% (semantic segmentation) of the baseline performance that uses full annotations.

3dsis RfD-Net: Point Scene Understanding by Semantic Instance Reconstruction
Yinyu Nie, Ji Hou, Xiaoguang Han, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2021
paper / video / bibtex / code

In this work, we introduce RfD-Net that jointly detects and reconstructs dense object surfaces directly from raw point clouds. Instead of representing scenes with regular grids, our method leverages the sparsity of point cloud data and focuses on predicting shapes that are recognized with high objectness.

revealnet RevealNet: Seeing Behind Objects in RGB-D Scans
Ji Hou, Angela Dai, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2020
paper / video / bibtex / project

This paper introduces the task of semantic instance completion: from an incomplete, RGB-D scan of a scene, we detect the individual object instances comprising the scene and jointly infer their complete object geometry.

3dsis 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
Ji Hou, Angela Dai, Matthias Nießner
Computer Vision and Pattern Recognition (CVPR), 2019
(Oral Presentation)
paper / video / bibtex / project / code

We introduce 3D-SIS, a novel neural network architecture for 3D semantic instance segmentation in commodity RGB-D scans. The core idea of our method is to jointly learn from both geometric and color signal, thus enabling accurate instance predictions.

Teaching Assistant, Seminar for 3D Machine Learning - Summer 2021
Teaching Assistant, Advanced Deep Learning for Computer Vision - Winter 2019/20 Teaching Assistant, Introdcution to Deep Learning - Summer 2018
Teaching Assistant, Deep Learning for Computer Vision - Winter 2017/18
Review Experiences
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
ACM Transactions on Multimedia Computing Communications and Applications (TOMM)
International Journal of Computer Vision (IJCV)
ISPRS Journal of Photogrammetry and Remote Sensing
IEEE Robotics and Automation Letters (RA-L)
Computers & Graphics
Special Interest Group on Computer Graphics and Interactive Techniques (SIGGRAPH)
Conference on Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
Visiting Map

Credits: Jon Barron