Jiayi Wu

Jiayi Wu

University of Maryland, College Park

👤 Short Bio

Welcome to my academic homepage. I am Jiayi Wu, a 2nd-year Ph.D. student supervised by Prof. Yiannis Aloimonos at PRG (Perception & Robotics Group) Lab, University of Maryland, College Park. My research focuses on 3D vision and robotics, specifically in the areas of 3D/4D Generation, Active Vision, and Field Robotics (especially Underwater Robotics). Inspired by nature, I am dedicated to creating innovative world generator & simulator and robot perception systems that challenge existing paradigms and redefine the future of 3D vision and robotics.

Before that, I worked with Prof. Md Jahidul Islam and completed my Master’s thesis under his supervision.

My research interest includes:

  • 3D/4D Generation: Controllable real world generation
  • General Environment Vision: Make computer vision systems more robust to the environment (scattering media, bad weather)
  • Active Vision: Feedback control through vision

📄Latest CV’s link

📢 News

  • 2022/08/23: Completed the summer internship at Vobile and got a return offer (recommended by Dr. Zhao).

  • 2022/05/23: Joined the team of Dr. Zhao (CTO of Vobile) of Vobile as an audio and video algorithm development engineer (summer internship).

  • 2022/01/10: Join Professor Judge’s Remote Sensing Lab (as Graduate Student Assistant) and be responsible for the development of a large-scale automated generation of 3D plant models.

🎓 Educations

University of Maryland, College Park

University of Florida

Zhejiang Sci-Tech University (ZSTU)

  • B.E. in Mechatronic Engineering ——— Sept. 2017- Jun. 2021
    2021 Outstanding Graduate, Zhejiang Sci-Tech University

📝 Publications

Conference Papers

Under Review
sym
  • Dehao Yuan, Levi Burner, Jiayi Wu, Minghui Liu, Jingxi Chen, Yiannis Aloimonos, Cornelia Fermüller. Learning Normal Flow Directly From Event Neighborhoods. Under Review
    [Arxiv] [Code] [Video demo]
CoRL 2024
sym
  • Xiong, Tianyi*, Wu, Jiayi*, Botao He, Cornelia Fermuller, Yiannis Aloimonos, Heng Huang, and Christopher Metzler. Event3DGS: Event-based 3D Gaussian Splatting for High-speed Robot Egomotion. CoRL 2024
    [Project Page] [Paper] [Video demo]
IROS 2024
sym
  • Wu, Jiayi*, Lin, Xiaomin*, Shahriar Negahdaripour, Cornelia Fermüller, and Yiannis Aloimonos. MARVIS: Motion & Geometry Aware Real and Virtual Image Segmentation. IEEE IROS 2024
    [Arxiv] [Poster]
CAI 2023 (Best Paper Award)
sym
  • Wu, Jiayi, Yu, Boxiao, Islam, Md Jahidul. 3D Reconstruction of Underwater Scenes using Nonlinear Domain Projection. IEEE CAI 2023 (Best Paper Award)
    [IEEE Xplore] [Poster] [Video demo]
ICRA 2023
sym
  • A. Kaleo Roberts, Kamal Sarabandi, Jasmeet Judge, Alejandro Monsivais-Huertero, Jiayi Wu. VALIDATION OF A FULL-WAVE BACKSCATTER MODEL FOR CORN FIELDS USING MEASUREMENTS FROM A GROUND-BASED SCATTEROMETER. IGARSS 2023

Journals and Thesis

RA-L
sym
  • Md Abu Bakr Siddique, Jiayi Wu, Ioannis Rekleitis, and Md Jahidul Islam. AquaFuse: Waterbody Fusion for Physics Guided View Synthesis of Underwater Scenes. RA-L
    [Project Page] [Arxiv] [Video demo]

  • Wu, Jiayi. Low-Cost Depth Estimation and 3D Reconstruction in Scattering Medium. Master’s Thesis. 2023
    [UFDC]

  • A. K. Roberts, J. Wu, A. Monsivais-Huertero, J. Judge, R. C. Moore and K. Sarabandi, Microwave Backscatter Phenomenology of Corn Fields at L-Band Using a Full-Wave Electromagnetic Solver, IEEE Transactions on Geoscience and Remote Sensing, vol. 62, pp. 1-11, 2024, Art no. 2000511, doi: 10.1109/TGRS.2023.3340198.
    [IEEE Xplore]

  • Kaiser Niknam, Jasmeet Judge, A. Kaleo Roberts, Alejandro Monsivais-Huertero, Robert Moore, Kamal Sarabandi, Jiayi Wu, A 3-D Full-Wave Model to Study the Impact of Soybean Components and Structure on L-Band Backscatter, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, doi: 10.1109/JSTARS.2024.3426366.
    [IEEE Xplore]

📖 Patents

📃 Projects

A Collaborative Project
sym

UDepth

  • A monocular underwater depth estimation pipeline, using RMI as the input space, constructs a lightweight domain projection module, a lightweight CNN feature extraction module, and a lightweight Transformer-based depth estimation network. And the depth estimation network is supervised with knowledge of underwater light attenuation as a prior. The results show that our depth estimation accuracy is close to the SOTA model, and much faster than them.
    This is a collaborative project on underwater depth estimation with Boxiao Yu, a Ph.D. student in RoboPI lab.
    [IEEE Xplore] [arXiv] [Code] [pre-print]
Master's Thesis
sym

SDU-SfM

  • A fast depth-guided semi-dense underwater 3D reconstruction pipeline without a deep learning model. Underwater image restoration is implemented through RMI channel and underwater imaging model to improve the number of feature matches. Reduce the computational complexity of feature extraction by using the depth map as a mask. In order to achieve higher quality(semi-dense) underwater 3D reconstruction, the depth maps are used as the supervision to refine the 3D point cloud and remove noise.
    This project is my master’s thesis, we have submitted a journal to the IEEE Transactions on Artificial Intelligence (TAI). Here is the video demo.
R&D Internship Project
sym

Learning-based Infringing Video Retrieval

  • An learning-based infringing video retrieval system based on the fusion of global features and local features. I did a summer internship in the Vobile’s R&D department under the supervision of Dr. Zhao, CTO of Vobile. I built a learning-based infringing video retrieval system based on the fusion of global features and local features. I used Vobile’s video database to train the model, and achieved good performance. Before I ended my internship, I put it together into a python package and organize each component in the system into an easy-to-use python toolkit. I also wrote the manual of the package and a document about future optimization steps to finally handed over to the person who took over the project.

🏭 Job Experience

  • Research Assistant ——— May. 2024- Present
    University of Maryland, College Park, MD, United States.

  • Teaching Assistant ——— Aug. 2023- May. 2024
    University of Maryland, College Park, MD, United States.

  • Digital Audio and Video Algorithm Engineer ——— May. 2022- Aug. 2022
    Vobile, Santa Clara, CA, United States.

  • Graduate Student Assistant ——— Jan. 2022- Jan. 2023
    Remote Sensing Laboratory at University of Florida, Gainesville, FL, United States.

🏅 Honors and Awards

SCHOLARSHIPS

  • 2024 UMIACS Fellowship
  • 2022/2021 UF Herbert Wertheim College of Engineering Engineering Achieve Award
  • 2019 First Class School Financial Aid for Overseas Exchange Program

COMPETITIONS

  • 2021.06 Individual 1st Prize in the National University Graduate Design Competition (Only two people won this award nationwide)
  • 2019.10 Provincial 1st Prize of National 3D Digital Innovative Design Competition
  • 2019.09 2nd Prize of National 3dds Competition Classic
  • 2019.06 3rd Prize of The 16th Zhejiang Province Mechanical Design Competition for College Student
  • 2019.04 3rd Prize of The Challenge Cup Extracurricular Academic Works Competition

💬 Academic Services

Reviewer

  • 2024 International Conference on Learning Representations (ICLR)
  • 2024 Conference on Neural Information Processing Systems (NeurIPS)
  • 2023/2024 IEEE International Conference on Robotics and Automation (ICRA)
  • 2023/2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
  • IEEE Robotics and Automation Letters (RAL)
  • IEEE Journal of Oceanic Engineering (JOE)