Explore the research
Separating Shape and Reflectance from a Single Image
A self-supervised method estimating scene properties like reflectance and shape from a single photo using relighting constraints.
Recovering Polyp Shape and Scale Using a Medical Suture
A study using a known-diameter suture and a lighting model to estimate approximate polyp shape and absolute scale from one endoscope image.
Polyp Shape Recovery with Two Lights and Normal Correction
A study combining a two-light photometric model with learned normal correction to estimate depth and shape from endoscope imagery.
Physics-based computer vision models the behavior of light and solves inverse problems to recover 3D shape, reflectance, material properties, and illumination from images.
Our laboratory studies reflection analysis, gloss estimation, transparent-object recognition, polarization cues, and inverse rendering.
Why It Is Difficult
Real-world lighting is complex. Interreflections, subsurface scattering, anisotropic reflection, and mixed materials often violate simple image formation models.
The problem is often ill-posed, so uncertainty, ambiguity, and physically plausible constraints must be handled explicitly.
Approach
We study hybrid methods that embed physical constraints into deep learning and optimization-based methods using differentiable rendering.
The goal is to connect data-driven representations with image formation models that remain meaningful outside controlled datasets.
Evaluation
We combine quantitative evaluation on synthetic data with qualitative and consistency-based evaluation on real data.
When ground truth is hard to obtain, we use re-rendering consistency and physically motivated diagnostics.
Current Questions
- 1Reflectance estimation in outdoor or ambient-light environments
- 23D reconstruction including transparent and translucent objects
- 3Illumination estimation from a single image
Related publications (8)
Efficient 2D Human Pose Estimation via Token Compression and Cross-Attention Restoration
Studies efficient human pose estimation by compressing image tokens and reconstructing information through cross-attention.
Hashizume, Wataru, Usami, Hiroyasu
Proceedings of the 2026 Tokai-Section Joint Conference on Electrical, Electronics, Information, and Related Engineering, 2026 (8)
1C-3-3, p. 188, Computer Vision and Image Media, August 30, 2026
Reflectance Decomposition and 3D Shape Reconstruction via Self-Supervised Single-Image Inverse Rendering with Relighting Constraints
Uses relighting consistency during training to study shape and reflectance estimation from a single image.
Hara, Keisuke, Usami, Hiroyasu
Proceedings of the 60th Tokai Fuzzy Workshop in Toyohashi (Gamaken 2026), 2026 (8)
S3-03, pp. 15-16, Best Presentation Award, August 19, 2026
2D Human Pose Estimation via Compressed Token Learning with Structural Consistency
Studies human pose estimation with compressed image tokens while considering structural consistency.
Hashizume, Wataru, Usami, Hiroyasu
Meeting on Image Recognition and Understanding (MIRU 2026), 2026 (8)
IS3-171, Paper ID 366, Interactive Session IS3, 2026-08-06
Anatomy-Preserving Diffusion-Based Data Augmentation for Medical Image Segmentation
Studies medical segmentation data augmentation with anatomical controls and selection of generated images.
Tominari, Hiroki, Nakayama, Natsuki, Arakawa, Naoko, Usami, Hiroyasu
Proceedings of the 19th International Joint Conference on Biomedical Engineering Systems and Technologies -- BIOIMAGING, 2026
BIOIMAGING 2026
Inverse Rendering Using Single-view Polarization Cues and Shape Guidance Under Ambient Light
Studies inverse rendering under ambient light using single-view polarization cues and shape guidance.
Tsuboi, Ayato, Usami, Hiroyasu
Meeting on Image Recognition and Understanding (MIRU 2025), 2025
IS1-016
Research and Evaluation of Effective Prompts for Stable Diffusion-based AI Image Generation Tools
Studies prompt design and evaluation for Stable Diffusion-based image generation.
Tominari, Hiroki, Fukuoka, Reiya, Usami, Hiroyasu
21st Workshop on Informatics (WiNF 2023), 2023
S3C-4
Polyp Shape Recovery from Single Endoscope Image using Medical Suture
A study using a known-diameter suture and a lighting model to estimate approximate polyp shape and absolute scale from one endoscope image.
Usami, Hiroyasu, Iwahori, Yuji, Wang, Aili, Bhuyan, M. K., Ogasawara, Naotaka, Kasugai, Kunio
The Open Bioinformatics Journal, 2019 (1)
Vol. 12, No. 1, pp. 1–17
Recovering Polyp Shape from an Endoscope Image Using Two Light Sources
A study combining a two-light photometric model with learned normal correction to estimate depth and shape from endoscope imagery.
Usami, Hiroyasu, Iwahori, Yuji, Hanai, Yuki, Kijsirikul, Boonserm, Kasugai, Kunio
International Journal of Software Innovation, 2017 (4)
Vol. 5, No. 2, pp. 33–54