Light of Normals: Unified Feature Representation for Universal Photometric Stereo

Hong Li*, Houyuan Chen*, Chongjie Ye$^\dagger$
Zhaoxi Chen, Bohan Li, Shaocong Xu,Xianda Guo
Xuhui Liu, Yikai Wang, Baochang Zhang, Satoshi Ikehata, Boxin Shi, Anyi Rao, Hao Zhao$^\ddagger$
* Equal contribution $^\dagger$ Part of project lead $^\ddagger$ Corresponding author

📊 Overview

LINO-UniPS is a method for Univeral Photometric Stereo (PS). It predicts the normal map from a given set of images. Key features include:

• Light-Agnostic: Does not require specific lighting parameters as input.
• Arbitrary-Resolution: Supports inputs of any resolution.
• Mask-Free: Also supports mask-free scene normal reconstruction.

✨ News

• November 2025 - Update the code for PBR material estimation.
• August 2025 - Update the eval code for a new benchmark -- DIR. We're still SOTA!!!
• August 2025 - Release training code.
• June 2025 - Release paper, project page, Hugging Face demo and model checkpoint.

📝 TODO List

• Release checkpoint and inference code.
• Release PS-Verse, the training dataset.
• Release training code.
• Adding support for PBR material estimation.

🔛 Get Started

💾 Installation

git clone https://github.com/houyuanchen111/lino.git
cd lino
conda create -n LINO python=3.10
conda activate LINO
pip install -r requirements.txt

📝 Dataset Preparation

Download the following benchmarks for evaluate our LiNO-UniPS.

• DiLiGenT: A widely-used benchmark for photometric stereo, consisting of 10 real-world objects with diverse shapes and materials under precisely calibrated directional lighting.
• LUCES: A challenging benchmark focused on complex material properties, captured under thousands of high-frequency lighting conditions from a screen-based setup.
• DiLiGent10²: The large-scale successor to DiLiGenT, featuring 100 real-world objects with more complex geometries and materials to better evaluate state-of-the-art methods.
• DIR: DIR is a dataset for Display Inverse Rendering (DIR). It contains assets captured from LCD & polarization-camera system, which is the latest benchmark for the Photometric Stereo After downloading, place them under data/ as the folloing directory tree.

|-- data
  DiLiGenT 
  ├── ballPNG
  └── ...
  LUCES 
  ├── BALL
  └── ...
  DiLiGenT_100 
  ├── BALL_ABS
  └── ...
  DIR_pms 
  ├── 01_ElephantPNG
  └── ...

🧑‍🏫 Evaluating on Benchmarks

To evaluate the performance of LiNO-UniPS on the DiLiGenT, LUCES, and DiLiGenT10² benchmarks, run the corresponding scripts below. The pretrained model weights will be downloaded automatically on the first run of eval.py.

# DiLiGenT
python eval.py --task_name DiLiGenT --data_root data/DiLiGenT/ --num_images 16 

# LUCES
python eval.py --task_name LUCES --data_root data/LUCES/ --num_images 16 

# DiLIGenT10²
python eval.py --task_name DiLiGenT_100 --data_root data/DiLiGenT_100/ --num_images 16

# DIR
python eval.py --task_name DIR --data_root data/DIR_pms/ --num_images 64

The evaluation results will be stored in the output directory.

Note for DiLiGenT10²: The ground truth normals for this benchmark are withheld for official evaluation. To get your score, you must submit the predicted results, which are saved in .mat format inside the submit directory, to the official evaluation server.

The performance of our released model exceeds the results originally reported in our paper. For this comparison, the results for all competing Universal PS methods (UniPS, SDM-UniPS, Uni MS-PS) are the best scores as reported by the Uni MS-PS.

Table 1: DiLiGenT. Method Average MAE↓ UniPS 14.7 SDM-UniPS 5.80 Uni MS-PS 4.97 LINO-UniPS (Paper) 4.94 LINO-UniPS (Release,K=96) 4.74

Table 2: LUCES. Method Average MAE↓ UniPS 23.77 SDM-UniPS 13.50 Uni MS-PS 11.10 LINO-UniPS (Paper) 10.69 LINO-UniPS (Release,K=32) 9.47

Table 3: DiLiGenT10². Method Average MAE↓ UniPS 23.79 SDM-UniPS 14.97 Uni MS-PS 13.20 LINO-UniPS (Paper) 12.21 LINO-UniPS (Release,K=64) 11.12

Normal reconstruction result of PlasterPNG on DIR benchmark. The MAE improves by 41.8% compared to the reported SOTA in the paper.

🧑🏼‍🏫 Evaluating on Real-World Data (In-the-Wild)

LINO-UniPS also demonstrates robust performance on multi-light images captured in-the-wild. In the data/Real_data/ directory, we provide two real-world scenes captured using a unique two-iPhone setup, where one device was used for recording and the other as a movable light source.

For more real-world data captured by camera, you can download from UniPS, SDM-UniPS.

After downloading, please place the extracted folders under the data/Real_data/ directory to match the following structure:

|-- data
  |-- Real_data
  A 
  ├── mask.png (optional)
  ├── [Prefix (default: "L")] imgfile1
  ├── [Prefix (default: "L")] imgfile2
  └── ...
  B 
  ├── mask.png (optional)
  ├── [Prefix (default: "L")] imgfile1
  ├── [Prefix (default: "L")] imgfile2
  └── ...

Then, run the following scripts:

# Real_data. If the value provided for --num_images exceeds the total number of available images, the maximum available number will be used instead.

python eval.py --task_name Real --data_root data/Real_data/ --num_images 12 

OLAT results correspond to issue #2. The data is sourced from the OpenIllumination, with a resolution of 3000x4096.

Captured by iPhone		Captured by Camera (provided by UniPS, SDM-UniPS)		OLAT (OpenIllumination)

🚀 Gradio demo

This gradio demo is a GUI interface that requires no expert knowledge, suitable for general users. To run the demo, first download the required data from this link. Then, unzip the archive and place its contents into the demo/ directory.

Simply run:

python app.py

You can also use it online. Click here.

📱 'Photometric Stereo' Your Own Data

Here, we provide a simple way to capture multi-light images.

Step 1. Prepare

What you need to prepare is:

• Camera: A smartphone camera is perfectly sufficient, though a dedicated digital camera will also work.
• Tripod: To hold your phone or camera steady and motionless.
• Light: Smartphone's flashlight is enough, others can also be used.
• Subject: The object or scene you want to 'Photometric Stereo'.

Step 2. Take Photos

Keep the camera and the subject stationary. Capture multiple images by moving the light source to a different position for each shot. For best results, we recommend capturing 4 or more images.

Step 3. Process and Predict

Organize your captured images into a new folder and place it inside the data/Real_data/ directory. The images should be saved in .jpg or .png format. For example:

|-- data
  |-- Real_data
    Your_Object_Name 
    ├── mask.png (optional)
    ├── L_1.jpg
    ├── L_2.jpg
    └── ...

mask.png is optional. If needed, it can be generated using RMBG-2.0.

💟 Downstream application

This is for issue #3. The high-quality normals predicted by LINO-UniPS can be utilized as normal bridge for Hi3DGen to generate a high-quality mesh. Compared to vanilla Hi3DGen, the superior quality of the LINO-UniPS normals results in a generated mesh with substantially higher fidelity and richer details. For more results, please refer to the Home page and try it out at Live Demo.

RGB	Hi3DGen	LINO-UniPS
Mesh 1: The result of vanilla Hi3DGen. Vertices: 216308, Faces: 432602 Mesh 2: The result of Hi3DGen using the normal of LINO-UniPS. Vertices: 362438, Faces: 725006	Mesh 1	Mesh 2

🔖 Citing

If you find this repository useful, please consider giving a star ⭐ and citation.

@article{li2025lightnormalsunifiedfeature,
      title={Light of Normals: Unified Feature Representation for Universal Photometric Stereo}, 
      author={Hong Li and Houyuan Chen and Chongjie Ye and Zhaoxi Chen and Bohan Li and Shaocong Xu and Xianda Guo and Xuhui Liu and Yikai Wang and Baochang Zhang and Satoshi Ikehata and Boxin Shi and Anyi Rao and Hao Zhao},
      journal={arXiv preprint arXiv:2506.18882},
      year={2025}
}