Thank you for visiting my theme! Replace this with your message to visitors.

You are here: Home / research

3D Reconstruction and Representation

3D reconstruction is one of fundmental tasks in computer vision. One can reconstruct geometric structure of real wold scene by estimating the depth and poses of cameras, which provides infrastructural data for object detection, segmentation and background modeling.

All 2020 2019 2018 2017 2016 More...

Depth estimation and optimization from multiple depth cues based on camera array [Finished]

PI: Qing Wang | Code: 61272287 | Support: NSFC | Start/End: 2013-01-01/2016-12-31

Accurate depth estimation is one of key problems in 3D scene reconstruction and visualization, which can be expanded to many computer vision applications, such as object tracking, scene segmentation, visual navigation and so on. At present, single depth cue based depth estimation is still an open problem in computer vision. To utilize multiple depth cues in a dense of camera array, we explore on the three key aspects, including acquisition of multiple depth cues, multiple cues based depth estimation and accurate depth map optimization. In order to extract multiple depth cues from light field, we have built a camera array system to capture target scene. The elemental cameras are accurately calibrated, which can be used for synthetic imaging with a synthetic 2D or 3D focal plane. Then we have extracted scene depth information related structure cues, parallax cue and focus cue of target scene from light field EPI, refocusing image and confocal image respectively. After that, we have proposed that there is complementary relationship between parallax cue and focus cue. For depth estimation, we have introduced a novel ground control points (GCPs) based method to obtain dense disparity map. Moreover, by focusing on the parallax cue, we have proposed a segmentation-tree based cost aggregation to produce more robust disparity estimation for each pixel. Besides, we have also proposed a multi-occlusion model in light field, which can be performed to deal with the occlusion area in depth estimation. Finally, based on the light field sampling analysis, we have proposed a multi-depth cues fusion algorithm to estimate depth under the framework of Markov Random Field, which can take both advantages of shape from stereo and shape from focus. Our algorithm is more accurate than single cue based depth estimation algorithms. To optimize the result of depth estimation, we have first proposed a method to remove outliers based on penalized linear regression, which can eliminate the distraction of outliers. As for the estimation of occluded area, we have proposed a global optimization based on the surface camera and stereo matching method, which can achieve sub-pixel accuracy for depth estimation. To address the issue of aliasing artifacts in the light field imaging, we have proposed an angular aliasing detection algorithm by shifting the aperture model randomly, and then we introduce a multi-scale anti-aliasing rendering algorithm to stitch different non-aliasing image parts together. Our algorithm can significantly improve the confocal imaging quality. We also carry out several researches on other depth estimation related techniques and applications, such as multi-view video synchronization, light field super-pixel segmentation, local feature extraction of light field and applications in live face detection and so on.

After four year work, we have submitted 4 patent applications in China and published 20 papers, including 2 papers on TIP and TCSVT journals and 2 papers on CCF Rank A conferences ICCV and CVPR. We have also cultivated 2 NSFC young scholar funds, 5 Ph.D. and 10 master students under the support of this NSFC fund.


Key words: Depth estimation; Camera array; Multiple depth cues; Global optimization; Depth evaluation model

Sparse Representation with Geometric Configuration Constraint for Line Segment Matching
Qing Wang, Tingwang Chen, Lipeng Si
Neurocomputing, 134(25):100-110, 2014
Paper | Code | BibTeX | Github

Reconstructing Scene Depth and Appearance behind Foreground Occlusion Using Camera Array
Zhaolin Xiao, Qing Wang, Lipeng Si, Guoqing Zhou
ICIP 2014, 41-45
Paper | Code | BibTeX | Github

A Resection Method Based on Enhanced Continuous Taboo Search
Guoqing Zhou, Qing Wang
电子学报 (ACTA ELECTRONICA SINICA), 42(12):2422-2428
Paper | Code | BibTeX | Github

Enhanced Continuous Tabu Search for Parameter Estimation in Multiview Geometry
Guoqing Zhou, Qing Wang
ICCV 2013, 3240-3247
Paper | Code | BibTeX | Github

"The man can be destroyed but not defeated。" - Ernest Miller Hemingway