Real-Time Depth Map Refinement

Abstract

In this paper, we propose a new refinement filter for depth maps. The filter convolutes a depth map by a jointly computed kernel on a natural image with a weight map. We call the filter weighted joint bilateral filter. The filter fits an outline of an object in the depth map to the outline of the object in the natural image, and it reduces noises. An additional filter of slope depth compensation filter removes blur across object boundary.

The filter set’s computational cost is low and is independent of depth ranges. Thus we can refine depth maps to generate accurate depth map with lower cost. In addition, we can apply the filters for various types of depth map, such as computed by simple block matching, Markov random field based optimization, and Depth sensors.

Experimental results show that the proposed filter has the best performance of improvement of depth map accuracy, and the proposed filter can perform real-time refinement.

Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement

T. Matsuo, N. Fukushima, Y. Ishibashi, "Weighted Joint Bilateral Filter with Slope Depth Compensation Filter for Depth Map Refinement," Proc. International Conference on Computer Vision Theory and Applications (VISAPP 2013), vol. 2, pp. 300-309, Feb. 2013. [pdf]

Download

These links are our paper (pdf), and it's code [pdf] [code: GitHub] .

The code is written in C++ with SIMD intrinsic (SSE4.1) and TBB parallel optimization, and use OpenCV 2.4 and Intel Threading Building Blocks (Intel TBB). The code also includes the fastest edge keep filters;

For example, the bilateral filter is about 10x faster than the OpenCV function, and guided filter is 20x faster than the Matlab function. The codes are tested on Visual Studio 2010.

Last update: 2013/03/28

Results and Example

Kinect depth refinement results

Link to Related Works

Return to Top