Benchmark data set and method for depth estimation from light field images
Convolutional neural networks (CNNs) have performed extremely well for many image analysis tasks. However, supervised training of deep CNN architectures requires huge amounts of labeled data, which is unavailable for light field images. In this paper, we leverage on synthetic light field images and...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English English English |
| Published: |
Institute of Electrical and Electronics Engineers Inc
2018
|
| Subjects: | |
| Online Access: | http://irep.iium.edu.my/64466/ http://irep.iium.edu.my/64466/ http://irep.iium.edu.my/64466/ http://irep.iium.edu.my/64466/1/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth.pdf http://irep.iium.edu.my/64466/2/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth_SCOPUS.pdf http://irep.iium.edu.my/64466/3/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth_WOS.pdf |
| id |
iium-64466 |
|---|---|
| recordtype |
eprints |
| spelling |
iium-644662019-03-13T06:36:01Z http://irep.iium.edu.my/64466/ Benchmark data set and method for depth estimation from light field images Feng, Mingtao Wang, Yaonan Liu, Jian Zhang, Liang Mohd Zaki, Hasan Firdaus Mian, Ajmal Q350 Information theory Convolutional neural networks (CNNs) have performed extremely well for many image analysis tasks. However, supervised training of deep CNN architectures requires huge amounts of labeled data, which is unavailable for light field images. In this paper, we leverage on synthetic light field images and propose a two-stream CNN network that learns to estimate the disparities of multiple correlated neighborhood pixels from their epipolar plane images (EPIs). Since the EPIs are unrelated except at their intersection, a two-stream network is proposed to learn convolution weights individually for the EPIs and then combine the outputs of the two streams for disparity estimation. The CNN estimated disparity map is then refined using the central RGB light field image as a prior in a variational technique. We also propose a new real world data set comprising light field images of 19 objects captured with the Lytro Illum camera in outdoor scenes and their corresponding 3D pointclouds, as ground truth, captured with the 3dMD scanner. This data set will be made public to allow more precise 3D pointcloud level comparison of algorithms in the future which is currently not possible. Experiments on the synthetic and real world data sets show that our algorithm outperforms existing state of the art for depth estimation from light field images. Institute of Electrical and Electronics Engineers Inc 2018-03-09 Article PeerReviewed application/pdf en http://irep.iium.edu.my/64466/1/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth.pdf application/pdf en http://irep.iium.edu.my/64466/2/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth_SCOPUS.pdf application/pdf en http://irep.iium.edu.my/64466/3/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth_WOS.pdf Feng, Mingtao and Wang, Yaonan and Liu, Jian and Zhang, Liang and Mohd Zaki, Hasan Firdaus and Mian, Ajmal (2018) Benchmark data set and method for depth estimation from light field images. IEEE Transactions on Image Processing, 27 (7). pp. 3586-3598. ISSN 1057-7149 https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8310559 10.1109/TIP.2018.2814217 |
| repository_type |
Digital Repository |
| institution_category |
Local University |
| institution |
International Islamic University Malaysia |
| building |
IIUM Repository |
| collection |
Online Access |
| language |
English English English |
| topic |
Q350 Information theory |
| spellingShingle |
Q350 Information theory Feng, Mingtao Wang, Yaonan Liu, Jian Zhang, Liang Mohd Zaki, Hasan Firdaus Mian, Ajmal Benchmark data set and method for depth estimation from light field images |
| description |
Convolutional neural networks (CNNs) have performed extremely well for many image analysis tasks. However, supervised training of deep CNN architectures requires huge amounts of labeled data, which is unavailable for light field images. In this paper, we leverage on synthetic light field images and propose a two-stream CNN network that learns to estimate the disparities of multiple correlated neighborhood pixels from their epipolar plane images (EPIs). Since the EPIs are unrelated except at their intersection, a two-stream network is proposed to learn convolution weights individually for the EPIs and then combine the outputs of the two streams for disparity estimation. The CNN estimated disparity map is then refined using the central RGB light field image as a prior in a variational technique. We also propose a new real world data set comprising light field images of 19 objects captured with the Lytro Illum camera in outdoor scenes and their corresponding 3D pointclouds, as ground truth, captured with the 3dMD scanner. This data set will be made public to allow more precise 3D pointcloud level comparison of algorithms in the future which is currently not possible. Experiments on the synthetic and real world data sets show that our algorithm outperforms existing state of the art for depth estimation from light field images. |
| format |
Article |
| author |
Feng, Mingtao Wang, Yaonan Liu, Jian Zhang, Liang Mohd Zaki, Hasan Firdaus Mian, Ajmal |
| author_facet |
Feng, Mingtao Wang, Yaonan Liu, Jian Zhang, Liang Mohd Zaki, Hasan Firdaus Mian, Ajmal |
| author_sort |
Feng, Mingtao |
| title |
Benchmark data set and method for depth estimation from light field images |
| title_short |
Benchmark data set and method for depth estimation from light field images |
| title_full |
Benchmark data set and method for depth estimation from light field images |
| title_fullStr |
Benchmark data set and method for depth estimation from light field images |
| title_full_unstemmed |
Benchmark data set and method for depth estimation from light field images |
| title_sort |
benchmark data set and method for depth estimation from light field images |
| publisher |
Institute of Electrical and Electronics Engineers Inc |
| publishDate |
2018 |
| url |
http://irep.iium.edu.my/64466/ http://irep.iium.edu.my/64466/ http://irep.iium.edu.my/64466/ http://irep.iium.edu.my/64466/1/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth.pdf http://irep.iium.edu.my/64466/2/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth_SCOPUS.pdf http://irep.iium.edu.my/64466/3/64466_Benchmark%20Data%20Set%20and%20Method%20for%20Depth_WOS.pdf |
| first_indexed |
2023-09-18T21:31:29Z |
| last_indexed |
2023-09-18T21:31:29Z |
| _version_ |
1777412540619292672 |