リケラボ論文検索は、全国の大学リポジトリにある学位論文・教授論文を一括検索できる論文検索サービスです。

リケラボ 全国の大学リポジトリにある学位論文・教授論文を一括検索するならリケラボ論文検索大学・研究所にある論文を検索できる

リケラボ 全国の大学リポジトリにある学位論文・教授論文を一括検索するならリケラボ論文検索大学・研究所にある論文を検索できる

大学・研究所にある論文を検索できる 「圧縮epsilon photographyによる撮影後の画像制御 (本文)」の論文概要。リケラボ論文検索は、全国の大学リポジトリにある学位論文・教授論文を一括検索できる論文検索サービスです。

コピーが完了しました

URLをコピーしました

論文の公開元へ論文の公開元へ
書き出し

圧縮epsilon photographyによる撮影後の画像制御 (本文)

伊藤, 厚史 慶應義塾大学

2021.08.26

概要

銀塩カメラからデジタルカメラへの切り替えが進み,撮影の利便性は著しく向上した.しかし,撮影者それぞれが持つ独自の好みに合致した「好ましい画像」を得るために,撮影者は未だ,フォーカス位置や絞りのサイズ,シャッタ速度や ISO 感度など,カメラの数多くのパラメータを撮影時に調整しなければならず,撮影後にそれらを変更・制御するのは従来,不可能であった.デジタルカメラ内の計算資源の向上や,光学技術の進歩に伴い,これらを自動で制御する技術は発展しているが,完全に間違えない撮影には至っておらず,ユーザの好みに応じて後から撮影画像を制御したい欲求は満たされない.ライトフィールド撮影は,フォーカス位置と撮影視点について撮影後の画像制御を可能にした.しかし,専用ハードウェアが必要であり,空間解像度を犠牲にして光線情報を取得するため,最終的な画像の解像度が低くなり,フォーカス位置や絞りの大きさの完全にフレキシブルな復元はできない.

それに対し,本論は,従来カメラを用いて,フォーカス位置や絞りの大きさ,露光時間を各々に変えて撮影された複数枚の撮影画像から,あらゆるカメラパラメータで撮影された画像を復元する技術を提案する.言い換えれば,あらかじめ設定されたパラメータによる連写画像を入力とし,従来であれば数千枚の撮像を必要とする完全な画像スタック(例:HDR の Aperture-Focus スタック)を,その 1%にも満たない 16~32 枚程度の撮像数から再構成する.

第1章では,本研究の背景と従来の研究を概説する.

第2章では,アルゴリズム構築の上で着目した,フォーカス位置・絞り値・露光レベルの全パラメータで撮影された画像スタックの統計的な冗長性を示す.

第3章では,数少ない撮影画像から完全な画像スタックを復元するアルゴリズムを解説する.前途の冗長性を活用した混合ガウンシアンモデル表現,グリーディ・アルゴリズムにより最適な組み合わせのパラメータ組合せについての説明が含まれる.

第4章では,実画像における実験により本手法の効果を示す.Focus スタック,Aperture- Focus スタック,複数露光レベルの Aperture-Focus スタックのそれぞれについて,定性・定量評価の結果を示す.

第5章では,画像スタック再構成が実現するアプリケーション例を示す.高精細な奥行情報を取得する Confocal ステレオ,および,撮影後のカメラ制御による画像のレタッチング,について解説する.

第6章では,結論として内容をまとめ,本研究の成果を要約する.

この論文で使われている画像

参考文献

[1] M. Levoy and P. Hanrahan, “Light field rendering,” Proc. ACM SIGGRAPH, pp.31– 42, 1996.

[2] LYTRO. “The lytro camera,” https://www.lytro.com/ .

[3] RAYTRIX. “3d light field camera technology,” http://www.raytrix.de/ .

[4] P. Green, W. Sun, W. Matusik, and F. Durand, “Multi-aperture photography,” ACM Trans. Graphics 26, 3, 68, 2007.

[5] T. Georgiev, A. Lumsdaine and S. Goma, "High Dynamic Range Image Capture with Plenoptic 2.0 Camera," Frontiers in Optics 2009/Laser Science XXV/Fall 2009 OSA Optics & Photonics Technical Digest, SWA7P, 2009.

[6] R. Raskar, “Computational photography: Epsilon to coded photography,” Emerging Trends in Visual Computing, Springer, 238–253, 2009.

[7] S. K. Nayar and Y. Nakagawa. “Shape from focus,” IEEE Trans. on Pattern Analysis and Machine Intelligence, 16(8):824-831, 1994.

[8] A. Levin and F.Durand. “Linear view synthesis using a dimensionality gap light field prior,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1831–1838, 2010.

[9] S. Kuthirummal, H. Nagahara, C. Zhou and S.K. Nayar, “Flexible depth of field photography,” IEEE Trans. Pattern Analysis and Machine Intelligence 33(1), 58–71, 2011.

[10] P. Debevec and J. Malik, “Recovering high dynamic range radiance maps from photographs,” Proc. ACM SIGGRAPH, 369–378, 1997.

[11] S. Hasinoff, and K. Kutulakos, “Confocal stereo,” Int. Journal of Computer Vision 81, 1, 82–104, 2009.

[12] M. Brown and D.G. Lowe, “Automatic panoramic image stitching using invariant features,” Int. Journal of Computer Vision 74, 1, 59–73, 2007.

[13] N. Joshi and M. F. Cohen, “Seeing Mt. Rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal,” In IEEE Intl. Conf. Computational Photography, 1–8, 2010.

[14] N. M. Law, C. D. Mackay and J. E. Baldwin, “Lucky Imaging: High angular resolution imaging in the visible from the ground,” arXiv preprint astro-ph/0507299, 2005.

[15] CASIO, “Exilim EX-10,” http://arch.casio.jp/dc/products/ex_10/

[16] G. Eilertsen, J. Kronander, G. Denes, R. Mantiuk and J. Unger, "HDR image reconstruction from a single exposure using deep CNNs," ACM Transactions on Graphics (TOG), 36, 1-15, 2017.

[17] C.A. Metzler, H. Ikoma, Y. Peng and G. Wetzstein, “Deep Optics for Single-Shot High-Dynamic-Range Imaging,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1375–1385, 2020.

[18] G. Wu, M. Zhao, L. Wang, Q. Dai, T. Chai and Y. Liu, “Light Field Reconstruction Using Deep Convolutional Network on EPI,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2017.

[19] N. K. Kalantari, T. C. Wang and R. Ramamoorthi, “Learning-based view synthesis for light field cameras,” ACM Transactions on Graphics (TOG), 35(6):193, 2016.

[20] P. Sakurikar, I. Mehta, V. N. Balasubramanian, P. J. Narayanan, “RefocusGAN: Scene Refocusing Using a Single Image.” Proc. IEEE European Conference on Computer Vision (ECCV), 2018.

[21] S. B. Dayan, D Mendlovic and R. Giryes, "Deep Sparse Light Field Refocusing," In Proc. of The British Machine Vision Conference (BMVC), 2020.

[22] R. Baraniuk, “Compressive sensing,” IEEE Signal Processing Magazine 24, 4, 118–121, 2007.

[23] E. J. Candes and M. B. Wakin, “An introduction to compressive sampling,” IEEE Signal Processing Magazine 25, 2,21–30, 2008.

[24] A. Bourrier, R. Gribonval, and P. Perez, “Compressive gaussian mixture estimation,” Proc. IEEE Acoustics, Speech and Signal Processing, 2013.

[25] T. Buades, Y. Lou, J. M. Morel and Z. Tang, “A note on multi-image denoising,” Proc. A Intl. Workshop on Local and Non-Local Approximation in Image Processing, 1-15, 2009.

[26] L. Zhang, S.Vaddadi, H.Jin, S. K. Nayar, "Multiple view image denoising," In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2009.

[27] D. Capel and A. Zisserman, “Super-resolution from multiple views using learnt image models,” IEEE Conf. on Computer Vision and Pattern Recognition, 2001.

[28] F. Sroubek and P. milanfar, “Robust multichannel blind deconvolution via fast alternating minimization,” IEEE Trans. on Image Processing, 21(4):1687-1700, 2012.

[29] L. Yuan, J. Sun, L. Quan and H. Y. Shum, “Image deblurring with blurred/noisy image pairs,” In ACM Trans. of Graphics, Vol 26, 1, 2007.

[30] S. Mann and R. Picard, “Being undigital with digital cameras,” MIT Media Lab Perceptual, 1994.

[31] S. W. Hasinoff, F. Durand and W. T. Freeman, “Noise-optimal capture for high dynamic range photography,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.

[32] S. W. Hasinoff and K. N. Kutulakos, “A layer-based restoration framework for variable-aperture photography,” Proc. IEEE Intl. Conf. Computer Vision, 2007.

[33] P. Favaro, A. Mennucci and S. Soatto, “Observing shape from defocused images,” Intl. Journal of Computer Vision (IJCV), 52(1), 25-43, 2003.

[34] A. Agarwala, M. Dontcheva, M. Agrawala, S. Drucker, A. Colburn, B. Curless, D. Salesin and M. Cohen, “Interactive digital photomontage,” In ACM Trans. of Graphics, vol 23, 294-302, 2004.

[35] J. G. McNally, T. Karpova, J. Cooper and J. A. Conchello, “Three-dimensional imaging by deconvolution microscopy,” Methods 19(3), 373–385, 1999.

[36] J. B. Sibarita, “Deconvolution microscopy,” In Microscopy Techniques. 201–243, 2005.

[37] E. Krotkov, “Focusing,” Intl. Journal of Computer Vision (IJCV), vol. 1, 3, 223-237, 1988.

[38] P. Grossmann, “Depth from focus,” Pattern Recognition Letters vol. 5, 1, 63–69, 1987.

[39] S. Gortler, R. Grzeszczuk, R. Szeliski and M. Cohen, “The lumigraph,” In SIGGRAPH, 43–54, 1996.

[40] T. Georgiev and A. Lumsdaine, "Superresolution with plenoptic 2.0 cameras", Signal Recovery and Synthesis. Meeting of the GSA, 2009.

[41] V. Boominathan, K. Mitra and A. Veeraraghavan, “Improving Resolution and Depth- of-Field of Light Field Cameras Using a Hybrid Imaging System,” IEEE Intl. Conf. on Computational Photography (ICCP), 2014.

[42] M. S. K. Gul, T. Wolf, M. Bätz, M. Ziegler and J. Keinert, “A High-Resolution High Dynamic Range Light-Field Dataset with an Application to View Synthesis and Tone- Mapping,” Proc. IEEE Int. Conf. on Multimedia & Expo Workshops, 2020.

[43] K. Marwah, G. Wetzstein, Y. Bando and R. Raskar, “Compressive light field photography using overcomplete dictionaries and optimized projections,” ACM Trans. Graphics 32, 4, 46, 2013.

[44] S. Tambe, A. Veeraraghavan and A. Agrawal, “Towards motion-aware light field video for dynamic scenes,” IEEE Intl. Conf. Computer Vision (ICCV), 2013.

[45] P. Peers, D. K. Mahajan, B. Lamond, A. Ghosh, W. Matusik, R. Ramamoorthi and P. Debevec, “Compressive light transport sensing,” ACM Trans. Graphics 28,1, 3, 2009.

[46] J. Y. Park and M. B. Wakin, “A multiscale framework for compressive sensing of video,” In Picture Coding Symposium, 1–4, 2009.

[47] A. Wagadarikar, R. John, R. Willett and D. Brady, “Single disperser design for coded aperture snapshot spectral imaging,” Applied optics 47, 10, B44–B51, 2008.

[48] D. Baron, S. Sarvotham and R. G. Baraniuk, “Bayesian compressive sensing via belief propagation,” IEEE Trans. Signal Processing vol. 58, 1, 269–280, 2010.

[49] G. Yu, G. Sapiro and S. Mallat, “Solving inverse problems with piecewise linear estimators: From gaussian mixture models to structured sparsity,” IEEE Trans. Image Processing 21, 5, 2012.

[50] J. Yang, X. Yuan, X. Liao, P. Llull, G. Sapiro, D. J. Brady and L. Carin, “Gaussian mixture model for video compressive sensing,” Proc. of Intl. Conf. on Image Processing, 2013.

[51] K. Mitra and A. Veeraraghavan, “Light field denoising, light field superresolution and stereo camera based refocusing using a gmm light field patch prior,” Proc. of IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), 2012.

[52] K. Mitra, O. Cossairt and A. Veeraraghavan, “Can we beat hadamard multiplexing? data driven design and analysis for computational imaging systems,” In IEEE Intl. Conf. on Computational Photography, 2014.

[53] A. Ito, S. Tambe, K. Mitra, A. C. Sankaranarayanan and A. Veeraraghavan, "Compressive Epsilon Photography for Post-Capture Control in Digital Imaging," ACM Transactions on Graphics 33(4):1-12, 2014.

[54] 伊藤 厚史, 青木 義満, “圧縮 Epsilon Photography に基づく任意の HDR Refocus 画像再構成,” 電子情報通信学会和文論文誌 D, Vol.J104-D,No.10, 2021.

[55] Adobe. “Adobe DNG Software Development Kit (SDK),” https://www.adobe.com/support/downloads/dng/dng_sdk.html

[56] 吉田道隆,長原一,鳥居秋彦,奥富正敏,谷口倫一郎,“Deep learning による圧縮ビデオセンシングの再構成,” 情報処理学会報告 2017-CVIM-208 (19),2017.

[57] M. Yoshida, A. Torii, M. Okutomi, K. Endo, Y. Sugiyama, R. Taniguchi and H. Nagahara, “Joint optimization for compressive video sensing and reconstruction under hardware constraints,” In Proc. of the European Conf. on Computer Vision (ECCV), 2018.

[58] Y. Hitomi, J. Gu, M. Gupta, T. Mitsunaga and S.K. Nayar, “Video from a Single Coded Exposure Photograph using a Learned Over-Complete Dictionary,” In Proc. of IEEE International Conference on Computer Vision (ICCV), 2011.

[59] H. Nagahara, Toshiki Sonoda, Kenta Endo, Y. Sugiyama and R. Taniguchi, “High- speed imaging using CMOS image sensor with quasi pixel-wise exposure,” Proc. of IEEE International Conference on Computational Photography (ICCP), 2016.

[60] A. Gupta, P. Bhat, M. Dontcheva, O. Deussen, B. Curless and M. Cohen, “Enhancing and experiencing spacetime resolution with videos and stills,” Proc. of Intl. Conf. on Computational Photography (ICCP), 2009.

[61] E. Van Den Berg and M. P. Friedlander, "Probing the Pareto Frontier for basis pursuit solutions", SIAM J. Scientific Comput., vol. 31, no. 2, pp. 890-912, 2008.

[62] Y. Pati, R. Rezaiifar and P. S. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition", Proc. Conf. Rec. 27th Asilomar Conf. Signals Syst. Comput., pp. 40-44, 1993.

[63] Y. Yagi, K. Takahashi, T. Fujii, T. Sonoda, H. Nagahara: "Designing Coded Aperture Camera Based on PCA and NMF for Light Field Acquisition", IEICE Transactions on Information and Systems, Vol. E101-D, No. 9, pp. 2190--2200, 2018.

[64] J. T. Flam, S. Chatterjee, K. Kansanen and T. Ekman, “On MMSE Estimation: A linear model under gaussian mixture statistics,” IEEE Trans. Signal Processing 60, 7, 3840–3845, 2012.

[65] Matlab. “makehdr,” https://www.mathworks.com/help/images/ref/makehdr.html

[66] F. Drago, W. L. Martens, K. Myszkowski, and N. Chiba "Design of a tone mapping operator for high-dynamic range images based upon psychophysical evaluation and preference mapping", Proc. SPIE 5007, Human Vision and Electronic Imaging VIII, 2003.

[67] Matlab. “tonemap,” https://www.mathworks.com/help/images/ref/tonemap.html

[68] J. Kim, R. Horstmeyer, I. J. Kim and R. Raskar, “Highlighted depth-of-field photography,” ACM Transactions on Graphics, Vol. 30 (3), 2011.

[69] Canon, “Canon EOS-40D,” https://global.canon/ja/c-museum/product/dslr795.html

[70] Canon, “EF50mm F1.4 USM,” https://cweb.canon.jp/ef/lineup/standard/ef50-f14/

[71] R. Yokoya and S. K. Nayar, “Extended Depth of Field Catadioptric Imaging Using Focal Sweep,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2019.

[72] Helicon Soft, “Helicon focus,” http://www.heliconsoft.com/heliconfocus.html

[73] R. K. Mantiuk, A. Tomaszewska and R. Mantiuk, “Comparison of Four Subjective Methods for Image Quality Assessment,” Computer Graphics Forum, Vol.31(8), pp 2478– 2491, 2012.

[74] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. on Image Processing, 13(4):600-612, 2004.

[75] A. Hore and D. Ziou, “Image Quality Metrics: PSNR vs. SSIM,” Proc. of the Intl. Conf. on Pattern Recognition (ICPR), page 2366–2369, 2010.

[76] Q. Chen, Y. Zhang, X. Cao, Y. Zhang and H. Xiong, “Depth map estimation with 4D light fields using confocal stereo,” Prof. of 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2016.

[77] J. Bolan, E. Hall, C. Clifford and B. Thurow, "Light-Field Imaging Toolkit," Software X, Vol. 5, pages 101 – 106, 2016.

[78] W. Fu, X. Tong, C. Shan, S. Zhu and B. Chen, “Implementing light field image refocusing algorithm,” Proc. of IEEE International Conference on Acoustics, Speech and Signal (ICASS), 2013.

[79] D. Walton, R. Anjos, S. Friston, D. Swann, K. Aksit, A. Steed and T. Ritschel, “Beyond blur,” ACM Transactions on Graphics, 40:4, pages 1-14, 2021.

[80] J. Abbott and B. Morse, “Interactive Depth-Aware Effects for Stereo Image Editing,” Proc. of Intl. Conf. on 3D Vision, pp. 263–270, 2013.

[81] L. Murmann, A. Davis, J. Kautz and F. Durand, “Computational bounce flash for indoor portraits,” ACM Transactions on Graphics, 35:6, pages 1-9, 2016.

[82] M. Gupta, D. Iso and S.K. Nayar, “Fibonacci Exposure Bracketing for High Dynamic Range Imaging,” Proc. of IEEE International Conference on Computer Vision (ICCV), 2013.

[83] H. Zhang and L. Carin “Multi-shot Imaging: Joint Alignment, Deblurring, and Resolution-Enhancement,” Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.

[84] R. Kingslake, “Optics in Photography,” SPIE Publications, 1992.

[85] X. Yang, L. Huang, Y. Luo, Y. Wu, H. Wang, Y. Rivenson and A. Ozcan, "Deep learning-based virtual refocusing of images using an engineered point-spread function," CoRR abs/2012.11892, 2020.

[86] S. S. Khan, V. R. Adarsh, V. Boominathan, J. Tan, A. Veeraraghavan, and K. Mitra, “Towards photorealistic reconstruction of highly multiplexed lensless images,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV, 2019), pp. 7860–7869, 2019.

[87] M.G. Retzlaff, J. Hanika, J. Beyerer and C. Dachsbacher, “Physically based computer graphics for realistic image formation to simulate optical measurement systems,” Journal of Sensors and Sensor Systems, 6 (1), 171 –184, 2017.

[88] D. Schraml, “Physically based synthetic image generation for machine learning: a review of pertinent literature,” Proc. of Photonics and Education in Measurement Science, 2019.

[89] M. Wang and W. Deng, “Deep visual domain adaptation: A survey,” In Neurocomputing 312, pp. 135–153, 2018.

[90] S. Sankaranarayanan, Y. Balaji, A. Jain, S. N. Lim, R. Chellappa, “Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation,” Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.

[91] M. Pharr, G. Humphreys and W. Jakob, “Physically based rendering: From theory to implementation,” third Cambridge, MA,2017.

参考文献をもっと見る

全国の大学の
卒論・修論・学位論文

一発検索!

この論文の関連論文を見る