Relative pose estimation with RGB camera : improvements and applications (本文)

[1] Sameer Agarwal, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M Seitz, and Richard Szeliski. Building Rome in a Day. Communications of the ACM, 54(10):105–112, October 2011.

[2] Maneesh Agrawala, Doantam Phan, Julie Heiser, John Haymaker, Jeff Klingner, Pat Hanrahan, and Barbara Tversky. Designing Effective Step-by-Step Assembly Instructions. ACM Transactions on Graphics (TOG), 22(3):828–837, July 2003.

[3] Guillem Aleny`a, Sergi Foix, and Carme Torras. Using ToF and RGBD Cameras for 3D Robot Perception and Manipulation in Human Envi- ronments. Intelligent Service Robotics, 7(4):211–220, September 2014.

[4] Aaron Bangor, Philip Kortum, and James Miller. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. Journal of Usability Studies archive, 4(3):114–123, May 2009.

[5] Connelly Barnes, Eli Shechtman, Adam Finkelstein, and Dan B. Gold- man. PatchMatch: A Randomized Correspondence Algorithm for Struc- tural Image Editing. ACM Transactions on Graphics (TOG), 28(3), July 2009.

[6] Nicolas Biard, Salom´e Cojean, and Eric Jamet. Effects of Segmentation and Pacing on Procedural Learning by Video. Computers in Human Behavior, 89:411–417, December 2018.

[7] Dorit Borrmann, Jan Elseberg, and Andreas Nu¨chter. Thermal 3D Map- ping of Building Fa¸cades. Intelligent Autonomous Systems 12, pages 173–182, 2013.

[8] Gary Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000.

[9] Paul Breedveld. Observation, Manipulation, and Eye-Hand Coordina- tion Problems in Minimally Invasive Surgery. In European Conference on Human Decision Making and Manual Control, pages 219–231, De- cember 1997.

[10] John Brooke. SUS: A Quick and Dirty Usability Scale. Usability Eval- uation in Industry, 189(194):4–7, 1996.

[11] Andreas Butz. BETTY: Planning and Generating Animations for the Visualization of Movements and Spatial Relations. In Proceedings of the Workshop on Advanced Visual Interfaces, pages 53–58, June 1994.

[12] Robert Castle, Georg Klein, and David W. Murray. Video-Rate Lo- calization in Multiple Maps for Wearable Augmented Reality. In IEEE International Symposium on Wearable Computers, pages 15–22, Septem- ber 2008.

[13] Arturo De la Escalera and Jose Mar´ıa Armingol. Automatic Chessboard Detection for Intrinsic and Extrinsic Camera Parameter Calibration. Sensors, 10(3):2027–2044, 2010.

[14] L.S. Homem De Mello and Arthur C. Sanderson. A Correct and Com- plete Algorithm for the Generation of Mechanical Assembly Sequences. IEEE Transactions on Robotics and Automation, 7(2):228–240, April 1991.

[15] Frederic Devernay and Olivier Faugeras. Straight Lines have to be Straight. Machine Vision and Applications, 13(1):14–24, August 2001.

[16] Jakob Engel, Vladlen Koltun, and Daniel Cremers. Direct Sparse Odom- etry. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 40(3):611–625, March 2017.

[17] Jakob Engel, Thomas Sch¨ops, and Daniel Cremers. LSD-SLAM: Large- scale Direct Monocular SLAM. In European Conference on Computer Vision (ECCV), pages 834–849, September 2014.

[18] Steven Feiner, Blair Macintyre, and Dor´ee Seligmann. Knowledge-Based Augmented Reality. Communications of the ACM, 36(7):53–62, July 1993.

[19] Mark Fiala. Magic Mirror System with Hand-held and Wearable Aug- mentations. IEEE Virtual Reality Conference, pages 251–254, March 2007.

[20] Christian Forster, Matia Pizzoli, and Davide Scaramuzza. SVO: Fast Semi-Direct Monocular Visual Odometry. In IEEE International Con- ference on Robotics and Automation (ICRA), pages 15–22, May 2014.

[21] Catherine O. Fritz, Peter E. Morris, and Jennifer J. Richler. Effect Size Estimates: Current Use, Calculations, and Interpretation. Journal of Experimental Psychology: General, 141(1):2–18, February 2012.

[22] Dorian Ga´lvez-Lo´pez and Juan D Tardos. Bags of Binary Words for Fast Place Recognition in Image Sequences. IEEE Transactions on Robotics, 28(5):1188–1197, October 2012.

[23] Xiang Gao, Rui Wang, Nikolaus Demmel, and Daniel Cremers. LDSO: Direct Sparse Odometry with Loop Closure. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2198–2204, October 2018.

[24] T. B. Garland and Christopher A. Sanchez. Rotational Perspective and Learning Procedural Tasks from Dynamic Media. Computers and Education, 69:31–37, November 2013.

[25] Ankit Gupta, Dieter Fox, Brian Curless, and Michael Cohen. Duplo- Track: A Real-Time System for Authoring and Guiding Duplo Block Assembly. In Annual ACM Symposium on User Interface Software and Technology (UIST), pages 389–402, October 2012.

[26] Youngjib Ham and Mani Golparvar-Fard. An Automated Vision-Based Method for Rapid 3D Energy Performance Modeling of Existing Build-ings using Thermal and Digital Imagery. Advanced Engineering Infor- matics, 27(3):395–409, August 2013.

[27] Sandra G. Hart and Lowell E. Staveland. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Ad- vances in Psychology, 52:139–183, 1988.

[28] Richard Hartley and Andrew Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.

[29] Julie Heiser, Doantam Phan, Maneesh Agrawala, Barbara Tversky, and Pat Hanrahan. Identification and Validation of Cognitive Design Princi- ples for Automated Generation of Assembly Instructions. In the Working Conference on Advanced Visual Interfaces, pages 311–319, May 2004.

[30] Steven Henderson and Steven Feiner. Exploring the Benefits of Augmented Reality Documentation for Maintenance and Repair. IEEE Transactions on Visualization and Computer Graphics (TVCG), 17(10):1355–1368, October 2011.

[31] Stefan Hinterstoisser, Vincent Lepetit, Slobodan Ilic, Stefan Holzer, Gary Bradski, Kurt Konolige, and Nassir Navab. Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes. In Asian Conference on Computer Vision (ACCV), pages 548–562, November 2012.

[32] Radu Horaud, Miles Hansard, Georgios Evangelidis, and Cl´ement M´enier. An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies. Machine Vision and Applications, 27(7):1005–1020, June 2016.

[33] Denis Kalkofen, Erick Mendez, and Dieter Schmalstieg. Comprehensible Visualization for Augmented Reality. IEEE Transactions on Visualiza- tion and Computer Graphics (TVCG), 15(2):193–204, March 2009.

[34] Denis Kalkofen, Markus Tatzgern, and Dieter Schmalstieg. Explosion Diagrams in Augmented Reality. In IEEE Virtual Reality Conference, pages 71–78, March 2009.

[35] Bernhard Kerbl, Denis Kalkofen, Markus Steinberger, and Dieter Schmalstieg. Interactive Disassembly Planning for Complex Objects. Computer Graphics Forum, 34(2):287–297, May 2015.

[36] Bernhard Kornberger. Fade2D Delaunay Triangulation Library. https://www.geom.at/contact/publications/.

[37] Hideyuki Kume, Arne Supp´e, and Takeo Kanade. Vehicle Localization along a Previously Driven Route Using an Image Database. In IAPR International Conference on Machine Vision Applications (MVA), pages 177–180, May 2013.

[38] Rainer Ku¨mmerle, Giorgio Grisetti, Hauke Strasdat, Kurt Konolige, and Wolfram Burgard. g2o: A General Framework for Graph Optimization. In IEEE International Conference on Robotics and Automation (ICRA), pages 3607–3613, June 2011.

[39] Florian Ledermann and Dieter Schmalstieg. APRIL A High-Level Framework for Creating Augmented Reality Presentations. In IEEE Virtual Reality Conference, pages 187–194, March 2005.

[40] Der-Tsai Lee and Bruce J Schachter. Two Algorithms for Construct- ing a Delaunay Triangulation. International Journal of Computer and Information Sciences, 9(3):219–242, June 1980.

[41] Cheng Li and Kris M. Kitani. Pixel-Level Hand Detection in Ego-Centric Videos. In IEEE Conference on Computer Vision and Pattern Recogni- tion (CVPR), pages 3570–3577, June 2013.

[42] Wilmot Li, Maneesh Agrawala, Brian Curless, and David Salesin. Au- tomated Generation of Interactive 3D Exploded View Diagrams. ACM Transactions on Graphics (TOG), 27(3):1–7, August 2008.

[43] Jacob Boesen Madsen, Markus Tatzgern, Claus B. Madsen, Dieter Schmalstieg, and Denis Kalkofen. Temporal Coherence Strategies for Augmented Reality Labeling. IEEE Transactions on Visualization and Computer Graphics (TVCG), 22(4):1415–1423, April 2016.

[44] Sotiris Makris, George Pintzos, Loukas Rentzos, and George Chrys- solouris. Assembly Support Using AR Technology Based on Automatic Sequence Generation. Cirp Annals-manufacturing Technology, 62(1):9– 12, 2013.

[45] Kazuki Matsumoto, Wataru Nakagawa, Hideo Saito, Maki Sugimoto, Takashi Shibata, and Shoji Yachida. AR Visualization of Thermal 3D Model by Hand-held Cameras. In International Conference on Computer Vision Theory and Applications (VISAPP), pages 480–487, March 2015.

[46] Paul Mijksenaar and Piet Westendorp. Open Here. The Art of Instruc- tional Design. Thames & Hudson, 1999.

[47] Niloy J. Mitra, Yong-Liang Yang, Dong-Ming Yan, Wilmot Li, and Ma- neesh Agrawala. Illustrating How Mechanical Assemblies Work. ACM Transactions on Graphics (TOG), 29(4):58:1–58:12, July 2010.

[48] Peter Mohr, Bernhard Kerbl, Michael Donoser, Dieter Schmalstieg, and Denis Kalkofen. Retargeting Technical Documentation to Augmented Reality. In Annual ACM Conference on Human Factors in Computing Systems (CHI), pages 3337–3346, April 2015.

[49] Peter Mohr, David Mandl, Markus Tatzgern, Eduardo E. Veas, Dieter Schmalstieg, and Denis Kalkofen. Retargeting Video Tutorials Showing Tools with Surface Contact to Augmented Reality. In Annual ACM Con- ference on Human Factors in Computing Systems (CHI), pages 6547– 6558, May 2017.

[50] Peter Mohr, Markus Tatzgern, Tobias Langlotz, Andreas Lang, Dieter Schmalstieg, and Denis Kalkofen. TrackCap: Enabling Smartphones for 3D Interaction on Mobile Head-Mounted Displays. In Annual ACM Conference on Human Factors in Computing Systems (CHI), pages 1– 11, May 2019.

[51] Raul Mur-Artal and Juan D Tard´os. ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D cameras. IEEE Transactions on Robotics, 33(5):1255–1262, June 2017.

[52] Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. KinectFusion: Real- time Dense Surface Mapping and Tracking. In IEEE International Sym- posium on Mixed and Augmented Reality (ISMAR), pages 127–136, Oc- tober 2011.

[53] Marc Nienhaus and Ju¨rgen Do¨llner. Dynamic Glyphs - Depicting Dy- namics in Images of 3D Scenes. In International Symposium on Smart Graphics, pages 102–111, June 2003.

[54] Masatoshi Okutomi and Takeo Kanade. A Multiple-Baseline Stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 15(4):353–363, April 1993.

[55] Fred G. Paas. Training Strategies for Attaining Transfer of Problem Solving Skills in Statistics : a Cognitive-load Approach. Journal of Educational Psychology, 84(4):429–434, December 1992.

[56] Vivek Pradeep, Christoph Rhemann, Shahram Izadi, Christopher Zach, Michael Bleyer, and Steven Bathiche. MonoFusion: Real-time 3D Re- construction of Small Scenes with a Single Web Camera. In IEEE Inter- national Symposium on Mixed and Augmented Reality (ISMAR), pages 83–88, October 2013.

[57] Surya Prakash, Pei Yean Lee, Terry Caelli, and Tim Raupach. Ro- bust Thermal Camera Calibration and 3D Mapping of Object Surface Temperatures. Thermosense XXVIII, 6205(1):62050J–62050J–8, April 2006.

[58] Victor A. Prisacariu and Ian D. Reid. PWP3D: Real-time Segmentation and Tracking of 3D Objects. International Journal of Computer Vision, 98(3):335–354, January 2012.

[59] Tong Qin, Peiliang Li, and Shaojie Shen. Relocalization, Global Op- timization and Map Merging for Monocular Visual-Inertial SLAM. In IEEE International Conference on Robotics and Automation (ICRA), pages 1197–1204, May 2018.

[60] Tong Qin, Peiliang Li, and Shaojie Shen. VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IEEE Transactions on Robotics, 34(4):1004–1020, July 2018.

[61] Carlos Ricolfe-Viala and Antonio-Jos´e Sa´nchez-Salmero´n. Robust Met- ric Calibration of Non-linear Camera Lens Distortion. Pattern Recogni- tion, 43(4):1688–1699, April 2010.

[62] Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. Grabcut: Interactive Foreground Extraction using Iterated Graph Cuts. ACM Transactions on Graphics (TOG), 23(3):309–314, August 2004.

[63] Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary R Bradski. ORB: an Efficient Alternative to SIFT or SURF. In IEEE International Conference on Computer Vision (ICCV), pages 2564–2571, November 2011.

[64] Philip Saponaro, Scott Sorensen, Stephen Rhein, and Chandra Kamb- hamettu. Improving Calibration of Thermal Stereo Cameras using Heated Calibration Board. In IEEE International Conference on Im- age Processing (ICIP), pages 4718–4722, September 2015.

[65] Thomas Schneider, Marcin Dymczyk, Marius Fehr, Kevin Egger, Simon Lynen, Igor Gilitschenski, and Roland Siegwart. Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization. IEEE Robotics and Automation Letters, 3(3):1418–1425, January 2018.

[66] Johannes Lutz Scho¨nberger and Jan-Michael Frahm. Structure-from- Motion Revisited. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4104–4113, June 2016.

[67] Johannes L Scho¨nberger, Enliang Zheng, Jan-Michael Frahm, and Marc Pollefeys. Pixelwise View Selection for Unstructured Multi-View Stereo. In European Conference on Computer Vision (ECCV), pages 501–518, September 2016.

[68] Tianjia Shao, Dongping Li, Yuliang Rong, Changxi Zheng, and Kun Zhou. Dynamic Furniture Modeling Through Assembly Instructions. ACM Transactions on Graphics (TOG), 35(6), November 2016.

[69] Tianjia Shao, Wilmot Li, Kun Zhou, Weiwei Xu, Baining Guo, and Niloy J. Mitra. Interpreting Concept Sketches. ACM Transactions on Graphics (TOG), 32(4), July 2013.

[70] Ingrid A.E. Spanjers, Tamara Van Gog, Pieter Wouters, and Jeroen J.G. Van Merri¨enboer. Explaining the Segmentation Effect in Learning from Animations: The Role of Pausing and Temporal Cueing. Computers and Education, 59(2):274–280, September 2012.

[71] Colin Studholme, Derek LG Hill, and David J Hawkes. An Overlap Invariant Entropy Measure of 3D Medical Image Alignment. Pattern Recognition, 32(1):71–86, January 1999.

[72] Mengu Sukan, Carmine Elvezio, Ohan Oda, Steven Feiner, and Barbara Tversky. Parafrustum: Visualization Techniques for Guiding a User to a Constrained Set of Viewing Positions and Orientations. In Annual ACM Symposium on User Interface Software and Technology (UIST), page 331–340, Octorber 2014.

[73] Satoshi Suzuki and Keiichi Abe. Topological Structural Analysis of Dig- itized Binary Images by Border Following. Computer Vision, Graphics, and Image Processing, 30(1):32–46, April 1985.

[74] Takafumi Taketomi, Tomokazu Sato, and Naokazu Yokoya. Real-Time and Accurate Extrinsic Camera Parameter Estimation using Feature Landmark Database for Augmented Reality. Computers and Graphics, 35(4):768–777, August 2011.

[75] Takafumi Taketomi, Hideaki Uchiyama, and Sei Ikeda. Visual SLAM Al- gorithms: a Survey from 2010 to 2016. IPSJ Transactions on Computer Vision and Applications, 9(1):16, June 2017.

[76] Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. Compar- ative Effectiveness of Augmented Reality in Object Assembly. In An-nual ACM Conference on Human Factors in Computing Systems (CHI), pages 73–80, April 2003.

[77] Markus Tatzgern, Raphael Grasset, Eduardo Veas, Denis Kalkofen, Hartmut Seichter, and Dieter Schmalstieg. Exploring Real World Points of Interest: Design and Evaluation of Object-centric Exploration Tech- niques for Augmented Reality. Pervasive and Mobile Computing, 18:55– 70, April 2015.

[78] Markus Tatzgern, Denis Kalkofen, Raphael Grasset, and Dieter Schmal- stieg. Hedgehog Labeling: View Management Techniques for External Labels in 3D Space. In IEEE Virtual Reality, pages 1–6, April 2014.

[79] Shinji Umeyama. Least-Squares Estimation of Transformation Parame- ters between Two Point Patterns. IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 13(4):376–380, April 1991.

[80] Jonathan Ventura, Clemens Arth, Gerhard Reitmayr, and Dieter Schmalstieg. Global Localization from Monocular SLAM on a Mobile Phone. IEEE Transactions on Visualization and Computer Graphics (TVCG), 20(4):531–539, March 2014.

[81] Jonathan Ventura and Tobias H¨ollerer. Wide-area Scene Mapping for Mobile Visual Tracking. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pages 3–12, November 2012.

[82] Stephen Vidas, Ruan Lakemond, Simon Denman, Clinton Fookes, Sridha Sridharan, and Tim Wark. A Mask-Based Approach for the Geometric Calibration of Thermal-Infrared Cameras. IEEE Transac-tions on Instrumentation and Measurement, 61(6):1625–1635, February 2012.

[83] Stephen Vidas, Peyman Moghadam, and Michael Bosse. 3D Thermal Mapping of Building Interiors using an RGB-D and Thermal Camera. In IEEE International Conference on Robotics and Automation (ICRA), pages 2311–2318, May 2013.

[84] Paul Viola and William M Wells III. Alignment by Maximization of Mu- tual Information. International Journal of Computer Vision, 24(2):137– 154, September 1997.

[85] Lukas Von Stumberg, Vladyslav Usenko, and Daniel Cremers. Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization. In IEEE International Conference on Robotics and Automation (ICRA), pages 2510–2517, May 2018.

[86] Bin Wang, Guofeng Wang, Andrei Sharf, Yangyan Li, Fan Zhong, Xuey- ing Qin, Daniel CohenOr, and Baoquan Chen. Active Assembly Guid- ance with Online Video Parsing. In IEEE Conference on Virtual Reality and 3D User Interfaces, pages 459–466, March 2018.

[87] Rui Wang, Martin Schworer, and Daniel Cremers. Stereo DSO: Large- Scale Direct Sparse Visual Odometry With Stereo Cameras. In IEEE In- ternational Conference on Computer Vision (ICCV), pages 3903–3911, October 2017.

[88] Martin Weinmann, Jens Leitloff, Ludwig Hoegner, Boris Jutzi, Uwe Stilla, and Stefan Hinz. Thermal 3D Mapping for Object Detection in Dynamic Scenes. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2(1):53–60, November 2014.

[89] Jeremy M. Wolfe. Guided Search 2.0 a Revised Model of Visual Search. Psychonomic Bulletin & Review, 1:202–238, June 1994.

[90] Jimmy Wu, Bolei Zhou, Rebecca Russell, Vincent Kee, Syler Wagner, Mitchell Hebert, Antonio Torralba, and David MS Johnson. Real-Time Object Pose Estimation with Pose Interpreter Networks. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 6798–6805, October 2018.

[91] Masahiro Yamaguchi, Shohei Mori, Peter Mohr, Markus Tatzgern, Ana Stanescu, Hideo Saito, and Denis Kalkofen. Video-Annotated Aug- mented Reality Assembly Tutorials. In Annual ACM Symposium on User Interface Software and Technology (UIST), pages 1010–1022, Oc- tober 2020.

[92] Masahiro Yamaguchi, Shohei Mori, Hideo Saito, Shoji Yachida, and Takashi Shibata. Global-Map-Registered Local Visual Odometry Using On-the-Fly Pose Graph Updates. In Augmented Reality, Virtual Reality, and Computer Graphics, pages 299–311. Springer, August 2020.

[93] Masahiro Yamaguchi, Trong Phuc Truong, Shohei Mori, Vincent Nozick, Hideo Saito, Shoji Yachida, and Hideaki Sato. Superimposing Thermal- Infrared Data on 3D Structure Reconstructed by RGB Visual Odom- etry. IEICE Transactions on Information and Systems, 101(5):1296– 1307, 2018.

[94] Kazutaka Yasuda, Takeshi Naemura, and Hiroshi Harashima. Thermo- Key: Human Region Segmentation from Video. IEEE Computer Graph- ics and Applications, 24(1):26–30, January 2004.

[95] Guangjun Zhang, Junji He, and Xianming Yang. Calibrating Camera Radial Distortion with Cross-Ratio Invariability. Optics & Laser Tech- nology, 35(6):457–461, September 2003.

[96] Zhengyou Zhang. A Flexible New Technique for Camera Calibra- tion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330–1334, November 2000.