Jongwoo Lim

Address: Room 505, IT/BT bldg., Hanyang University

222 Wangsimni-ro, Seongdong-gu, Seoul, Korea

Phone: +82-2-2220-2376 (office) +82-10-6334-7422 (mobile)

E-mail: jongwoo.lim@gmail.com, jlim@hanyang.ac.kr

Homepage: http://cvlab.hanyang.ac.kr/˜jwlim

Research Interest

Geometric scene understanding for intelligent robots and smart vehicles.

- reconstruct scene geometry together with its semantic meaning from visual and other inputs.

- design intelligent behaviors of robots and vehicles using the understandings of the environment.

Visual tracking and scene understanding.

- track target objects in input videos and reason the subjects’ activities and interactions.

* research area : computer vision, machine learning, and robotics.

Education

University of Illinois, Urbana Champaign, Urbana, IL, USA

Ph.D. in Computer Science, Dec. 2005

Thesis: On Clustering Images of Objects

Advisor: Prof. David J. Kriegman

Aug. 2000 - Dec. 2005 (MS/PhD GPA: 4.0/4.0)

University of Illinois, Urbana Champaign, Urbana, IL, USA

M.S. in Computer Science, Dec. 2003

Thesis: Tracking Humans Using Prior and Learned Representations of Shape and Appearance

Advisor: Prof. David J. Kriegman

Seoul National University, Seoul, Korea

B.S. in Computer Science, Feb. 1997

summa cum laude (GPA 4.01/4.3)

Experience

Associate Professor, Hanyang University, Seoul, Korea, Mar. 2015 - Present

Division of Computer Science & Engineering

Assistant Professor, Hanyang University, Seoul, Korea, Mar. 2012 - Feb. 2014

Division of Computer Science & Engineering

1

Software Engineer, Google, inc., Mountain View, CA, USA, Feb. 2011 - Feb. 2012

worked in Street View team, developing new algorithms for geometric environment modeling.

Senior Scientist, Honda Research Institute USA, inc., Mountain View, CA, Jul. 2005 - Feb. 2011

worked in computer vision, robotics and machine learning research for humanoid robots and

intelligent vehicles.

Summer Intern, University of California, San Diego, La Jolla, CA, USA, Jun. 2004 - Aug. 2004

conducted research on multi-camera people tracking systems.

Summer Intern, Honda Research Institute, Mountain View, CA, USA, May. 2003 - Aug. 2003

developed the human face tracking and recognition system.

Research Assistant, University of Illinois, Urbana Champaign, Jan. 2001 - May. 2005

developed human tracking modules in the Active Space project (http://devius.cs.uiuc.edu/gaia).

Senior Programmer/Engineer, Triton Tech inc., Seoul, Korea, Jan. 1997 - Aug. 2000

designed and developed the CLUE system (content-based text auto-categorization system).

developed a Metadesk enterprise document management system.

Scholarships and Honors

Best Paper Award for “Visual Robot Localization and Map Building”

in Honda Research Institute Global Workshop, May 2010

Most Valuable Performance Award for 2007-2008

awarded extra $50,000 research budget by Honda Research Institute USA inc., Apr 2008

Doctoral Study Abroad Scholarship

Korea Foundation for Advanced Studies, Aug. 2000 - Jul. 2005

College Student Scholarship

Korea Foundation for Advanced Studies, Mar. 1995 - Feb. 1997

Undergraduate Student Scholarship

Dongwon Education Foundation, Mar. 1996 - Feb. 1997

Honorary Scholarship

Seoul National University, Sep. 1993 - Feb. 1996

Publications

Ph.D. Thesis, On Clustering Images of Objects, University of Illinois at Urbana-Champaign, 2005

M.S. Thesis, Tracking Humans Using Prior and Learned Representations of Shape and Appearance,

University of Illinois at Urbana-Champaign, 2003

2

Journal Publications

1. David Ross, Jongwoo Lim, Ruei-Sung Lin, Ming-Hsuan Yang, “Incremental Learning for Robust

Visual Tracking,” International Journal of Computer Vision (Special Issue: Learning for Vision), Vol.

77, No. 1-3, Pg. 125–141, May 2008 [ 828 citations1 ]

2. Vivek Pradeep, Jongwoo Lim*, “Egomotion Estimation Using Assorted Features,” International

Journal of Computer Vision, Vol. 98, Issue 2, Page 202–216, June 2012

3. Jongwoo Lim*, Jan-Michael Frahm, Marc Pollefeys, “Online Environment Mapping using

Metric-topological Maps,” The International Journal of Robotics Research, Volume 31, Issue 12, Page

1394–1408, October 2012

4. Sehyung Lee, Jin Han Lee, Jongwoo Lim*, Il Hong Suh, “Robust Stereo Maching using Adaptive

Random Walk with Restart Algorithm,” Image and Vision Computing, Volume 37, Page 1–11, May

2015

5. Jongwoo Lim*, Minsoo Ryu, “Optimized Projection Patterns for Stereo Systems,” Image and

Vision Computing, Vol. 39, Page 10-22, July 2015

6. Yi Wu, Jongwoo Lim*, Ming-Hsuan Yang, “Object Tracking Benchmark,” IEEE Transaction on

Pattern Analysis and Machine Intelligence, Vol. 37 , Issue: 9, Page 1834–1848, January 2015

7. Guoxuan Zhang, Jin Han Lee, Jongwoo Lim, Il Hong Suh*, “Building a 3-D Line-Based Map Using

Stereo SLAM,” IEEE Transactions on Robotics, Vol. 31, Issue 6, Page 1364–1377, December, 2015

8. Mohamed A. Naiel, M. Omair Ahmad, M.N.S. Swamy, Jongwoo Lim, Ming-Hsuan Yang*, “Online

Multi-Object Tracking via Robust Collaborative Model and Sample Selection,” Computer Vision and

Image Understanding, Vol. 154, Page 94–107, January, 2017

Conference Publications

9. Jeffrey Ho, Ming-Hsuan Yang, Jongwoo Lim, Kuang-Chih Lee, David Kriegman, “Clustering

Appearances of Objects Under Varying Illumination Conditions”, IEEE Conference on Computer

Vision and Pattern Recognition (CVPR), 2003, vol. 1, pp. 11-18

[ oral presentation, acceptance rate 6.6%, 206 citations ]

10. Jongwoo Lim, Jeffrey Ho, Ming-Hsuan Yang, Kuang-Chih Lee, David Kriegman, “Image Clustering

with Metric, Local Linear Structure and Affinity Symmetry”, The 8th European Conference on

Computer Vision (ECCV), 2004, vol 1, pp. 456-468 [ acceptance rate 34.2% ]

11. David Ross, Jongwoo Lim, Ming-Hsuan Yang, “Adaptive Probabilistic Visual Tracking with

Incremental Subspace Update”, The 8th European Conference on Computer Vision (ECCV), 2004,

vol 2, pp. 470-482 [ acceptance rate 34.2%, 126 citations ]

1All citation counts are from Google Scholar on May., 2014.

3

12. Jongwoo Lim, David Kriegman, “Tracking Humans Using Prior and Learned Representations of

Shape and Appearance”, The 6th International Conference on Automatic Face and Gesture

Recognition (FG), 2004, pp. 869-874

13. Ruei-Sung Lin, David Ross, Jongwoo Lim, Ming-Hsuan Yang, “Adaptive Discriminative Generative

Model and Its Applications”, The 18th Annual Conference on Neural Information Processing Systems

(NIPS), 2004, pp. 801-808 [ acceptance rate ˜30%, 81 citations ]

14. Jongwoo Lim, David Ross, Ruei-Sung Lin, Ming-Hsuan Yang, “Incremental Learning for Visual

Tracking”, The 18th Annual Conference on Neural Information Processing Systems (NIPS), 2004, pp.

793-800 [ acceptance rate ˜30%, 244 citations ]

15. Sameer Agarwal, Jongwoo Lim, Lihi Zelnik-Manor, Pietro Perona, David Kriegman, Serge

Belongie, “Beyond Pairwise Clustering ”, IEEE Conference on Computer Vision and Pattern

Recognition (CVPR), 2005, vol. 2, pp. 838-845 [ oral presentation, acceptance rate 6.5%, 104 citations ]

16. Jongwoo Lim, Ming-Hsuan Yang, “A Direct Method for Modeling Non-rigid Motion with Thin Plate

Spline ”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2005, vol. 1, pp.

1196-1202 [ acceptance rate 28.1%, 54 citations ]

17. Jongwoo Lim, Jeffrey Ho, Ming-hsuan Yang, David Kriegman, “Passive Photometric Stereo from

Motion ”, 10th IEEE International Conference on Computer Vision (ICCV), 2005, vol. 2, pp.

1635-1642 [ acceptance rate 19.9%, 74 citations ]

18. Jeffrey Ho, Jongwoo Lim, Ming-hsuan Yang, David Kriegman, “Integrating Surface Normal Vectors

Using Fast Marching Method ”, The 9th European Conference on Computer Vision (ECCV), 2006,

part. 3, pp. 239-250 [ acceptance rate 21.4% ]

19. Benjamin Laxton, Jongwoo Lim, David Kriegman, “Leveraging Temporal, Contextual and Ordering

Constraints for Recognizing Complex Activities In Video”, IEEE Conference on Computer Vision and

Pattern Recognition (CVPR), 2007, pp. 1-8 [ acceptance rate 27.5%, 78 citations ]

20. Victor Ng-Thow-Hing, Thor List, Kris Thorisson, Jongwoo Lim, Joel Wormer, “Design and

Evaluation of Communication Middleware in a Humanoid Robot Architecture”, IROS 2007 Workshop

on Measures and Procedures for the Evaluation of Robot Architectures and Middleware, Oct. 29,

2007, San Diego, CA, 2007

21. Victor Ng-Thow-Hing, Jongwoo Lim, Joel Wormer, Ravi Kiran Sarvadevabhatla, Carlos Rocha,

Kikuo Fujimura, Yoshiaki Sakagami, “The memory game: Creating a human-robot interactive

scenario for ASIMO”, The 2008 IEEE/RSJ International Conference on Intelligent Robots and

Systems (IROS), 2008, pp. 779-786

22. Jongwoo Lim, “Optimized Projection Pattern Supplementing Stereo Systems”, 2009 IEEE

International Conference on Robotics and Automation (ICRA), 2009, pp. 2823 - 2829

4

23. Manmohan K. Chandraker, Jongwoo Lim, David J. Kriegman, “Moving in Stereo: Efficient

Structure and Motion Using Lines”, 12th IEEE International Conference on Computer Vision

(ICCV), 2009, pp 1741 - 1748 [ acceptance rate ˜24%, 29 citations ]

24. Brian Clipp, Christopher Zach, Jongwoo Lim, Jan-Michael Frahm, Marc Pollefeys, “Adaptive,

Real-Time Visual Simultaneous Localization and Mapping”, IEEE Workshop on Applications of

Computer Vision (WACV), 2009, pp. 1-8

25. Vivek Pradeep, Jongwoo Lim, “Egomotion using Assorted Features”, IEEE Conference on

Computer Vision and Pattern Recognition (CVPR), 2010, pp. 1514 - 1521 [ acceptance rate 26.8% ]

26. Brian Clipp, Jongwoo Lim, Jan-Michael Frahm, Marc Pollefeys, “Parallel, Real-Time Visual

SLAM ”, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2010

[ 21 citations ]

27. Jongwoo Lim, Jan-Michael Frahm, Marc Pollefeys, “Online Environment Mapping”, IEEE

Conference on Computer Vision and Pattern Recognition (CVPR), 2011 [ acceptance rate 26.4% ]

28. Ananth Ranganathan, Jongwoo Lim, “Visual Place Categorization in Maps”, 2011 IEEE/RSJ

International Conference on Intelligent Robots and Systems (IROS), 2011

29. Christian Hane, Christopher Zach, Jongwoo Lim, Ananth Ranganathan, Marc Pollefeys, “Stereo

Depth Map Fusion for Robot Navigation”, 2011 IEEE/RSJ International Conference on Intelligent

Robots and Systems (IROS), 2011

30. Mark Colbert, Jean-Yves Bouguet, Jeff Beis, Spudde Childs, Daniel Filip, Luc Vincent, Jongwoo

Lim, Scott Satkin, “Building indoor multi-panorama experiences at scale”, ACM SIGGRAPH 2012

Talks, Article No. 24, 2012

31. Jin Han Lee, Guoxuan Zhang, Jongwoo Lim, Il Hong Suh, “Place Recognition using Straight Lines

for Vision-Based SLAM ”, 2013 IEEE International Conference on Robotics and Automation, 2013

32. Yi Wu, Jongwoo Lim*, Ming-Hsuan Yang, “Online Object Tracking: A Benchmark”, IEEE

Conference on Computer Vision and Pattern Recognition (CVPR), 2013

[ corresponding author, acceptance rate 28.4%, 46 citations ]

33. Hyon Lim, Jongwoo Lim*, H. Jin Kim, “Real-Time 6-DOF Monocular Visual SLAM in Large-Scale

Environments”, 2014 IEEE International Conference on Robotics and Automation (ICRA), 2014

34. Jin Han Lee, Sehyung Lee, Guoxuan Zhang, Jongwoo Lim, Wan Kyun Chung, Il Hong Suh,

“Outdoor Place Recognition in Urban Environments Using Straight Lines”, 2014 IEEE International

Conference on Robotics and Automation, 2014

35. Jinshan Pan, Jongwoo Lim, Zhixun Su, Ming-Hsuan Yang, “L0-Regularized Object Representation

for Visual Tracking,” British Machine Vision Conference (BMVC), 1–12, 2014

5

36. Hyon Lim, Jongwoo Lim, H. Jin Kim, “Online 3D Reconstruction and 6-DoF Pose Estimation for

RGB-D Sensors,”, Computer Vision in Vehicle Technology with Special Session in Micro Aerial

Vehicles (ECCV 2014 workshop), 1–15, 2014

37. Jongwoo Lim, Bohyung Han, “Generalized Background Subtraction using Superpixels with Label

Integrated Motion Estimation,” European Conference on Computer Vision (ECCV), 173–187, 2014

38. Juhan Bae, Youngbae Hwang, Jongwoo Lim, “Semi-Online Video Stabilization Using Probabilistic

Keyframe Update and Inter-Keyframe Motion Smoothing,” International Conference on Image

Processing (ICIP), 1–5, 2014

39. Ju Hong Yoon, Ming-Hsuan Yang, Jongwoo Lim, Kuk-Jin Yoon, “Bayesian Multi-Object Tracking

Using Motion Context from Multiple Objects,” Winter Conference on Applications on Computer

Vision (WACV), 1–8, 2015

40. Sehyung Lee, Jongwoo Lim, Il Hong Suh, “Incremental Learning from a Single Seed Image for

Object Detection,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),

1905–1912, 2015

41. Euntae Hong, Juhan Bae, Jongwoo Lim, “Robust visual tracking through deep learning-based

confidence evaluation,” International Conference on Ubiquitous Robots and Ambient Intelligence

(URAI), 581–584, 2015

42. Yuankai Qi, Shengping Zhang, Lei Qin, Hongxun Yao, Qingming Huang, Jongwoo Lim,

Ming-Hsuan Yang, “Hedged Deep Tracking,” IEEE Conference on Computer Vision and Pattern

Recognition (CVPR), 4303–4311, 2016

43. Shun Zhang, Yihong Gong, Jia-Bin Huang, Jongwoo Lim, Jinjun Wang, Narendra Ahuja,

Ming-Hsuan Yang, “Tracking Persons-of-Interest via Adaptive Discriminative Features,” European

Conference on Computer Vision (ECCV), 415–433, 2016

44. Hyunchul Yang, Jongwoo Lim, Sung-eui Yoon*, “Anytime RRBT for Handling Uncertainty and

Dynamic Objects,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),

4786–4793, 2016

45. Sehyung Lee, Jongwoo Lim, Il Hong Suh*, “Key-frame Selection based Online Object Learning and

Detection,” IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS),

5272–5278, 2016

Patents

1. Jeffrey Ho, Jongwoo Lim, Ming-hsuan Yang, “Image clustering with metric, local linear structure,

and affine symmetry”, [7,248,738] Jul 2007, [10/989,967] Nov 2004

6

2. Ming-hsuan Yang, Jongwoo Lim, David Ross, Ruei-sung Lin, “Adaptive probabilistic visual tracking

with incremental subspace update”, [7,463,754] Dec 2008, [10/989,966] Nov 2004

3. Ming-hsuan Yang, Jongwoo Lim, David Ross, Takahiro Ohashi, “Method, apparatus and program

for detecting an object”, [7,224,831] May 2007, [10/858,878] Jun 2004

4. Ming-hsuan Yang, Jongwoo Lim, David Ross, Takahiro Ohashi, “Face recognition system”,

[7,430,315] Sep 2008, [10/858,930] Jun 2004

5. Ming-hsuan Yang, Ruei-sung Lin, Jongwoo Lim, David Ross, “Adaptive discriminative generative

model and application to visual tracking”, [7,369,682] May 2008, [11/179,881] Jul 2005

6. Jongwoo Lim, Ming-hsuan Yang, “Direct method for modeling non-rigid motion with thin plate

spline transformation”, [7,623,731] Nov 2009, [11/450,045] Jun 2006

7. Jongwoo Lim, Benjamin Laxton, “Leveraging Temporal, Contextual and Ordering Constraints for

Recognizing Complex Activities in Video”, [8,165,405] Apr 2012, [11/876,724] Oct 2007

8. Jongwoo Lim, “Camera-Projector Duality: Multi- Projector 3D Reconstruction”, [8,172,407] May

2012, [12/121,056] May 2008

9. Jongwoo Lim, “Optimized Projection Pattern for Long-Range Depth Sensing”, [8,142,023] Mar

2012, [12/337,391] Dec 2008

10. Manmohan Chandraker, Jongwoo Lim, “Structure and Motion with Stereo Using Lines”,

[8,401,241] Mar 2013, [12/506,560] Jul 2009

11. Jongwoo Lim, Vivek Pradeep, “Egomotion Using Assorted Features”, [8,855,406] Oct 2014,

[13/219,184] Aug 2011

12. Rakesh Gupta, Ananth Ranganathan, Jongwoo Lim, “Road Departure Warning System”,

[9,077,958] Jul 2015, [13/221,132] Aug 2011

13. Jongwoo Lim, Jan-Michael Frahm, Marc Pollefeys, “Online Environment Mapping”, [8,913,055]

Dec 2014, [13/487,122] May 2012

14. Jongwoo Lim, “Apparatus and Method for Feature Matching in Distorted Images”, [8,818,101] Aug

2014, [13/342,644] Jan 2012

Professional Activities

7

* Area Chair of Asian Conference on Computer Vision (ACCV) 2016, Taiwan.

* Area Chair of IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2016, Las Vegas.

* Local Chair of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2016,

Daejeon, Korea.

* Area Chair of Asian Conference on Computer Vision (ACCV) 2014, Singapore.

* 한국정보과학회 이사 및 학술사업위원(국제2) (2014.03 - 2015.02)

* 한국정보과학회 인공지능 소사이어티 국제부 이사 (2013.08 - )

* 한국자동차공학회 전기·전자·ITS부문 이사 (2013.08 - )

* 한국정보과학회 ‘정보과학회 논문지: 소프트웨어 및 응용’ 편집위원 (2013.03 - 2015.02)

* Co-organizer and lecturer of the tutorial on ‘RGBD Image Processing for 3D Modeling and Texturing’ at

ICIP 2013, Melbourne, Australia.

* Local Chair of Asian Conference on Computer Vision (ACCV) 2012, Daejeon, Korea.

8