Home >Documents >CMSC426 Computer Vision · PDF file Point to Point Iterative Closest Point P2P-ICP 4/24/2018...

CMSC426 Computer Vision · PDF file Point to Point Iterative Closest Point P2P-ICP 4/24/2018...

Date post:18-Oct-2020
Category:
View:1 times
Download:0 times
Share this document with a friend
Transcript:
  • z

    CMSC426 Computer Vision SEXY SEMANTIC MAPPING

    Nitin J. Sanket

    4/24/2018 1

  • z

    Who Am I?

    4/24/2018 2

    Nitin J. Sanket

    • BE in Electronics and Communication from India • MS in Robotics from UPenn • PhD Candidate in Computer Science at UMD

  • z

    Perception and Robotics Group

    4/24/2018 3

    prg.cs.umd.edu 11 PhDs5 Masters 3 Undergrads

  • z

    What do we work on?

    4/24/2018 4

  • z

    Sexy Semantic Mapping

    4/24/2018 5

  • z

    Sexy Semantic Mapping

    4/24/2018 6

    Ecins, Aleksandrs, Cornelia Fermüller, and Yiannis Aloimonos. "Cluttered scene segmentation using the symmetry constraint." Robotics and Automation (ICRA), 2016 IEEE International Conference on. IEEE, 2016.

  • z

    Sexy Semantic Mapping

    4/24/2018 7

  • z

    Data

    4/24/2018 8

  • z

    Data

    4/24/2018 9

  • z

    Data

    4/24/2018 10

    (𝑟𝑟,𝑔𝑔, 𝑏𝑏) at (𝑢𝑢, 𝑣𝑣)

    (𝑥𝑥,𝑦𝑦, 𝑧𝑧) at (𝑢𝑢𝑢, 𝑣𝑣𝑢)

    Point Cloud: (𝑥𝑥, 𝑦𝑦, 𝑧𝑧, 𝑟𝑟,𝑔𝑔, 𝑏𝑏)

  • z

    Point Cloud from RGB-D

    4/24/2018 11

    Taken from Bhoram Lee’s slides at University of Pennsylvania

    Given all camera parameters (𝑅𝑅, 𝑡𝑡, 𝑓𝑓), find the corresponding points of a RGB and a depth image.

    Point Cloud: (𝑥𝑥, 𝑦𝑦, 𝑧𝑧, 𝑟𝑟,𝑔𝑔, 𝑏𝑏)

  • z

    Point Cloud from RGB-D

    4/24/2018 12

    Recall 𝑢𝑢 𝑓𝑓𝑢𝑢

    = 𝑥𝑥 𝑧𝑧

    1. Compute 3D co-ordinate 𝑋𝑋𝐼𝐼𝐼𝐼 in the 𝐼𝐼𝑅𝑅 camera frame 𝑥𝑥𝐼𝐼𝐼𝐼 = 𝑢𝑢𝑧𝑧/𝑓𝑓𝐼𝐼𝐼𝐼 𝑦𝑦𝐼𝐼𝐼𝐼 = 𝑣𝑣𝑧𝑧/𝑓𝑓𝐼𝐼𝐼𝐼 𝑋𝑋𝐼𝐼𝐼𝐼 = 𝑥𝑥𝐼𝐼𝐼𝐼 𝑦𝑦𝐼𝐼𝐼𝐼 𝑧𝑧𝐼𝐼𝐼𝐼

    2. Transform into the RGB frame 𝑋𝑋𝐼𝐼𝑅𝑅𝑅𝑅 = 𝑅𝑅𝑋𝑋𝐼𝐼𝐼𝐼 + 𝑡𝑡

    3. Re-project them into the image plane 𝑢𝑢𝐼𝐼𝑅𝑅𝑅𝑅 = 𝑓𝑓𝐼𝐼𝑅𝑅𝑅𝑅 𝑥𝑥

    𝑅𝑅𝑅𝑅𝑅𝑅

    𝑧𝑧𝑅𝑅𝑅𝑅𝑅𝑅 𝑣𝑣𝐼𝐼𝑅𝑅𝑅𝑅 = 𝑓𝑓𝐼𝐼𝑅𝑅𝑅𝑅 𝑦𝑦

    𝑅𝑅𝑅𝑅𝑅𝑅

    𝑧𝑧𝑅𝑅𝑅𝑅𝑅𝑅 4. Read (𝑟𝑟,𝑔𝑔, 𝑏𝑏) at 𝑢𝑢, 𝑣𝑣 𝐼𝐼𝑅𝑅𝑅𝑅

    𝑟𝑟,𝑔𝑔, 𝑏𝑏 is the color of 𝑋𝑋𝐼𝐼𝐼𝐼 point.

    This is implemented in depthToCloud_full_RGB.p given to you.

  • z

    Obtain ROI by Filtering

    4/24/2018 13

  • z

    Maintain ROI by Mean-Shift Tracking

    4/24/2018 14

  • z

    Remove the Table and wall using RANSAC

    4/24/2018 15

    𝑎𝑎𝑥𝑥 + 𝑏𝑏𝑦𝑦 + 𝑐𝑐𝑧𝑧 + 𝑒𝑒 = 0 𝑒𝑒𝑟𝑟𝑟𝑟 = 𝑎𝑎𝑥𝑥 + 𝑏𝑏𝑦𝑦 + 𝑐𝑐𝑧𝑧 + 𝑒𝑒

    𝑒𝑒𝑟𝑟𝑟𝑟 ≤ 𝜏𝜏

  • z

    Plane Removal Output

    4/24/2018 16

  • z

    ROI Output

    4/24/2018 17

  • z

    Iterative Closest Point

    4/24/2018 18

  • z

    Point to Point Iterative Closest Point P2P-ICP

    4/24/2018 19

    Use KDTreeSearcher or knnsearch for point to point correspondence search.

    Arun, K. Somani, Thomas S. Huang, and Steven D. Blostein. "Least- squares fitting of two 3-D point sets." IEEE Transactions on pattern analysis and machine intelligence 5 (1987): 698-700.

  • z

    Point to Point Iterative Closest Point P2P-ICP

    4/24/2018 20

  • z

    Point to Point Iterative Closest Point P2P-ICP

    4/24/2018 21

  • z

    Point to Plane Iterative Closest Point P2Pl-ICP

    4/24/2018 22

    pcnormals

    Low, Kok-Lim. "Linear least-squares optimization for point-to-plane icp surface registration." Chapel Hill, University of North Carolina 4 (2004).

  • z

    Point to Plane Iterative Closest Point P2Pl-ICP

    4/24/2018 23

    P2Pl P2P

  • z

    Reconstructed Model

    4/24/2018 24

  • z

    Option 1: Segmenting Scene

    4/24/2018 25

  • z

    Option 1: Segmenting Scene

    4/24/2018 26

    Use ICP to match single object point cloud to 3D scene. Use 3D Match to match in 3D.

    http://3dmatch.cs.princeton.edu

  • z

    Option 2: Semantic Map

    4/24/2018 27

    Teddy is bigger than penguin by 18% in volume. Teddy is to the right of penguin.

    Xiang, Yu, and Dieter Fox. "DA-RNN: Semantic mapping with data associated recurrent neural networks." arXiv preprint arXiv:1703.03098 (2017).

  • z

    Thank you!

    4/24/2018 28

    CMSC426 �Computer Vision Who Am I? Perception and Robotics Group What do we work on? Sexy Semantic Mapping Sexy Semantic Mapping Sexy Semantic Mapping Data Data Data Point Cloud from RGB-D Point Cloud from RGB-D Obtain ROI by Filtering Maintain ROI by Mean-Shift Tracking Remove the Table and wall using RANSAC Plane Removal Output ROI Output Iterative Closest Point Point to Point Iterative Closest Point�P2P-ICP Point to Point Iterative Closest Point�P2P-ICP Point to Point Iterative Closest Point�P2P-ICP Point to Plane Iterative Closest Point�P2Pl-ICP Point to Plane Iterative Closest Point�P2Pl-ICP Reconstructed Model Option 1: Segmenting Scene Option 1: Segmenting Scene Option 2: Semantic Map Thank you!

Click here to load reader

Reader Image
Embed Size (px)
Recommended