Post on 26-Jul-2020
transcript
Stereo SLAM
Davide Migliore, PhD migliore@elet.polimi.it
Department of Electronics and Information, Politecnico di Milano, Italy
Monday, 15 June 2009
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Do you remember the pin-hole camera?
2
Monday, 15 June 2009
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
- Each camera has a P matrix
3
Monday, 15 June 2009
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
4
Monday, 15 June 2009
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
5
Monday, 15 June 2009
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Two cameras that perceive the world
6
Monday, 15 June 2009
Slide n°
Davide Migliore
What is a Stereo Camera? ‣Error modeling problem
7
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣The idea - Use the Unified Inverse Depth parametrization (Montiel et al.
2006)
- Rectify images and initialize the point using
8
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Measurement Equations
9
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Measurement Equations
10
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)11
Monday, 15 June 2009
Slide n°
PhD Davide Migliore - migliore@elet.polimi.it
Classic EKF SLAM 12
Monday, 15 June 2009
Slide n°
PhD Davide Migliore - migliore@elet.polimi.it
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Monday, 15 June 2009
Slide n°
PhD Davide Migliore - migliore@elet.polimi.it
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
Slide n°
PhD Davide Migliore - migliore@elet.polimi.it
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
Slide n°
PhD Davide Migliore - migliore@elet.polimi.it
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Data Association
DA
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
Slide n°
PhD Davide Migliore - migliore@elet.polimi.it
Classic EKF SLAM
‣Extended Kalman Filter
12
Video Frame
Feature
Detection
FD
Data Association
DA
Feature Initialization
Prediction
Update
SLAM Filter
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Data Association Trouble
13
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Data Association Trouble
14
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Data Association Trouble
15
Monday, 15 June 2009
Slide n°
Davide Migliore
Compatibility 16
Monday, 15 June 2009
Slide n°
Davide Migliore
NN Data Association 17
Monday, 15 June 2009
Slide n°
Davide Migliore
NN Data Association 18
Monday, 15 June 2009
Slide n°
Davide Migliore
Joint Compatibility 19
Monday, 15 June 2009
Slide n°
Davide Migliore
JCBB 20
Monday, 15 June 2009
Slide n°
Davide Migliore
JCBB 21
Monday, 15 June 2009
Slide n°
Davide Migliore
Demo Time‣Switch on Matlab
22
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Joint Compatibility Branch & Bound Results
23
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Results
24
Monday, 15 June 2009
Slide n°
Davide Migliore
Scaling problem 25
Monday, 15 June 2009
Slide n°
Davide Migliore
Scaling problem 26
O(n2)
Monday, 15 June 2009
Slide n°
Davide Migliore
Solution: local maps 27
‣Switch to matlab again
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Results
28
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Paz et al. 2008)
‣Results
29
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
30
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
30
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
31
Monday, 15 June 2009
Slide n°
Davide Migliore
Stereo SLAM (Tomono 2009)
‣Results
32
Monday, 15 June 2009
Slide n°
Davide Migliore
Inverse Scaling?‣ Is it possible to use the inverse scaling?‣Yes
‣Results? Coming soon!!
33
Monday, 15 June 2009
Slide n°
Davide Migliore PhD - migliore@elet.polimi.it
Thanks for your attention34
Monday, 15 June 2009
Slide n°
Davide Migliore PhD - migliore@elet.polimi.it
Thanks for your attention34
Questions
Monday, 15 June 2009
Omnidirectional SLAM
Davide Migliore, PhD migliore@elet.polimi.it
Department of Electronics and Information, Politecnico di Milano, Italy
Monday, 15 June 2009
Slide n°
Davide Migliore
What is an Omni Camera? Omnidirectional sensors come in many varieties, but
by definition must have a wide field-of-view.
~180º FOV
wide FOV dioptric cameras (e.g. fisheye)
~360º FOV
polydioptric cameras (e.g. multiple overlapping cameras)
>180º FOV
catadioptric cameras (e.g. cameras and mirror systems)
Monday, 15 June 2009
Slide n°
Davide Migliore
(Poly-)Dioptric solutions
Pros: - High resolution
per viewing angle
Cons:- Bandwidth
- Multiple cameras
One to two fish-eye cameras or many synchornized cameras
Monday, 15 June 2009
Slide n°
Davide Migliore
(Poly-)Dioptric solutionsOne to two fish-eye cameras or many synchornized
cameras
Homebrewed polydioptric cameras are cheaper, but require calibrating and synchronizing; commercial designs tend to be expensive
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric solutionsUsually single camera combined with convex mirror
Cons:- Blind spots
- Low resolution
Pros: - Single image
Monday, 15 June 2009
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
Slide n°
Davide Migliore
Camera ModelsPerspective camera
Single effective viewpoint
Image plane (CCD)
40
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras• mirror
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Catadioptric cameras• mirror• perspective camera
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Central catadioptric cameras
• mirror
• camera
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Central catadioptric cameras
• mirror
• camera
• single effective viewpoint
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
Central catadioptric cameras
• mirror
• camera
• single effective viewpoint
(surface of revolution of a conic)
Camera Models
Monday, 15 June 2009
Slide n°
Davide Migliore
F1
F2
Types of central catadioptric cameras 43
Monday, 15 June 2009
Slide n°
Davide Migliore
• hyperbola + perspective camera
F1
F2
Types of central catadioptric cameras 43
Monday, 15 June 2009
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
Slide n°
Davide Migliore
• hyperbola + perspective camera• parabola + orthographic lens
• ...F1
F2
F1
Types of central catadioptric cameras 43
Monday, 15 June 2009
Slide n°
Davide Migliore
Other types of central cameras 44
Monday, 15 June 2009
Slide n°
Davide Migliore
Other types of central cameras 44
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Why do we need calibration? 45
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X
Y
Z
p =
Calibration gives the relation between 2D & 3D
For each pixel → 3D vector emanating from the
single viewpoint
Why do we need calibration? 45
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X Y
Z
What?
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image
What?
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image • Camera focal length
Focal length
What?
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image • Camera focal length• Orientation and position between camera & mirror
Focal length
R, T
What?
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
X Y
Z
• Center of the omnidirectional image • Camera focal length• Orientation and position between camera & mirror• Mirror shape
Focal length
R, T
What?
Monday, 15 June 2009
Slide n°
Davide Migliore
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
Slide n°
Davide Migliore
1. Mirror and camera axes are aligned =>
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
Slide n°
Davide Migliore
1. Mirror and camera axes are aligned =>
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
Slide n°
Davide Migliore
1. Mirror and camera axes are aligned =>
2. x-y mirror axes coincide with u-v camera axes =>
u
v
Focal length
R, T
X Y
Z
Assumptions
Monday, 15 June 2009
Slide n°
Davide Migliore
Reflected rays do not intersect in a point but are tangent to a “caustic”
And how about non-central cameras?
Monday, 15 June 2009
Slide n°
Davide Migliore
Reflected rays do not intersect in a point but are tangent to a “caustic”
And how about non-central cameras?
Monday, 15 June 2009
Slide n°
Davide Migliore
Visual Odometry (Scaramuzza et al. 2009)49
Monday, 15 June 2009
Slide n°
Davide Migliore
Omni SFM (Lhuillier et al. 2008)50
Monday, 15 June 2009
Slide n°
Davide Migliore
Omni SFM (Lhuillier et al. 2008)51
Monday, 15 June 2009
Slide n°
Davide Migliore
Omni SFM (Lhuillier et al. 2008)52
Monday, 15 June 2009
Slide n°
Davide Migliore PhD - migliore@elet.polimi.it
Thanks for your attention53
Monday, 15 June 2009
Slide n°
Davide Migliore PhD - migliore@elet.polimi.it
Thanks for your attention53
Questions
Monday, 15 June 2009