+ All Categories
Transcript
Page 1: Seminario Ruggero Pintus, 4-10-2012

www.crs4.it/vic/

TecnologieTecnologie didi Visual ComputingVisual Computingper per ii BeniBeni CulturaliCulturaliper per ii BeniBeni CulturaliCulturali

R. PintusCRS4 Visual Computing

Page 2: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

TecnologieTecnologie per per ii benibeni culturaliculturali

• Focus: digitalizzazione accurata (forma e colore) di siti e manufatti + …– Partire dai dati: Acquisizione -> Trattamento !

– Modelli misurabili

• Molti usi oltre la visualizzazione• Molti usi oltre la visualizzazione

– Riproduzione materica

– Studio di opere d’arte

– Documentazione in-situ di scavi archeologici

– Supporto al restauro e sua documentazione

– Valorizzazione

Page 3: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

TecnologieTecnologie per per ii benibeni culturaliculturali

• Le quantità di dati prodotte dai moderni sensori sono però difficili da trattare, archiviare, distribuire, visualizzare– Scalabilità!

• Tecniche attuali sub-ottimali• Tecniche attuali sub-ottimali– Costi, tempi, qualità

• Bisogno di ricerca in tecnologie abilitanti scalabili– Acquisizione

– Processamento geometrico

– Visualizzazione

– …

Page 4: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Tecnologie per i beni culturaliTecnologie per i beni culturali

• Come acquisire e processare efficacemente forma e colore di siti e manufatti?siti e manufatti?– Tecniche di fusione multi-sensore, stream-processing, multiresolution, external

memory algorithms, parallel programming, GPGPUs

• Come archiviare e distribuire efficacemente i modelli?– Multiresolution, adaptive streaming, compression

• Come visualizzarli efficacemente?– Multiresolution, adaptive rendering, out-of-core methods, GPU programming,

parallelization, rasterization, ray-casting

• Come esplorarli?– Novel 3D displays, specific interaction techniques

– Portable devices

Page 5: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

AlcuniAlcuni esempiesempi

• Allineamento geometria/colore

• Colorazione di modelli 3D

• Fusione di dati e ricostruzione geometrica

• Visualizzazione scalabile ed interattiva

• Distribuzione di dati in rete• Distribuzione di dati in rete

• Esplorazione su display innovativi

(… e molto altro)

Page 6: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Our GoalOur Goal

6

Page 7: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Modelling vs AcquisitionModelling vs Acquisition

ModellingSubjective Reality

7

AcquisitionObjective reality

Page 8: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

3D Reconstruction3D Reconstruction

• Acquire geometry and color• A lot of techniques

– Structured light, laser scanning (triangulation or time-of-flight), photometric stereo, shape-from-X, …

• Which technique?

8

• Which technique?– Object type (big/small, material….) – Cost– Accuracy/Resolution– Time– Complexity

Page 9: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

OutlineOutline

• 3D Reconstruction Techniques

• 3D Reconstruction Pipeline – Photo mapping/blending

– Printing

9

– Printing

• Case study

Page 10: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

10

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 11: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

11

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 12: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

12

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 13: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– StereoStereo

13

Page 14: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– StereoStereo

14

Page 15: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

15

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 16: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– MultiviewMultiview

16

Page 17: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– MultiviewMultiview

17

Page 18: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

18

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 19: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– SilhouettesSilhouettes

19

Page 20: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

20

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 21: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– Depth from Depth from focus/defocusfocus/defocus

21

Page 22: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

22

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 23: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– TransmissiveTransmissive

Computed Tomography

23

Density Function

Trasmissive Ultrasound

Page 24: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

24

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 25: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– NonNon--OpticalOptical

25

Ultrasound Radar MRI

Page 26: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

26

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 27: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– TimeTime--ofof--FlightFlight

27

nssm

m

c

dt 17

103

0.528 ≈

×==

Page 28: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

28

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 29: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– Laser StripingLaser Striping

29

Page 30: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

30

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 31: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– Structured LightingStructured Lighting

31

Page 32: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy (nonTaxonomy (non--destructive)destructive)

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

32

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 33: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– Photometric StereoPhotometric Stereo

33

Page 34: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Photometric Stereo Photometric Stereo –– SEMSEM

34

Page 35: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy Taxonomy –– Photometric StereoPhotometric Stereo

35

Page 36: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Taxonomy SEMTaxonomy SEM

• Contact– Direct Measurements

• rulers, calipers, pantographs, coordinate measuring machines (CMM), AFM

• Non-Contact

36

– Passive• Shape-from-X

– Stereo

– Multiview

– Silhouettes

– Focus/Defocus

– Active• Transmissive

– Computed Tomography (CT)

– Transmissive Ultrasound

• Reflective– Non-Optical Methods

» reflective ultrasound, radar, sonar, MRI

– Time-of-Flight

– Triangulation

» laser striping

» structured lighting

– Photometric Stereo

Page 37: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Cultural HeritageCultural Heritage

• Techniques– Triangulation (laser scanner)

– Time of Flight

– Texture Mapping

– Multi-view reconstruction

– Photometric Stereo

37

– Photometric Stereo

• Deal with multiple acquisitions

• Manage a huge amount of data for visualization purposes

Page 38: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

3D Reconstruction Pipeline3D Reconstruction PipelineReal Object Acquisition Devices

Photos

38

3D Digital Model

=== Processing ===-Cleaning- Merging

- Photo Alignment- Color Projection

- …

Geometry

Page 39: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

3D Reconstruction Pipeline3D Reconstruction Pipeline

• Real Model Inspection (onsite)

• Scans design (offsite/onsite)

• Acquisition (onsite)

• Alignment (offsite)

39

• Editing (offsite)

• Merge (offsite)

• Texture (offsite)

• Final Model (offsite)

• 3D Printing (offsite)

Page 40: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

3D Reconstruction Pipeline3D Reconstruction Pipeline

• Real Model Inspection (onsite)

• Scans design (offsite/onsite)

• Acquisition (onsite)

• Alignment (offsite)

40

• Editing (offsite)

• Merge (offsite)

• Texture (offsite)

• Final Model (offsite)

• 3D Printing (offsite)

Page 41: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

GoalGoal

• Fast and low-cost technique for creating accurate colored models

• Acquisition – 3D – laser scanners

– Color – digital cameras– Color – digital cameras

• Mapping photo-to-geometry– Fast and Robust Semi-Automatic Registration of Photographs

to 3D Geometry

• Photo blending– A Streaming Framework for Seamless Detailed Photo

Blending on Massive Point Clouds

Page 42: Seminario Ruggero Pintus, 4-10-2012

www.crs4.it/vic/

Photo MappingPhoto MappingPhoto MappingPhoto Mapping

Ruggero Pintus, Enrico Gobbetti, and Roberto Combet. “Fast and Robust Semi-Automatic Registration of Photographs to 3D Geometry”. In The 12th International Symposium on Virtual Reality, Archaeology and Cultural Heritage, October 2011.

Page 43: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Problem StatementProblem Statement

3D Geometry Unordered SetOf n Uncalibrated

Photos

n Camera Poses(2D/3D Registration)

Page 44: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Related workRelated work

• Manual selection of 2D-3D matches– Massive user intervention – Tiring and time-consuming

• Automatic feature matching– Not robust enough for a generic dataset

• Semi-automatic statistical correlation• Semi-automatic statistical correlation– Point cloud attributes not always provided

• Geometric multi-view reconstruction– 2D-3D problem � 3D-3D registration task

– dense and ordered frame sequence

• Our contribution– Minimize user intervention / Large datasets / Semi-

automatic / Multi-view based approach / No Attributes

Page 45: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Input DataInput Data

• Dense Geometry– Point cloud, triangle

mesh, etc.

– No attributes

– No particular features

User

SfM Reconstruction

Dense 3D n Photos

– No particular features

• n photos– Naïve constraints:

• Blur, Noise, Under- or over-exposured

– Sufficient overlap

Coarse Registration

Refinement

Output Data

Page 46: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

MultiMulti--viewview

• Bundler [Snavely et al. 2006]– SfM system for unordered

image collections

– http://phototour.cs.washington.edu/bundler/

User

SfM Reconstruction

Dense 3D n Photos

n.edu/bundler/

• Output– A sparse point cloud

– n camera poses

– SIFT keypoints (projections of sparse 3D points)

Coarse Registration

Refinement

Output Data

Page 47: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Coarse registrationCoarse registration

• Register two point clouds with different:

– scales

– reference frames

– resolutions

• Automatic methods are not

User

SfM Reconstruction

Dense 3D n Photos

• Automatic methods are not robust and efficient enough

• User aligns few images (one or more) to the dense geometry

• Affine transformation is applied to all cameras and sparse points

Coarse Registration

Refinement

Output Data

Page 48: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

RefinementRefinement

User

SfM Reconstruction

Dense 3D n Photos1C

jpjs ,1

js ,2

( )jpCQ ,2

jP

( )jF pNN

Coarse Registration

Refinement

Output Data

2C

js ,2

( )( )jF pNNCQ ,2

( ) ( )( )∑∑= =

−=P CN

j

N

ijijFiij spNNCQvPCE

1 1

2

,,,

Page 49: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

RefinementRefinement

• Sparse Bundle Adjustment (SBA)– Constants – SIFT keypoints,

dense 3D points

– Variables – Camera poses, sparse 3D points

User

SfM Reconstruction

Dense 3D n Photos

sparse 3D points

– SBA

• A Generic SBA C/C++ Package Based on the Levenberg-Marquardt Algorithm

• http://www.ics.forth.gr/~lourakis/sba/

Coarse Registration

Refinement

Output Data

Page 50: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Output dataOutput data

• n camera poses

• Input of photo blending

User

SfM Reconstruction

Dense 3D n Photos

blending– n photos

– n camera poses

– Dense 3D geometry

Coarse Registration

Refinement

Output Data

Page 51: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Results Results –– Photo mappingPhoto mapping

Page 52: Seminario Ruggero Pintus, 4-10-2012

www.crs4.it/vic/

Photo BlendingPhoto BlendingPhoto BlendingPhoto Blending

Ruggero Pintus, Enrico Gobbetti, and Marco Callieri. A Streaming Framework for Seamless Detailed Photo Blending on Massive Point Clouds. In Proc. Eurographics Area Papers. Pages 25- 32, 2011.

Page 53: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Problem StatementProblem Statement

Point Cloud CalibratedPhotos

Page 54: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Problem StatementProblem Statement

Point Cloud CalibratedPhotos

P

Page 55: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Problem StatementProblem Statement

Point Cloud CalibratedPhotos

P

ColoredPoint Cloud

Page 56: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Problem StatementProblem Statement

Point Cloud CalibratedPhotos

P

ColoredPoint Cloud

• Problem ���� Unlimited size of 3D model (Gpoints) and unlimited number of images

Page 57: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Related workRelated work

• State-of-the-art techniques

– Image quality estimation

– Stitching or blending

• Data representation

– Triangle meshes – exploit connectivity

– Meshless approaches– Meshless approaches

• Both triangle meshes and point clouds

• Memory settings

– All in-core – no massive geometry/images

– 3D in-core and images out-of-core – no massive geometry

– All out-of-core – Low performances

• Our contribution– Blending function / Streaming framework / Massive point cloud /

Adaptive geometry refinement

Page 58: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

PipelinePipeline

Photo StencilPer-pixelWeight

MaskedPer-pixelWeight

Page 59: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Simple blendingSimple blending

Page 60: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Edge extraction and Distance Edge extraction and Distance TransformTransform

Page 61: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Smooth weightSmooth weight

Page 62: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Smooth weightSmooth weight

Page 63: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Single band blendingSingle band blending

Page 64: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Multi band blendingMulti band blending

Page 65: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Adaptive point refinementAdaptive point refinement

Page 66: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Adaptive point refinementAdaptive point refinement

Page 67: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Adaptive point refinementAdaptive point refinement

Page 68: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Adaptive point refinementAdaptive point refinement

Page 69: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ResultsResults

• Callieri et. al 2008 – David 28M

– Disk space occupancy –6.2GB

– Computation time – 15.5 hours

David470Mpoints

– Computation time – 15.5 hours

Page 70: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Results Results –– Church’s ApseChurch’s Apse

14 Mpoint Geometry 40 photos

Page 71: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Results Results –– Church’s ApseChurch’s Apse

Page 72: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Results Results –– Grave Grave

8 Mpoint Geometry21 photos

Page 73: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Results Results –– Grave Grave

Page 74: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ResultsResults

Page 75: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ResultsResults

Page 76: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ResultsResults

David470Mpoints470Mpoints

Image size – 19456x532481Gpixel

Page 77: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ResultsResults

Page 78: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ConclusionConclusion

• Image-to-geometry registration approach

• Minimum user intervention

• No constraints on geometry, attributes and features

• Specific robust cost function and SBA

• Out-of-core photo blending approach (Point clouds of unlimited size)

• Incremental color accumulation (Unlimited number of images)• Incremental color accumulation (Unlimited number of images)

• Smooth weight function (Seamless color blending)

• Streaming framework (Performance improvement)

• Adaptive point refinement

• Future work

– Automatic sparse-to-dense geometry registration

– Interactive blending - adding and removing images in an interactive tool

– Fast visual check of previous alignment step

Page 79: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ConclusionConclusion

• Low cost

– Personal computer

– Digital camera

– Decreased manual intervention

• Open Source / Free Software

– Bundler – SfM reconstruction –http://phototour.cs.washington.edu/bundler/http://phototour.cs.washington.edu/bundler/

– Sparse Bundle Adjustment – SBA – Minimization –http://www.ics.forth.gr/~lourakis/sba/

– Opengl / GLSL shaders – Rendering – http://www.opengl.org/

– Qt – Interface – http://qt.nokia.com/

– Opencv – Manual registration – http://opencv.willowgarage.com/wiki/

– Spaceland Library – Geometric computation –http://spacelib.sourceforge.net/

– IIPImage – Web-based Viewer – http://iipimage.sourceforge.net/

Page 80: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

3D Printing3D Printing

80

Page 81: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Printing ProcessPrinting Process

• Original model

• Slice representation

81

representation

• Layer by layer deposition

• Cleaning

• Printed model

Page 82: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Printing ProcessPrinting Process

• Original model

• Slice representation

82

representation

• Layer by layer deposition

• Cleaning

• Printed model

Page 83: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Printing ProcessPrinting Process

• Original model

• Slice representation

83

representation

• Layer by layer deposition

• Cleaning

• Printed model

Page 84: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Printing ProcessPrinting Process

• Original model

• Slice representation

84

representation

• Layer by layer deposition

• Cleaning

• Printed model

Page 85: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Printing ProcessPrinting Process

• Original model

• Slice representation

85

representation

• Layer by layer deposition

• Cleaning

• Printed model

Page 86: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Geometry processingGeometry processing

86

Page 87: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Geometry processingGeometry processing

87

Page 88: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Geometry processingGeometry processing

88

Page 89: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

SubSub--surface scatteringsurface scattering

89

Page 90: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Color Color enhancementenhancement

90

Page 91: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Color Color enhancementenhancement

91

Page 92: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Color Color enhancementenhancement

92

Page 93: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ConclusioniConclusioni

• Lavorare su dati misurati è un pre-requisito di molti lavori (tutti?) nel contesto dei beni culturali– Applicazioni specialistiche o per grande pubblico

• Le moderne tecnologie di acquisizione • Le moderne tecnologie di acquisizione consentono di acquisire una grande quantità di informazioni (forma e colore)– Laser scanning, camere digitali, ecc.

• Uso potenziale vasto!– Valorizzazione, restauro, studio, ecc.

Page 94: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ConclusioniConclusioni

• Queste quantità di dati sono però difficili da trattare, archiviare, distribuire, visualizzare– Scalabilità!

• Tecniche attuali sub-ottimali• Tecniche attuali sub-ottimali– Costi, tempi, qualità

Page 95: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ConclusioniConclusioni

• Il CRS4 è impegnato in attività di ricerca per migliorare le tecnologie…– Stato dell’arte internazionale

– Collaborazioni e ricadute locali

• PMI, Contro Restauro SS, Soprintendenze, CNR, UniCA• PMI, Contro Restauro SS, Soprintendenze, CNR, UniCA

• … e per applicarle a casi concreti– Collaborazioni multidisciplinari!

Page 96: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

ConclusioniConclusioni

96

Page 97: Seminario Ruggero Pintus, 4-10-2012

R. Pintus – CRS4/ViC, October 2012

Questions & ContactsQuestions & Contacts

• CRS4 – VIC www.crs4.it/vic/

• Ruggero Pintus • Ruggero Pintus [email protected]


Top Related