3D Scene Reconstruction
Carleton College Computer Science | 2016
Introduction
This project aims to reconstruct a 3D scene using an ordered series of photos around a scene. The process involves the following pipeline.
Pipeline
Select Images
We begin by searching for pictures to triangulate. These images could come from online sources like Flickr or Facebook. Images can also be captured with any digital camera that saves EXIF data.
Feature Detection
Feature Detection is the process identifying points of interest, also called "features" or "keypoints." Using the SIFT algorithm, we find points of high contrast change (dark-to-light) in each photograph.
Feature Matching
Feature matching is the process of finding the corresponding keypoint of one image in the other image. For instance, if keypoint A in image 1 is of the corner of the chapel, what keypoint in image 2 corresponds to the same corner of the chapel.
Pose Estimation
Pose estimation is the process of retrieving the camera pose (orientation and location) of the images' corresponding cameras.
Triangulation
Triangulation is the process of converting a series of matches across a series of photos to 3D points. The final result of this process provides us with a series of 3D points or just a point cloud.
Texture Mapping
The process of covering the point cloud with the appropriate texture (e.g. color) from the series of photos.
Applications
3D Scene Reconstruction has many potential uses. By creating a 3D model, users could virtually tour campuses or cities, investigators could keep spatial records of crime scenes, and architects could maintain models of artifacts.
Code
- Github: https://github.com/CarletonScenes/Scene-Reconstruction/
- Zip: https://github.com/CarletonScenes/Scene-Reconstruction/archive/master.zip
Sources
- Photo Tourism (http://phototour.cs.washington.edu/)
- Lund Institute of Technology Mathematical Imaging Group Computer Vision 2013
- Multiple View Geometry in Computer Vision by Hartley, Richard, Zisserman, Andrew, Cambridge University Press, March 2004
Packages Used
- exifread
- flask
- OpenCV 3 - Open Source Computer Vision library
- Meshlab - 3d mesh processing software
- three.js - Javascript 3d library
Students
Hami Abdi
Hami lives in British Columbia, Canada. He enjoys using computer science and math to solve real world problems. On his free time, he plays the piano, attends the gym, and works on his own side projects. Currently, he sees himself pursuing a field in field related to data mining.
Adam Canady
Adam Canady is from Iowa City, Iowa. He loves solving hard programming problems, building distributed systems, playing lacrosse, rock climbing, and skiing. Adam is also an avid robotics enthusiast.
Raven Pillmann
Raven Pillmann comes from the bustling Twin Cities, Minnesota. He enjoys front-end development and theoretical computer science, but also passionately listens to music, runs, and grabs coffee with friends. He thoroughly enjoys Summit EPA.
Greg Sharpe
Greg Sharpe is from Bethesda, MD. He is a CS-Econ double major but secretly likes CS better. He also enjoys hiking, cooking, and long walks on the beach.
David Pickart
David Pickart hails from the sunny shores of Falmouth, MA. He writes music in his spare time and is an American Music concentrator. His other interests include mobile software development and improv comedy.
Ian Kapelke
Ian Kapelke is from New Brighton, MN. His favorite programming language is C. In his spare time he enjoys biking, cooking, and reading Philip K Dick novels.