Carleton Comps Project: 3D Scanning on the Cheap

Final Results

3D Scanning on the Cheap

Advisor: Jadrian Miles

Background

There are a variety of communities interested in producing high-fidelity 3D models of real-world objects: video-game designers, archaeologists, movie studios, biomedical engineers, 3D printing hobbyists, and many more. Some of these groups have access to dedicated 3D scanners, using highly calibrated lasers and all sorts of other expensive equipment. At the other end of things, online services exist that attempt to reconstruct a model of your object from a handful of photos that you send to them. The results are sometimes serviceable, but the process is somewhat unpredictable.

Perhaps there's a middle ground. The Microsoft Kinect is accessible to consumers, and has a depth camera as well as a video camera. What if we could just wave a Kinect camera around an object, send the data to a program, and get out a highly detailed model of the object, complete with its surface appearance painted right on the model?

3D scanning is an active research area and a rather difficult computational problem. Your project is to dive in head-first, and attempt, through a mix of your own ingenuity and your reading of the efforts of others (that is, academic papers), to make high-quality 3D scanning available to the hobbyist on a budget.

The Project

Your objective is to create a program that can take some kind of input from a Kinect and produce an accurate and precise model of the 3D surface structure and appearance of a real-world object.

Suggested Scan Subjects

From simplest to most difficult: globe, candlestick, cereal box, big wooden serving fork, painted bowl, coffee cup, egg carton, sculptural bust, model airplane.

Baseline Features

With the Kinect camera in a fixed position, and your object perfectly in the center of a turntable, rotate the turntable to some number of precise predetermined angles, taking multiple snapshots at each angle with the Kinect. From this, reconstruct a basic model of the 3D shape of the object.

Advanced Features

Use information from the continuous rotation (optionally at a highly calibrated speed) of the turntable.
Handle objects not placed in the center of the turntable.
Reconstruct the surface appearance of the object.
Use super-resolution techniques to get a high-quality appearance model.
Use super-resolution techniques to get a high-quality shape model.
Use topological filtering or other computational-geometry techniques to fix errors in the shape model.
Rather than rotating the object in front of a fixed camera, reconstruct from the input of a freely-moving camera. One way to make this easier is to build a “calibration cage” with easily distinguished corner points, within which you will place the object to be scanned. This way, the camera can reconstruct its position relative to the known position of the cage.

Both 3D scanning and the abstract operations involved in it are active topics of research within the field of “computational geometry”. Super-resolution, appearance modeling, and “multi-view scene reconstruction” are topics widely investigated in the computer vision and graphics research communities. Doing a good job on this project will involve reading up on and implementing published techniques.