Posts Tagged ‘photosynth’

« Older Entries
Newer Entries »

Structure from motion projects

December 20th, 2010

I’ve introduced my tracking algorithm in the previous post. One of the issue I have is that the point cloud generated by my SFMToolkit (using Bundler) is not always accurate. This is a list of structure from motion projects alternative I’m interested in:

Building Rome in a Day:

Project home is using Bundler (GPL)

Building Rome on a Cloudless Day:

Project home | Source code (Non-profit license, I’ve ported their source to windows)

Samantha:

Project home (I’ve contacted them without response but they said that they were going to release the source code: check at 28:50)

Samantha Bundler

PhotoSynth:

Website – Microsoft closed-source SFM application: check out my PhotoSynthToolkit

PhotoSynth Bundler

ETH-V3D Structure-and-Motion software:

Project home with source code (GPL, I’ve partially ported it to windows)

Simple Sparse Bundle Adjustment:

Project home with source code (LGPL, I’ve ported it to windows)

A multi-stage linear approach to structure from motion:

Project home | paper

Results from the paper of LinearSFM (Microsoft Research)

This list is not exhaustive, I’ve seen other projects (Efficient Large Scale Multi-View Stereo for Ultra High Resolution Image Sets: not sure how it is related to ETH-V3D Structure-and-Motion software)

Share

11 comments »

Posted in photogrammetry

Tags:

Augmented Reality outdoor tracking becoming reality

December 13th, 2010

My interest in structure from motion was primary motivated by the capability of creating a point cloud that can be used as a reference for tracking reference. The video below is more a proof-of-concept than a prototype but this is an overview of my outdoor tracking algorithm for Augmented Reality:

Analysis

In a pre-processing step I’ve built a sparse point cloud of the place using my SFMToolkit. Each vertex of the point cloud has several 2D Sift features correspondences. I’ve only kept one Sift descriptor per vertex (mean of the descriptors) and put all descriptors in an index using Flann.

For each frame of the video to be augmented, I’ve extracted Sift feature with SiftGPU and then matched them using Flann 2-nearest neighbor search and a distance ratio threshold. The Flann matching is done in parallel with boost::threadpool. The matches computed contains a lot of outliers. So I have implemented a Ransac pose estimator using EPnP that permits to filter bad 2d/3d correspondences.

Performance

My implementation is slow (due to my implementation of Ransac EPnP that could be improved).

Sift first octave: -1
Sift extraction: 49ms 2917 features
Sift matching: 57ms (parallel matching using Flann)
Ransac EPnP: 110ms 121 inliers of 208 matches
Global: 4.6fps (9.4fps without pose estimation)

Sift first octave: 0
Sift extraction: 32ms 707 features
Sift matching: 15ms (parallel matching using Flann)
Ransac EPnP: 144ms 62 inliers of 93 matches
Global: 5.2fps (21.2fps without pose estimation)

The slowness is not a so big issue because it doesn’t need to run at 30fps. Indeed the goal of my prototype is to have absolute pose with this tracking system each second and relative pose using inertial system available on mobile device (or using KLT tracking).

Issue

  • Performance (faster is better ;-) )
  • Point cloud reference is not always accurate (Bundler fault)

In another post I’ll introduce alternative to Bundler: faster and more accurate.

Share

12 comments »

Posted in ogre3d, photogrammetry, photosynth

Tags:

PhotoSynthToolkit results

November 19th, 2010

This is just a small post to show you want kind of results you can get with my PhotoSynthToolkit:

Download location and source code introduced in my previous post.

 
Share

4 comments »

Posted in ogre3d, photogrammetry

Tags:

PhotoSynth Toolkit updated

November 9th, 2010

Overview

I have updated my PhotoSynth toolkit for easier usage (the same way as SFMToolkit). This is an example of dense mesh creation from 12 pictures using this toolkit:

The 12 pictures were shot with a Canon PowerShot A700:

Thanks to this toolkit, PMVS2 and MeshLab you can create a dense mesh from these 12 pictures:

triangulated mesh with vertex color -> triangulated mesh with vertex color and SSAO -> triangulated mesh shaded with SSAO -> triangulated mesh wireframe -> photosynth sparse point cloud
(sparse point cloud : 8600 vertices, dense point cloud: 417k vertices, mesh: 917k triangles)

 

You can also take a loot at the PhotoSynth reconstruction of the sculpture.

PhotoSynthToolkit is composed of several programs:

  • PhotoSynthDownloader: download PhotoSynth point cloud and cameras parameters
  • PhotoSynth2PMVS: enable to run PMVS2 with a downloaded PhotoSynth point cloud
  • PMVS2 : http://grail.cs.washington.edu/software/pmvs/ created by Yasutaka Furukawa
  • PhotoSynthViewer: Ogre3D PhotoSynth viewer [not working yet]

Download

The source code is available under MIT license on my github. I have also released a win32 binary version with windows scripting (WSH) for easier usage: PhotoSynthToolkit4.zip.

Help

If you need some help or just want to discuss about photogrammetry, please join the photogrammetry forum created by Olafur Haraldsson. You may also be interested by Josh Harle’s video tutorials, they are partially out-dated due to the new PhotoSynthToolkit version but these videos are very good to learn how to use MeshLab.

Please go to the PhotoSynthToolkit page to get the latest version

Share

15 comments »

Posted in ogre3d, photogrammetry

Tags:

Structure From Motion Toolkit released

November 5th, 2010

Overview

I have finally released my Structure-From-Motion Toolkit (SFMToolkit). So what can you do with it ? Let’s say you have a nice place like the one just bellow:

Place de la Bourse, Bordeaux, FRANCE (picture from Bing)
 

Well, now you can take a lot of pictures of the place (around 50 in my case):

 

And then compute structure from motion and get a sparse point cloud using Bundler:

Finally you have a dense point cloud divided in cluster by CMVS and computed by PMVS2:

You can also take a loot at the PhotoSynth reconstruction of the place with 53 pictures and 26 (without the fountain).

This is the SFMToolkit workflow:

SFMToolkit is composed of several programs:

Download

As you can see this “toolkit” is composed of several open-source component. This is why I have decided to open-source my part of the job too. You can download the source code from the SFMToolkit github. You can also download a pre-compiled x64 version of the toolkit with windows scripting (WSH) for easier usage (but not cross-platform): SFMToolkit1.zip.

Help

If you need some help or just want to discuss about photogrammetry, please join the photogrammetry forum created by Olafur Haraldsson. You may also be interested by Josh Harle’s video tutorials, they are partially out-dated due to the new SFMToolkit but these videos are very good to learn how to use MeshLab.

Please go to the SFMToolkit page to get the latest version

Share

19 comments »

Posted in ogre3d, photogrammetry

Tags: