Posts Tagged ‘bundler’

Structure from motion projects

December 20th, 2010

I’ve introduced my tracking algorithm in the previous post. One of the issue I have is that the point cloud generated by my SFMToolkit (using Bundler) is not always accurate. This is a list of structure from motion projects alternative I’m interested in:

Building Rome in a Day:

Project home is using Bundler (GPL)

Building Rome on a Cloudless Day:

Project home | Source code (Non-profit license, I’ve ported their source to windows)

Samantha:

Project home (I’ve contacted them without response but they said that they were going to release the source code: check at 28:50)

Samantha Bundler

PhotoSynth:

Website – Microsoft closed-source SFM application: check out my PhotoSynthToolkit

PhotoSynth Bundler

ETH-V3D Structure-and-Motion software:

Project home with source code (GPL, I’ve partially ported it to windows)

Simple Sparse Bundle Adjustment:

Project home with source code (LGPL, I’ve ported it to windows)

A multi-stage linear approach to structure from motion:

Project home | paper

Results from the paper of LinearSFM (Microsoft Research)

This list is not exhaustive, I’ve seen other projects (Efficient Large Scale Multi-View Stereo for Ultra High Resolution Image Sets: not sure how it is related to ETH-V3D Structure-and-Motion software)

Share

Augmented Reality outdoor tracking becoming reality

December 13th, 2010

My interest in structure from motion was primary motivated by the capability of creating a point cloud that can be used as a reference for tracking reference. The video below is more a proof-of-concept than a prototype but this is an overview of my outdoor tracking algorithm for Augmented Reality:

Analysis

In a pre-processing step I’ve built a sparse point cloud of the place using my SFMToolkit. Each vertex of the point cloud has several 2D Sift features correspondences. I’ve only kept one Sift descriptor per vertex (mean of the descriptors) and put all descriptors in an index using Flann.

For each frame of the video to be augmented, I’ve extracted Sift feature with SiftGPU and then matched them using Flann 2-nearest neighbor search and a distance ratio threshold. The Flann matching is done in parallel with boost::threadpool. The matches computed contains a lot of outliers. So I have implemented a Ransac pose estimator using EPnP that permits to filter bad 2d/3d correspondences.

Performance

My implementation is slow (due to my implementation of Ransac EPnP that could be improved).

Sift first octave: -1
Sift extraction: 49ms 2917 features
Sift matching: 57ms (parallel matching using Flann)
Ransac EPnP: 110ms 121 inliers of 208 matches
Global: 4.6fps (9.4fps without pose estimation)

Sift first octave: 0
Sift extraction: 32ms 707 features
Sift matching: 15ms (parallel matching using Flann)
Ransac EPnP: 144ms 62 inliers of 93 matches
Global: 5.2fps (21.2fps without pose estimation)

The slowness is not a so big issue because it doesn’t need to run at 30fps. Indeed the goal of my prototype is to have absolute pose with this tracking system each second and relative pose using inertial system available on mobile device (or using KLT tracking).

Issue

  • Performance (faster is better ;-) )
  • Point cloud reference is not always accurate (Bundler fault)

In another post I’ll introduce alternative to Bundler: faster and more accurate.

Share

Structure From Motion Toolkit released

November 5th, 2010

Overview

I have finally released my Structure-From-Motion Toolkit (SFMToolkit). So what can you do with it ? Let’s say you have a nice place like the one just bellow:

Place de la Bourse, Bordeaux, FRANCE (picture from Bing)
 

Well, now you can take a lot of pictures of the place (around 50 in my case):

 

And then compute structure from motion and get a sparse point cloud using Bundler:

Finally you have a dense point cloud divided in cluster by CMVS and computed by PMVS2:

You can also take a loot at the PhotoSynth reconstruction of the place with 53 pictures and 26 (without the fountain).

This is the SFMToolkit workflow:

SFMToolkit is composed of several programs:

Download

As you can see this “toolkit” is composed of several open-source component. This is why I have decided to open-source my part of the job too. You can download the source code from the SFMToolkit github. You can also download a pre-compiled x64 version of the toolkit with windows scripting (WSH) for easier usage (but not cross-platform): SFMToolkit1.zip.

Help

If you need some help or just want to discuss about photogrammetry, please join the photogrammetry forum created by Olafur Haraldsson. You may also be interested by Josh Harle’s video tutorials, they are partially out-dated due to the new SFMToolkit but these videos are very good to learn how to use MeshLab.

Please go to the SFMToolkit page to get the latest version

Share

PMVS2 x64 and videos tutorials

September 23rd, 2010

PMVS2 x64

I’ve finally managed to spend a couple of hours to compile a 64 bit version of PMVS2 for windows! You can download PMVS2_x64.zip right now. I’ll hope that this version will help some persons, I’ve personally managed to create a very dense model thanks to this version and PMVS2 was using more than 4Gb of ram on a 8-cores machines.

How to compile PMVS2 x64 by yourself:
download the CMake package of CMVS (containing PMVS) created by Pierre Moulon.
download and compile gsl 1.8
download precompiled pthread x64 lib from equalizer svn
download and compile clapack 3.2.1 using CMake

PhotoSynthTileDownloader

As requested by some persons, I’ve updated my PhotoSynthTileDownloader: you can now resume a partial download ! It’s already available for download: PhotoSynthTileDownloader2.zip.

Videos tutorials

Josh Harle has done some very nice videos tutorials on how to use my PhotoSynth ToolKit and has created another toolkit for Bundler that is using my BundlerMatcher.

PhotoSynth Toolkit post on Josh Harle’s blog.

Note: In fact your synths doesn’t need to be 100% synthy. My tool (PhotoSynth2PMVS) is capable of using an uncomplete synth. And now you could use my 64bit version of PMVS2 instead.

Please go to the PhotoSynthToolkit page to get the latest version


Bundler photogrammetry package post on Josh Harle’s blog.

Please go to the SFMToolkit page to get the latest version

Share

Introducing OpenSynther

September 8th, 2010

PhotoSynthTileDownloader

In my previous post I have released my PhotoSynth Tookit but the PhotoSynth tile downloader wasn’t available yet. You can now download the picture of your Synth in HD using PhotoSynthTileDownloader1.zip. But please respect author’s copyright ! I have included a new confirmation dialog box that warn you about the Synth status (unlisted, public) but this is not shown in the video presentation of PhotoSynthTileDownloader.

BundlerMatcher

I have finally released BundlerMatcher, a feature extraction and picture matching tool built with SiftGPU. The main goal of this tool was to replace the slow matching step packaged with Bundler by a faster one using GPU without needing to modify Bundler’s code. You can download BundlerMatcher1.zip or checkout the code on my google code. Warning: this tool needs a 64bit windows OS (tested on 7 and vista) and a decent GPU. All my demos are available under MIT license but SiftGPU isn’t released under MIT so you should take a look at SiftGPU license.

OpenSynther

I’m proud to introduce OpenSynther which is under heavy development. The first goal of this tool is the same as BundlerMatcher: provide a faster matching engine for Structure from Motion tools. To achieve high performance it is coded in C++ and using both multi-core and GPU. Furthermore OpenSynther is using Surf instead of Sift and may in the future also compute the 3d reconstruction (as PhotoSynth does).

OpenSynther current feature list:

  • Jpeg loading + Exif reading
  • multi-threaded Surf feature extraction (based on OpenSurf & ParallelSurf)
  • multi-threaded Surf feature matching (based on OpenSurf matching + Cuda GPU matching)
    • Quadratic complexity O(n²) with n number of pictures

OpenSynther TODO:

  • multi-threaded Surf feature approximate matching (based on FLANN)
    • Linear complexity (hypothetical)
  • integrate 3d reconstruction ?

Benchmarks

These benchmarks were done on an Intel Corei7 920 (8 cores @ 2.66Ghz) + Nvidia GTX 285. The pictures size were 1600×1200 (~2M pixels), I heard that it was the limit size at which PhotoSynth extract features (it can of course use bigger pictures but feature extraction are done at a max size of 2M pixels).

Note:

  • Bundler: I had to reduce picture to 640×480, otherwise I got too many features (would be unfair).
  • PhotoSynth: I didn’t take into account the time spent in tile creation.
  • OpenSynther: in the future I may consider using GPUSurf for feature extraction.
  • BundlerMatcher: to be fair with PhotoSynth I didn’t write the ascii key file (so in fact it’s slower).


Note:

  • PhotoSynth: they did a very good job and managed to have a linear matching !
  • OpenSynther: the GPU is in fact doing the most part of the job as you can see in the table below:
App Name PhotoSynth OpenSynther BundlerMatcher Bundler
Extraction in sec 10 22 39 21 40 78 7 12 24 105 220
Matching in sec 15 31 58 31 104 322 40 170 542 75 440
% matching
done by CPU
100 100 100 22 12 13 0 0 0 100 100
% matching
done by GPU
0 0 0 78 88 87 100 100 100 0 0
Nb pictures 49 98 196 49 98 196 49 98 196 49 98 196



As you can see OpenSynther matching is in fact done by the GPU at 85%.

Share