--parallel_epipolar
--ba sba

BTW those options should be passed as command arguments to bundler.exe directly (and not been added to the options.txt file).

The parallel_epipolar option allows to estimate the fundamental matrices (ransac + lm) in parallel. This problem is embarrassingly parallel but due to some global variables used in the callback passed to lmfit it wasn’t that easy to implement: I’ve used a functor to hide the global variables used by the callback but sadly lmfit doesn’t have a void* userData parameter (only a pointer to the callback). Thus I’ve modified lmfit and add this missing parameter that allow me to pass a pointer to my functor. Furthermore I had to compile almost everything in C++ instead of C to use my functor inside lmfit. Thus I had to fix a lot of malloc calls that weren’t compiling in C++ due to missing cast. To keep a “backward behavior compatibility” this option is disabled by default.

The ba option allows to change the bundle adjustment “engine” used. Here is the list of available “engine”:

• sba (default)
• none (for debug only)
• pba_cpu_double
• pba_cpu_float
• pba_gpu_float

Pba stand for Parallel Bundle Adjustment: I’ve integrated mcba from Changchang Wu.

So if you have an Nvidia GPU card and installed the Cuda runtime you can add those options:

bundler.exe list_focal_absolute.txt --ba pba_gpu_float --parallel_epipolar
--options_file options.txt //on the same line

Timing on a 245 pictures dataset:

You can download this new version of bundler: bundler-multiBA-parallelEpipolar-x64.zip
Update: the source code is available on the MCBA branch of my Bundler fork.

New feature coming:

• CMVS support (coming very soon)
• White space handling in the toolkit scripting (PhotoSynthGrabber is already ready)
• Expose HTTP Proxy parameters for the downloader

You can download the new version: PhotoSynthToolkit8.zip or keep the previous one in case you miss some advanced feature (vis.dat generation, SoftImage XSI / 3DS Max scripting for example): PhotoSynthToolkit7.zip.

The only trouble with this demo is that it is using a proxy PHP page to bypass ajax cross-domain security issue. Thus I won’t put it online (as it would kill my bandwidth limit). But I’ll try to update my Google Chrome extension so that you can give it a try too (will take time and I’m going in holiday…sorry).

On the tech side, this demo is using:

• Three.js: for the rendering
• Downloadify: to allow downloading PLY generated on the client-side
• jDataView: to parse binary xhr response
• ImageFlow: coverflow system
• Accordion(except css) + Tween + SynthParser created by me

Other news

On another side I’ve been up to:

• compile a bundler version with PBA support (with bad result: fast BA but erroneous point cloud)
• create a new framework for fast matching: already 3x faster than OpenSynther (which is already really fast). This new framework is compatible with multi-gpu and multi-cpu + you can add constraints to memory usage!
• create an hybrid AR solution for my current work based on my panorama tracking demo and using gyroscope information to fix the computing time of the vision part

ETH-V3D Structure-and-Motion software was created by Christopher Zach. The original source code with dataset is available at Christopher Zach Open-Source page (GPL license). I have created a windows port of V3DSfMToolkit with scripting wich is available as both binary (V3dSfMToolkit1.zip) and source (github).

I’ve tested this toolkit with the dataset given by Christopher Zach (see above screenshot) the reconstruction looks good but I only managed to get partial reconstruction from my own dataset.

MissStereo

Quasi-Euclidean Epipolar Rectification: MissStereo created by Pascal Monasse, Neus Sabater, Zhongwei Tang. The original source code is available at the IPOL related page under GPL license. You can download my windows port as both binary (MissStereo1.zip) and source (github).

I’m interested by this method to estimate the fundamental matrix without prior focal length knowledge.

PhotoSynthToolkit with XSI support

With the help of Julien Carmagnac (3D Graphist and XSI advanced user), I’ve duplicated the 3DS Max texture projection rendering solution for Softimage XSI:

As usual, the new version of PhotoSynthToolkit including this new feature is available on his dedicated page.

SFMToolkit with sequence matching

SFMToolkit was packaged with BundlerMatcher, my own matching solution using SiftGPU: this is a trivial quadratic exhaustive matching implementation. This implementation is well-suited for unstructured (unordered) pictures but if you are using a sequence of images as input (movie) you can reduce the complexity of the matching to linear. You can do that by only comparing frame N with frame N+1, N+2 [...], N+p. I’ve also fixed a bug that may occur on some system (Windows 7 x64 with UAC activated): thanks to Josh Harle for the bug reporting!

The new version of SFMToolkit is available on his dedicated page: this new feature is hidden, you need to edit “1 – Bundler.wsf” and set SEQUENCE_MATCHING_ENABLED to true (replace false by true).

3D-Arch’2011

I’m going to the 3D-Arch’2011 Workshop at Trento: 3D Virtual Reconstruction and Visualization of Complex Architectures. I hope to see amazing things about 3D reconstruction

Jean-Philippe Pons (CSTB, Sophia-Antipolis, France): High-resolution large-scale multi-view stereo

I’ve also fixed other errors in BundlerMatcher too (small memory leak + no more 4096 match limit), so you should download this version too even if the previous version was working on your system. The new version is available on his dedicated page: please do not make direct link to the zip file but to this page so people downloading will always get the latest version.

Building Rome in a Day:

Project home is using Bundler (GPL)

Building Rome on a Cloudless Day:

Project home | Source code (Non-profit license, I’ve ported their source to windows)

Samantha:

Project home (I’ve contacted them without response but they said that they were going to release the source code: check at 28:50)

Samantha	Bundler

PhotoSynth:

Website – Microsoft closed-source SFM application: check out my PhotoSynthToolkit

PhotoSynth	Bundler

ETH-V3D Structure-and-Motion software:

Project home with source code (GPL, I’ve partially ported it to windows)

Simple Sparse Bundle Adjustment:

Project home with source code (LGPL, I’ve ported it to windows)

A multi-stage linear approach to structure from motion:

Project home | paper

Results from the paper of LinearSFM (Microsoft Research)

This list is not exhaustive, I’ve seen other projects (Efficient Large Scale Multi-View Stereo for Ultra High Resolution Image Sets: not sure how it is related to ETH-V3D Structure-and-Motion software)

Analysis

In a pre-processing step I’ve built a sparse point cloud of the place using my SFMToolkit. Each vertex of the point cloud has several 2D Sift features correspondences. I’ve only kept one Sift descriptor per vertex (mean of the descriptors) and put all descriptors in an index using Flann.

For each frame of the video to be augmented, I’ve extracted Sift feature with SiftGPU and then matched them using Flann 2-nearest neighbor search and a distance ratio threshold. The Flann matching is done in parallel with boost::threadpool. The matches computed contains a lot of outliers. So I have implemented a Ransac pose estimator using EPnP that permits to filter bad 2d/3d correspondences.

Performance

My implementation is slow (due to my implementation of Ransac EPnP that could be improved).

The slowness is not a so big issue because it doesn’t need to run at 30fps. Indeed the goal of my prototype is to have absolute pose with this tracking system each second and relative pose using inertial system available on mobile device (or using KLT tracking).

Issue

• Performance (faster is better )
• Point cloud reference is not always accurate (Bundler fault)

In another post I’ll introduce alternative to Bundler: faster and more accurate.

I have finally released my Structure-From-Motion Toolkit (SFMToolkit). So what can you do with it ? Let’s say you have a nice place like the one just bellow:

Place de la Bourse, Bordeaux, FRANCE (picture from Bing)

Well, now you can take a lot of pictures of the place (around 50 in my case):

And then compute structure from motion and get a sparse point cloud using Bundler:

Finally you have a dense point cloud divided in cluster by CMVS and computed by PMVS2:

You can also take a loot at the PhotoSynth reconstruction of the place with 53 pictures and 26 (without the fountain).

This is the SFMToolkit workflow:

SFMToolkit is composed of several programs:

• BundlerFocalExtractor : extract CCD width from Exif using XML database.
• BundlerMatcher : extract and match feature using SiftGPU.
• Bundler : http://phototour.cs.washington.edu/bundler/ created by Noah Snavely.
• CMVS : http://grail.cs.washington.edu/software/cmvs/ created by Yasutaka Furukawa.
• PMVS2 : http://grail.cs.washington.edu/software/pmvs/ created by Yasutaka Furukawa.
• BundlerViewer : Bundler and PMVS2 output viewer based on Ogre3D (OpenSource 3D rendering engine).

Download

As you can see this “toolkit” is composed of several open-source component. This is why I have decided to open-source my part of the job too. You can download the source code from the SFMToolkit github. You can also download a pre-compiled x64 version of the toolkit with windows scripting (WSH) for easier usage (but not cross-platform): SFMToolkit1.zip.

Help

If you need some help or just want to discuss about photogrammetry, please join the photogrammetry forum created by Olafur Haraldsson. You may also be interested by Josh Harle’s video tutorials, they are partially out-dated due to the new SFMToolkit but these videos are very good to learn how to use MeshLab.

Please go to the SFMToolkit page to get the latest version

I’ve finally managed to spend a couple of hours to compile a 64 bit version of PMVS2 for windows! You can download PMVS2_x64.zip right now. I’ll hope that this version will help some persons, I’ve personally managed to create a very dense model thanks to this version and PMVS2 was using more than 4Gb of ram on a 8-cores machines.

How to compile PMVS2 x64 by yourself:
download the CMake package of CMVS (containing PMVS) created by Pierre Moulon.
download and compile gsl 1.8
download precompiled pthread x64 lib from equalizer svn
download and compile clapack 3.2.1 using CMake

PhotoSynthTileDownloader

As requested by some persons, I’ve updated my PhotoSynthTileDownloader: you can now resume a partial download ! It’s already available for download: PhotoSynthTileDownloader2.zip.

Videos tutorials

Josh Harle has done some very nice videos tutorials on how to use my PhotoSynth ToolKit and has created another toolkit for Bundler that is using my BundlerMatcher.

PhotoSynth Toolkit post on Josh Harle’s blog.

Note: In fact your synths doesn’t need to be 100% synthy. My tool (PhotoSynth2PMVS) is capable of using an uncomplete synth. And now you could use my 64bit version of PMVS2 instead.

Please go to the PhotoSynthToolkit page to get the latest version


Bundler photogrammetry package post on Josh Harle’s blog.

Please go to the SFMToolkit page to get the latest version

In my previous post I have released my PhotoSynth Tookit but the PhotoSynth tile downloader wasn’t available yet. You can now download the picture of your Synth in HD using PhotoSynthTileDownloader1.zip. But please respect author’s copyright ! I have included a new confirmation dialog box that warn you about the Synth status (unlisted, public) but this is not shown in the video presentation of PhotoSynthTileDownloader.

BundlerMatcher

I have finally released BundlerMatcher, a feature extraction and picture matching tool built with SiftGPU. The main goal of this tool was to replace the slow matching step packaged with Bundler by a faster one using GPU without needing to modify Bundler’s code. You can download BundlerMatcher1.zip or checkout the code on my google code. Warning: this tool needs a 64bit windows OS (tested on 7 and vista) and a decent GPU. All my demos are available under MIT license but SiftGPU isn’t released under MIT so you should take a look at SiftGPU license.

OpenSynther

I’m proud to introduce OpenSynther which is under heavy development. The first goal of this tool is the same as BundlerMatcher: provide a faster matching engine for Structure from Motion tools. To achieve high performance it is coded in C++ and using both multi-core and GPU. Furthermore OpenSynther is using Surf instead of Sift and may in the future also compute the 3d reconstruction (as PhotoSynth does).

OpenSynther current feature list:

• Jpeg loading + Exif reading
• multi-threaded Surf feature extraction (based on OpenSurf & ParallelSurf)
• multi-threaded Surf feature matching (based on OpenSurf matching + Cuda GPU matching)
  • Quadratic complexity O(n²) with n number of pictures

OpenSynther TODO:

• multi-threaded Surf feature approximate matching (based on FLANN)
  • Linear complexity (hypothetical)
• integrate 3d reconstruction ?
  • using Bundler code or insight3D, both are using SBA.

Benchmarks

These benchmarks were done on an Intel Corei7 920 (8 cores @ 2.66Ghz) + Nvidia GTX 285. The pictures size were 1600×1200 (~2M pixels), I heard that it was the limit size at which PhotoSynth extract features (it can of course use bigger pictures but feature extraction are done at a max size of 2M pixels).

Note:

• Bundler: I had to reduce picture to 640×480, otherwise I got too many features (would be unfair).
• PhotoSynth: I didn’t take into account the time spent in tile creation.
• OpenSynther: in the future I may consider using GPUSurf for feature extraction.
• BundlerMatcher: to be fair with PhotoSynth I didn’t write the ascii key file (so in fact it’s slower).

Note:

• PhotoSynth: they did a very good job and managed to have a linear matching !
• OpenSynther: the GPU is in fact doing the most part of the job as you can see in the table below:

As you can see OpenSynther matching is in fact done by the GPU at 85%.