Nghiaho optimizations are disabled by default as I’ve found that it generates a point cloud less dense. You can enable the optimization by adding:

--optimize

as last PMVS2.exe parameter in your command prompt or just activate the optimization by modifying the following files:

• 3 – Prepare for PMVS2.wsf line 49 -> var USE_OPTIMIZED_PMVS = true;
• 4 – Prepare for CMVS.wsf line 46 -> var USE_OPTIMIZED_PMVS = true;

You can download PhotoSynthToolkit11.zip.

This version introduce a new feature that will allow you to process a PhotoSynth without needing to wait for the end of the upload. In fact you can even cancel your synth as soon as the computation is complete. But be aware that all pictures uploaded in the meantime will stay on photosynth server (AFAIK).

How to proceed:
- wait for synth to be processed (but not fully uploaded).
- copy the 2 following files from %temp%\Photosynther to another folder (d:\my_synth for example)

• collection.dzcz
• collection.synth.bin

- then you can launch the step 1 of the PhotoSynthToolkit by providing your folder (d:\my_synth).
- you’ll need then to copy your original jpegs to the distort folder and you’re done!

You can download PhotoSynthToolkit10.zip.

• CMVS support finally integrated
• proper handling of folder with white space

How to use CMVS with PhotoSynthToolkit:

1. As usual run step 1 and move pictures from “hd” to “distort” folder.
2. You need to run first “Prepare for PMVS2″ then do NOT launch the generated bat file.
3. You can run “Prepare for CMVS” and run the generated bat file.

Warning: I’ve fixed all my programs to handle folder with white space (thanks to boost::filesystem). Sadly PMVS2/CMVS don’t handle them. The fix is trivial but instead of recompiling a new version I’ve found a bad workaround. I’m creating a temporary virtual drive z: pointing to your synth folder during the execution of PMVS2/CMVS using SUBST. The downside are that if you already have a z: drive it won’t work and if you kill PMVS2/CMVS you’ll have to manually remove the z: drive. To remove the z: drive type “subst z: /d” in your console.

If you need XSI/3DS max integration you’ll have to use the old version: PhotoSynthToolkit7.zip otherwise PhotoSynthToolkit9.zip is the latest version.

New feature coming:

• CMVS support (coming very soon)
• White space handling in the toolkit scripting (PhotoSynthGrabber is already ready)
• Expose HTTP Proxy parameters for the downloader

You can download the new version: PhotoSynthToolkit8.zip or keep the previous one in case you miss some advanced feature (vis.dat generation, SoftImage XSI / 3DS Max scripting for example): PhotoSynthToolkit7.zip.

I’m working on the development of a calibration system (like bundler) at Acute3D. I’ve decided to work with them as I think that they do have the best technology available and I wanted to work with very skilled people working in computer vision.

I don’t know if I’ll be able to continue writing on this blog as this new job is really more interesting and challenging for me.

1. Creating Synth: http://vimeo.com/14796939.
2. Downloading and Viewing: http://vimeo.com/25072862.
3. Viewing, Dense Reconstruction: http://vimeo.com/25071703.
4. Meshing model, creating texture: http://vimeo.com/25071366.
5. Projecting Hi-Res Images: http://vimeo.com/25072168.

I’ve also added the video tutorials links on the PhotoSynthToolkit page.

You can thank him on the pgrammetry forum: see Josh Harle announcement.

Thanks Josh!

OpenSynther has been designed as a library (OpenSyntherLib) which has already proven to be useful for several programs written by myself:

• OpenSynther: work in progress… used by my augmented reality demo
• PhotoSynth2CMVS: this allow to use CMVS with PhotoSynthToolkit
• BundlerMatcher: this is the matching solution used by SFMToolkit

Outdoor augmented reality demo using OpenSynther

I’ve improved my first attempt of outdoor augmented reality: I’m now relying on PhotoSynth capability of creating a point cloud of the scene instead of Bundler. Then I’m doing some processing with OpenSynther and here is what what you get:

You can also take a look at the 3 others youtube videos showing this tracking in action around this church: MVI_6380.avi, MVI_6381.avi, MVI_6382.avi.

PhotoSynth2CMVS

This is not ready yet, I still have some stuff to fix before releasing it. But I’m already producing a valid “bundle.out” file compatible with CMVS processing from PhotoSynth. I’ve processed the V3D dataset with PhotoSynth2CMVS and sent the bundle.out file to Olafur Haraldsson who has managed to create the corresponding 36 million vertices point cloud using CMVS and PMVS2:

The V3D dataset was created by Christopher Zach.

BundlerMatcher

The new unstructured linear matching is really fast as you can see on the above chart compared to PhotoSynth. But the quality of the generated point cloud is not as good as PhotoSynth.




This benchmark was computed on a Core i7 with an Nvidia 470 GTX. I’ve also compared the quality of the matching methods implemented in OpenSynther (linear VS quadratic). I’ve used Bundler as a comparator with a dataset of 245 pictures:

On the one hand, both the matching and the bundle adjustment are faster with linear matching but on the other hand, having only 193 out of 245 pictures registered is not acceptable. I have some idea on how to improve the linear matching pictures registering ratio but this is not implemented yet (this is why PhotoSynth2CMVS is not released for now).

Future

I’ve been playing with LDAHash last week and I’d like to support this in OpenSynther to improve matching speed and accuracy. It would also help to reduce the memory used by OpenSynther (by a factor 16: 128 floats -> 256bits per feature). I’m also wondering if the Cuda knn implementation could speed-up the matching (if applicable)? I ‘d also like to restore the previous Surf version of OpenSynther which was really fun to implement. Adding a sequential bundle adjustment (as in bundler) would be really interesting too…

Off-topic

I’ve made some modifications to my blog: switched to WordPress 3.x, activated page caching, added social sharing buttons and added my LinkedIn account next to the donate button…

First of all, I’m sorry this post is not about CMVS support in PhotoSynthToolkit . Releasing the PhotoSynthToolkit with CMVS support is way more complicated than predicted… This is because it is not just a file conversion process (as in my PhotoSynth2PMVS). I have designed a library (OpenSyntherLib) that extract features, match them, build tracks and then triangulate them using PhotoSynth cameras parameters. The problem is that this library is highly configurable to match each dataset needs. So providing an automatic solution with good parameters is difficult.

The new PhotoSynth2CMVS tool generates bundler-compatible file (“bundle.out”). I’ve sent the bundle.out file of the V3D dataset to Olafur Haraldsson and he has managed to create a 36 million vertices point cloud with it! It will be showcased in a next post.

PhotoSynth WebGL Viewer:

As soon as I’ve seen the SpiderGL presentation at the 3D-Arch Workshop I’ve decided to implement a PhotoSynth viewer with WebGL! Thanks to Cesar Andres Lopez I’ve found XB PointStream which is very well designed and documented. Implementing a PhotoSynth Parser turns out to be super easy! Porting my PhotoSynthParser.cpp took less than 10min thanks to the jDataView and binary ajax developed by Vjeux. After 2 hours of hacking I got this first viewer:

I can’t host the viewer on my website has it is using a proxy to fix ajax cross-domain security issue (thus it will kill my bandwidth limit). So I’ve made a video to show you how it looks:

The viewer source code will be hosted on my GitHub and I’m working on a Google Chrome extension to by-pass the need of a proxy… Having a Google Chrome extension that replace the Silverlight viewer on photosynth.net would be nice too (if you don’t have Silverlight).

• Andrea Fusiello showcased amazing results with Samantha: reconstruction without any camera calibration needed (neither Exif data).
• Jean-Philippe Pons announced that his dense multi-view reconstruction solution will be incorporated in Autodesk PhotoFly.
• Marc Pierrot-Deseilligny presented Apero: an open-source bundle adjustment software for automatic calibration and orientation of set of images (needs calibrated camera).

I should have published this post sooner but I wanted to make a double post with my new OpenSynther results… CMVS support in PhotoSynthToolkit is coming! You should expect another post next week with nice results

ETH-V3D Structure-and-Motion software was created by Christopher Zach. The original source code with dataset is available at Christopher Zach Open-Source page (GPL license). I have created a windows port of V3DSfMToolkit with scripting wich is available as both binary (V3dSfMToolkit1.zip) and source (github).

I’ve tested this toolkit with the dataset given by Christopher Zach (see above screenshot) the reconstruction looks good but I only managed to get partial reconstruction from my own dataset.

MissStereo

Quasi-Euclidean Epipolar Rectification: MissStereo created by Pascal Monasse, Neus Sabater, Zhongwei Tang. The original source code is available at the IPOL related page under GPL license. You can download my windows port as both binary (MissStereo1.zip) and source (github).

I’m interested by this method to estimate the fundamental matrix without prior focal length knowledge.

PhotoSynthToolkit with XSI support

With the help of Julien Carmagnac (3D Graphist and XSI advanced user), I’ve duplicated the 3DS Max texture projection rendering solution for Softimage XSI:

As usual, the new version of PhotoSynthToolkit including this new feature is available on his dedicated page.

SFMToolkit with sequence matching

SFMToolkit was packaged with BundlerMatcher, my own matching solution using SiftGPU: this is a trivial quadratic exhaustive matching implementation. This implementation is well-suited for unstructured (unordered) pictures but if you are using a sequence of images as input (movie) you can reduce the complexity of the matching to linear. You can do that by only comparing frame N with frame N+1, N+2 [...], N+p. I’ve also fixed a bug that may occur on some system (Windows 7 x64 with UAC activated): thanks to Josh Harle for the bug reporting!

The new version of SFMToolkit is available on his dedicated page: this new feature is hidden, you need to edit “1 – Bundler.wsf” and set SEQUENCE_MATCHING_ENABLED to true (replace false by true).

3D-Arch’2011

I’m going to the 3D-Arch’2011 Workshop at Trento: 3D Virtual Reconstruction and Visualization of Complex Architectures. I hope to see amazing things about 3D reconstruction

Jean-Philippe Pons (CSTB, Sophia-Antipolis, France): High-resolution large-scale multi-view stereo