Archive for June, 2010

GPU-Surf video demo

June 25th, 2010

In the previous post I’ve been announcing GPU-Surf first release. Now I’m glad to show you a live video demo of GPU-Surf and another demo using Bundler (structure from motion tools):

There are three demos in this video:

  1. GPU-Surf live demo.
  2. PlyReader displaying Notre-Dame dataset.
  3. PlyReader displaying my own dataset (Place de la Bourse, Bordeaux).


You’ll get more information on the dedicated demo section.
In this video GPU-Surf was running slowly because of Ogre::Canvas but it should be running really faster.

PlyReader displaying Notre-Dame dataset

I’m also interested in structure from motion algorithm, that’s why I have tested Bundler, which comes with a good dataset of Notre-Dame de Paris.

I have created a very simple PlyReader using Ogre3D, the first version was using billboard to display point cloud but it was slow (30fps with 130k points). Now I’m using custom vertex buffer and it runs at 800fps with 130k points.

The reconstruction was done by the team who created Bundler from 715 pictures of Notre-Dame de Paris (thanks to Flickr). In fact, in this demo they have done the big part of the job, I have just grab their output to check if my PlyReader was capable of reading such a big file.

PlyReader displaying my own dataset

If you already used Bundler you know that structure from motion algorithm needs a very slow pre-processing step to get “matches” between pictures of the dataset. Bundler is packaged to use Lowe’s Sift binary, but it’s very slow because it’s taking pgm as picture input and the output is written in a text file. Then a matching step is executed using KeyMatchFull.exe which is optimized using libANN but still very slow.

I have replaced the feature extraction and matching steps by my own tool: BundlerMatcher. It is using SiftGPU, which gives a very nice speed-up. As my current implementation of GPU-Surf isn’t complete I can’t use it instead of SiftGPU but this is my intention.

23 pictures taken with a classic camera
(Canon Powershot A700)
Point cloud generated using Bundler

I have created this dataset with my camera and matched the pictures using my own tool: BundlerMatcher. This tool creates the same .key file as Lowe Sift tool and creates a matches.txt file that is used by Bundler. I have tried to get rid off this temporary .key file and keep everything in memory but changing Bundler code to handle this structure was harder than I predicted… I’m now more interested by insight3d implementation (presentation, source) which seems to be easier to hack with.


GPUSurf and Ogre::GPGPU

June 23rd, 2010

In this post I’d like to introduce my GPU-Surf implementation and a new library for Ogre3D Ogre::GPGPU.

What is GPUSurf ?

It is a GPU accelerated version of Surf algorithm based on a paper of Nico Cornelis1.
This version is using GPGPU technique (pixel shader) and Cuda for computing. The Cuda part was done using my Ogre::Cuda library and the GPGPU part was done using a new library called Ogre::GPGPU. This new library is just a little helper that hide the fact that GPGPU computing is done using quad rendering.

Screenshot of my GPU-Surf implementation (3 octaves displayed)

GPU-Surf could be used to help panoramic image creation, create tracking algorithm, speed-up structure from motion… I’m currently using SiftGPU to speed-up image matching step of structure from motion tools (bundler), but SiftGPU v360 as a memory issue (it’s eating a lot of virtual memory, your program as to be 64bit to bypass common 32bit application limitation: 2Go of virtual memory under windows), and Sift matching is more expensive than Surf (descriptor of 128 float vs 64 for Surf). That’s why I have decided to create my own implementation of GPU-Surf.

Structure from motion using PhotoSynth

Implementation details:

The current version of my implementation of GPU-Surf is uncomplete (descriptor missing, only detector is available). You’ll get all information about license (MIT), svn repository, demo ( and documentation on the GPUSurf page.

[1]: N. Cornelis, L. Van Gool: Fast Scale Invariant Feature Detection and Matching on Programmable Graphics Hardware (ncorneli_cvpr2008.pdf).


Ogre::Canvas, a 2D API for Ogre3D

June 20th, 2010

In my previous post I have announced Ogre::Canvas but It wasn’t available for download.
Now I have released the source code of Ogre::Canvas and some demo.

What is Ogre::Canvas ?

It is a new 2D API for Ogre3D that is based on the Html5 Canvas specifications. I have chosen this API because it is easy to understand, has a lot of cool demos, and it’s easy to prototype in browser using javascript. But you could be asking: what the point of having fast prototyping in a web browser if you’ll have to translate everything in C++ ? This is why I have also implemented Ogre::CanvasV8: a javascript binding for Ogre::Canvas using V8 the javascript engine of Google Chrome. That means that your prototype written in javascript for a browser will run in Ogre::CanvasV8 (with minor tweaks). The three Canvas demos belows are all working using Ogre::CanvasV8, grab and try yourself !

Tutorials from MDC Clocks Canvascape

What could I do with Ogre::Canvas ?

Well, it’s a 2D API, so you get a context on which you can execute drawing calls:

  • Transformation: scale, rotate, …
  • Image drawing: drawImage
  • Line styles: lineWidth, lineCap, …
  • Colors, styles and shadows: strokeStyle, fillStyle, …
  • Paths: moveTo, lineTo, arc, …
  • Text: fillText, strokeText, …
  • If you need a full overview of the Html5 Canvas API, grab the excellent Cheat Sheet.

You could create really complex UI element using Canvas. If you need an example of what you could create using this, take a look at the timeline I have created for a multitouch prototype. In fact it is because I got poor performance using Awesomium that I have decided to create this library. Awesomium is really a good product but integrating a full web page is overkill if you are only interested by a 2D Canvas tag…

What is the development status ?

It is not really “production ready”, I still have some issues with drawing images with alpha channels… You’ll get all information about license (MIT), svn repository and documentation on the Ogre::Canvas page.