Posts Tagged ‘ogre3d’

Ogre3D PhotoSynth Viewer

January 26th, 2011

This viewer is now integrated with the new version of my PhotoSynthToolkit (v5). This toolkit allow you to download synth point cloud and thumbnails pictures. You can also densify the sparse point cloud generated by PhotoSynth using PMVS2 and then create great accurate mesh using MeshLab.

New feature of PhotoSynthToolkit v5:

Thumbnails downloading should be faster (8x)
New C++ HD picture downloader (download tiles and re-compose them)
Tools to generate “vis.dat” from previous PMVS2 call (analysing .patch file)
Working Ogre3D PhotoSynth viewer:
- Can read dense point cloud created with my PhotoSynthToolkit using PMVS2
- Click on a picture to change camera viewpoint
- No-roll camera system

Warning: the PhotoSynth viewer may need a very powerful GPU (depending on the synth complexity: point cloud size and number of thumbnails). I’ve currently tested a scene with 820 pictures and 900k vertices on a Nvidia 8800 GTX with 768mo and it was working at 25fps (75fps with a 470 GTX and 1280mo). I wish I could have used Microsoft Seadragon .

Download:

The PhotoSynthToolkit v5 is available on his dedicated page, please do not make direct link to the zip file but to this page instead. So people willing to download the toolkit will always get the latest version.

Video demo:

Future version

Josh Harle has created CameraExport: a solution for 3DS Max that enable to render the picture of the Synth using camera projection. I don’t have tested it yet but I’ll try to generate a file compatible with his 3DS Max script directly from my toolkit, thus avoiding to download the Synth again using a modified version of SynthExport. Josh has also created a very interesting tutorial on how to use mask with PMVS2:

Masks with the PhotoSynth Toolkit 4 – tutorial from Josh Harle on Vimeo.

7 comments »

Posted in ogre3d, photogrammetry

Tags: ogre3d photosynth photosynthtoolkit toolkit

2010 visual experiments

January 7th, 2011

Happy new year everyone!

2010 was a year full of visual experiments for me, I hope that you like what you see on this blog. In this post I’m making a little overview of all visual experiments created by me during this year. This is an opportunity to catch-up something you’ve missed! I’d like also to thanks some person that have been helping me too:

Olafur Haraldsson: for creating the photogrammetry forum
Josh Harle: for his videos tutorials and his nice blog
You: for reading this

Visual experiments created in 2010:

During this year I have added some features to Ogre3D:

ArToolKitPlus: augmented reality marker-based system
Cuda: for beginner only (at least advanced user could grab some useful code)
OpenCL: for beginner only (at least advanced user could grab some useful code)
Html5 Canvas: implementation based on skia for graphics and V8 for javascript scripting
Kinect: this is a very hacky solution, I’ll improve it later

I also have learned GPGPU programming by myself while coding a partial GPUSurf implementation based on Nico Cornelis paper. But this implementation is not complete and I’m willing to rewrite it with a GPGPU framework based on OpenGL and CG only (not Ogre3D). With such a framework writing Sift/Surf detector should be easier and more efficient.

I have created some visual experiments related to Augmented Reality:

My outdoor 3D tracking algorithm for augmented reality needs an accurate point cloud: this is why I’m interested in structure from motion and I’ve created two SfM toolkit:

SFMToolkit (SiftGPU -> Bundler -> CMVS -> PMVS2)
PhotoSynthToolkit (PhotoSynth -> PMVS2)

Posts published in 2010:

2010/12/22: Outdoor tracking using panoramic image
2010/12/20: Structure from motion projects
2010/12/13: Augmented Reality outdoor tracking becoming reality
2010/11/20: Kinect experiment with Ogre3D
2010/11/19: PhotoSynthToolkit results
2010/11/09: PhotoSynth Toolkit updated
2010/11/05: Structure From Motion Toolkit released
2010/09/27: My 5 years old Quiksee competitor
2010/09/23: PMVS2 x64 and videos tutorials
2010/09/08: Introducing OpenSynther
2010/08/22: Dense point cloud created with PhotoSynth and PMVS2
2010/08/19: My PhotoSynth ToolKit
2010/07/12: Pose Estimation using SfM point cloud
2010/07/11: Remote Augmented Reality Prototype
2010/07/08: Structure From Motion Experiment
2010/06/25: GPU-Surf video demo
2010/06/23: GPUSurf and Ogre::GPGPU
2010/06/20: Ogre::Canvas, a 2D API for Ogre3D
2010/05/09: Ogre::OpenCL and Ogre::Canvas
2010/04/26: Cuda integration with Ogre3D
2010/04/09: Multitouch prototype done using Awesomium and Ogre3D
2010/03/05: ArToolKitPlus integration with Ogre3D
2010/02/20: Hello World !

6 comments »

Posted in augmented reality, ogre3d, photogrammetry, photosynth

Tags: 2010 artoolkit augmented reality canvas cuda gpusurf ogre3d opencl visual experiments

Outdoor tracking using panoramic image

December 22nd, 2010

I have made this experiment in 2 days:

First of all, I must admit that this is more a “proof-of-concept” rather than a prototype… But the goal was to illustrate a concept needed for my job. I love this kind of challenge! Building something like this in 2 days was only possible thanks to great open-source library:

Ogre3D (MIT)
OpenCV (BSD)
SiftGPU (non-profit license)

Analysis

I’m using a panoramic image as reference. For each frame of the video I’m extracting Sift feature using SiftGPU and matching them with those of the reference image. Then I’m computing the homography between the 2 images using Ransac homography estimator (OpenCV cvFindHomography).

Performance

The performance are low due to complexity of the Sift detection and matching and that I’m applying the homography using cvWarpPerspective.

Sift extraction:	28ms	1228 features
Sift matching:	17ms	using SiftGPU
Ransac Homography estimation:	2ms	89 inliers of 208 matches
Homography application:	36ms	done on the CPU with OpenCV
Global: 12fps

I’m working on another version using Fast (or Agast) as feature detector and Brief as descriptor. This should lead to a significant speed-up and may eventually run on a mobile… Using the GPU vertex and pixel shader instead of the CPU to apply the homography should also gives a nice speed-up.

I’m also aware that it is not correct to apply an homography on a cylindric panoramic image (especially if you don’t undistort the input video frame too )

6 comments »

Posted in augmented reality, ogre3d

Tags: augmented reality ogre3d opencv sift

Kinect experiment with Ogre3D

November 20th, 2010

I’ve just bought a Kinect and decided to do some experiment with it:

This demo is a ripp-off the Kinect-v11 demo made by Zephod. In fact I’ve designed a new Ogre::Kinect library that provide Kinect connection through Zephod library. Then I’ve replace the Zephod OpenGL demo by an Ogre3D demo using my library. The nice part is that I’ve managed to move some depth to rgb conversion to the GPU (using pixel shader).

Links

Binary demo: OgreKinectDemo1.zip
Source code: svn on code.google
Documentation: doxygen
License: MIT

16 comments »

Posted in ogre3d

Tags: kinect ogre3d

Structure From Motion Toolkit released

November 5th, 2010

Overview

I have finally released my Structure-From-Motion Toolkit (SFMToolkit). So what can you do with it ? Let’s say you have a nice place like the one just bellow:

Place de la Bourse, Bordeaux, FRANCE (picture from Bing)

Well, now you can take a lot of pictures of the place (around 50 in my case):

And then compute structure from motion and get a sparse point cloud using Bundler:

Finally you have a dense point cloud divided in cluster by CMVS and computed by PMVS2:

You can also take a loot at the PhotoSynth reconstruction of the place with 53 pictures and 26 (without the fountain).

This is the SFMToolkit workflow:

SFMToolkit is composed of several programs:

BundlerFocalExtractor : extract CCD width from Exif using XML database.
BundlerMatcher : extract and match feature using SiftGPU.
Bundler : http://phototour.cs.washington.edu/bundler/ created by Noah Snavely.
CMVS : http://grail.cs.washington.edu/software/cmvs/ created by Yasutaka Furukawa.
PMVS2 : http://grail.cs.washington.edu/software/pmvs/ created by Yasutaka Furukawa.
BundlerViewer : Bundler and PMVS2 output viewer based on Ogre3D (OpenSource 3D rendering engine).

Download

As you can see this “toolkit” is composed of several open-source component. This is why I have decided to open-source my part of the job too. You can download the source code from the SFMToolkit github. You can also download a pre-compiled x64 version of the toolkit with windows scripting (WSH) for easier usage (but not cross-platform): SFMToolkit1.zip.

Help

If you need some help or just want to discuss about photogrammetry, please join the photogrammetry forum created by Olafur Haraldsson. You may also be interested by Josh Harle’s video tutorials, they are partially out-dated due to the new SFMToolkit but these videos are very good to learn how to use MeshLab.

Please go to the SFMToolkit page to get the latest version

19 comments »

Posted in ogre3d, photogrammetry

Tags: bundler bundlermatcher ogre3d photosynth sfmtoolkit

« Older Entries

I'm Henri Astre, a French guy who likes visual experiments. I've done some nice demos in various domains (augmented reality, GPU computing, structure from motion, 3d rendering, web app...) that you can see on this blog. [+ resume]
Subscribe