Posts Tagged ‘webgl’

« Older Entries
Newer Entries »

Three.js PhotoSynth Viewer with pictures and new UI

October 3rd, 2011

I really like JS development and my Google Chrome PhotoSynth extension is a really good opportunity to experiment new thing with WebGL. So I’ve added new features to my previous version:

  • Navigation system (like the Microsoft Silverlight one)
  • Thumbnails display in the WebGL content [beta]

Adding the pictures wasn’t easy: it is not possible to draw a texture coming from another domain on a WebGL context for security reason. I have found a workaround for this issue (AFAIK unknown: see my post about it). I’ve also managed to “clone” the UI of the Silverlight viewer and to find out how to parse some navigation information from the JSON file provided with each Synth. But I wish that the PhotoSynth team gives me some information about the meaning of some other undocumented parameters…

I won’t release this new version as it is CPU hungry (because of the WebGL restriction workaround), but if the PhotoSynth team adds the missing CORS header this could solve the current WebGL restriction in a cleaner way ;-) .

A new feature is coming too (related to my new work at Acute3D).

Share

1 comment »

Posted in html

Tags:

How to bypass WebGL Sop restriction

October 3rd, 2011

Google Chrome and Firefox are blocking cross-domain image in WebGL (because of a security issue). This is really annoying when you want to display image from other domain in WebGL. To bypass this same-origin-policy restriction you have several options:

Ok, to be honest my “trick” only work for browser extensions (tested with Google Chrome extension only). How is it working?

  • Add the domain in the list of allowed cross-domain ajax call in the manifest of the extension.
  • Do a binary ajax request on the jpeg
  • Parse the jpeg in javascript using jpgjs
  • Fill the WebGL texture with the jpeg decoded in JS

The drawbacks are that it can only be used by browser extensions and that JS jpeg decoding is slow compared to native decoding.

I’ve used this workaround to add the pictures in my WebGL PhotoSynth Viewer extension.

I’ve posted a new version which is really much faster.

Share

6 comments »

Posted in html

Tags:

PhotoSynth viewer with pose estimator WebService

August 8th, 2011

I’ve spent 2 days implementing a WebService in C++. I’ve lost 1 day trying to use WebSocket (this wasn’t a good idea as Firefox and Google Chrome were not implementing the same version). I’ve also lost a lot of time with the beautiful drag and drop API… who is responsible for this mess? Anyway I’ve managed to create this cool demo:

The principle is very simple: the user drag and drop an unknown picture on the webpage. The picture is resized on the client side using canvas and then sent to the webservice using a classic POST Ajax call with the picture sent in jpeg base64. Then the pose estimator web service try to compute the pose and send an xml response. Then the pose is applied to the Three.js camera used in my PhotoSynth viewer.

I have cheated a little as I’ve used shots of a video with known intrinsic parameters. But I’ve two options to handle really unknown pictures:

  • If the jpeg as Exif I can use it to find the focal (using a CCD database and a JS Exif parser)
  • Otherwise I am planning to implement this in C++

I hope you enjoy this demo as much as I do and that Google and Microsoft are working in that direction too (especially Microsoft with Read/Write World…)

Share

4 comments »

Posted in html, photogrammetry

Tags:

Google Chrome PhotoSynth Viewer using Three.js

July 21st, 2011

I’m finally back from holiday and found enough time to update the Google Chrome extension with my new PhotoSynth Viewer based on Three.js.

You can download the new Google Chrome extension from the chrome web store: Three.js Google Chrome extension. The old one is still available but this is not a good idea to have both activated at the same time… Do not hesitate to use the donate button ;-) (located in the right side-panel) if you find this extension useful… thanks!

Warning: this extension may use a lot of ram (point cloud are store in memory as regular js array not TypedArray).

Possible improvement

  • Add picture thumbnails in the 3d environment (may become impossible without Microsoft adding CORS to the jpeg due to WebGL security issue)
  • Add a free flight camera (easy to add but harder without roll: I’ve added features to Three.js Quaternion class for this purpose, but still not ready)
  • Make the download point cloud link work! This is not my fault (I hope) I think that this is a Chrome bug. The only workaround that I can think of (without being sure that it would work) for this issue would be that Microsoft host downloadify.swf on photosynth.net… please ask them to do that!
  • Add a Meshlab like camera system (mouse handling + white sphere): if you know Three.js please give it a try!

Credits

Please take a look at the list of all dependencies used for this demo in my previous post describing this PhotoSynth viewer.

Share

13 comments »

Posted in html, photogrammetry

Tags:

PhotoSynth Viewer using Three.js

June 20th, 2011

I wasn’t happy with my previous WebGL PhotoSynth Viewer: my javascript parser was aware of the position of the cameras and I wasn’t doing anything useful with it… Furthermore I was only relying on XB-PointStream for the rendering (which manage to display perfectly a dense point cloud). But I definitively want to do more fancy stuff. So Goodbye XB-PointStream, Hello Three.js!

The only trouble with this demo is that it is using a proxy PHP page to bypass ajax cross-domain security issue. Thus I won’t put it online (as it would kill my bandwidth limit). But I’ll try to update my Google Chrome extension so that you can give it a try too (will take time and I’m going in holiday…sorry).

On the tech side, this demo is using:

Other news

On another side I’ve been up to:

  • compile a bundler version with PBA support (with bad result: fast BA but erroneous point cloud)
  • create a new framework for fast matching: already 3x faster than OpenSynther (which is already really fast). This new framework is compatible with multi-gpu and multi-cpu + you can add constraints to memory usage!
  • create an hybrid AR solution for my current work based on my panorama tracking demo and using gyroscope information to fix the computing time of the vision part
Share

10 comments »

Posted in html, photogrammetry

Tags: