Archive for the ‘html’ category

How to bypass WebGL Sop restriction v2

December 5th, 2011

I’ve previously posted a trick to bypass the WebGL same origin policy restriction in a particular context. But my solution had a major drawback: it was slow due to jpeg decoding in javascript. Hopefully I’ve implemented a new solution which is way much faster thanks to John Bauman suggestion and it works like a charm!

var xhr = new XMLHttpRequest();"GET", JPEG_URL, true);
xhr.responseType = "arraybuffer";
xhr.onload = function() {
	var img = new Image();
	img.onload = function() {
		var tex = new THREE.Texture(this);
		//do something with this texture...
	img.src = arrayBufferDataUri(xhr.response);

The function “arrayBufferDataUri” at the line 10 allow to speed-up the conversion from ArrayBuffer to base64 ascii string by avoiding an extra copy needed to use window.btoa. I’ve found this function here:

function arrayBufferDataUri(raw) {
	var base64 = '';
	var encodings = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
	var bytes = new Uint8Array(raw);
	var byteLength = bytes.byteLength;
	var byteRemainder = byteLength % 3;
	var mainLength = byteLength - byteRemainder;
	var a, b, c, d;
	var chunk;
	// Main loop deals with bytes in chunks of 3
	for (var i = 0; i < mainLength; i = i + 3) {
		// Combine the three bytes into a single integer
		chunk = (bytes[i] << 16) | (bytes[i + 1] << 8 ) | bytes[i + 2];
		// Use bitmasks to extract 6-bit segments from the triplet
		a = (chunk & 16515072) >> 18; // 16515072 = (2^6 - 1) << 18
		b = (chunk & 258048) >> 12;   // 258048   = (2^6 - 1) << 12
		c = (chunk & 4032) >> 6;      // 4032     = (2^6 - 1) << 6
		d = chunk & 63;               // 63       = 2^6 - 1
		// Convert the raw binary segments to the appropriate ASCII encoding
		base64 += encodings[a] + encodings[b] + encodings[c] + encodings[d]
	// Deal with the remaining bytes and padding
	if (byteRemainder == 1) {
		chunk = bytes[mainLength];
		a = (chunk & 252) >> 2; // 252 = (2^6 - 1) << 2
		// Set the 4 least significant bits to zero
		b = (chunk & 3) << 4;   // 3   = 2^2 - 1
		base64 += encodings[a] + encodings[b] + '==';
	else if (byteRemainder == 2) {
		chunk = (bytes[mainLength] << 8 ) | bytes[mainLength + 1];
		a = (chunk & 16128) >> 8; // 16128 = (2^6 - 1) << 8
		b = (chunk & 1008) >> 4;  // 1008  = (2^6 - 1) << 4
		// Set the 2 least significant bits to zero
		c = (chunk & 15) << 2;    // 15    = 2^4 - 1
		base64 += encodings[a] + encodings[b] + encodings[c] + '=';
	return "data:image/jpeg;base64," + base64;

I’ve only used this code for my Google Chrome PhotoSynth WebGL extension. This is why I don’t have added extra check for other browser (line 10: xhr.mozResponseArrayBuffer || xhr.response).


PhotoSynth WebGL panorama viewer

November 1st, 2011

I’ve updated my PhotoSynth Downloader/Parser to be able to process Synth and Panorama as well. The network part of my PhotoSynth downloader was really ugly: I’ve rewrote it from scratch using libcurl. This version is really much faster and doesn’t crash anymore. But I’m a bit concerned about releasing the binary/source code. Allowing the download of a panorama created with an iPhone doesn’t bother me but it’s really different with gigapixel panorama (and my downloader can download both).

Live Demo (require WebGL enabled browser)

My implementation is very strait-forward: my PhotoSynthHDDownloader creates 6 jpg from a panorama url. The 6 faces of the cube are downloaded at full resolution. This solution is suitable for panorama created with an iPhone (low res) but not acceptable for gigapixel panorama (you need progressive download/display). I’m using Three.js as WebGL engine and I’d like to implement a megatexture solution to display very large panorama (I’ve found a very impressive working implementation and played with it: it is working with Google Chrome / Firefox 10).

To be able to handle panorama as well, my Google Chrome extension needs to use megatexture and it would be really more efficient if Microsoft could add the missing cors header to allow WebGL cross-origin (even only for And as they’ve just moved their data to Windows Azure, I guess that it may be easier to add this header ;-) . Otherwise I’ll have to use my special trick to bypass webgl same-origin-policy restriction.


Three.js PhotoSynth Viewer with pictures and new UI

October 3rd, 2011

I really like JS development and my Google Chrome PhotoSynth extension is a really good opportunity to experiment new thing with WebGL. So I’ve added new features to my previous version:

  • Navigation system (like the Microsoft Silverlight one)
  • Thumbnails display in the WebGL content [beta]

Adding the pictures wasn’t easy: it is not possible to draw a texture coming from another domain on a WebGL context for security reason. I have found a workaround for this issue (AFAIK unknown: see my post about it). I’ve also managed to “clone” the UI of the Silverlight viewer and to find out how to parse some navigation information from the JSON file provided with each Synth. But I wish that the PhotoSynth team gives me some information about the meaning of some other undocumented parameters…

I won’t release this new version as it is CPU hungry (because of the WebGL restriction workaround), but if the PhotoSynth team adds the missing CORS header this could solve the current WebGL restriction in a cleaner way ;-) .

A new feature is coming too (related to my new work at Acute3D).


How to bypass WebGL Sop restriction

October 3rd, 2011

Google Chrome and Firefox are blocking cross-domain image in WebGL (because of a security issue). This is really annoying when you want to display image from other domain in WebGL. To bypass this same-origin-policy restriction you have several options:

Ok, to be honest my “trick” only work for browser extensions (tested with Google Chrome extension only). How is it working?

  • Add the domain in the list of allowed cross-domain ajax call in the manifest of the extension.
  • Do a binary ajax request on the jpeg
  • Parse the jpeg in javascript using jpgjs
  • Fill the WebGL texture with the jpeg decoded in JS

The drawbacks are that it can only be used by browser extensions and that JS jpeg decoding is slow compared to native decoding.

I’ve used this workaround to add the pictures in my WebGL PhotoSynth Viewer extension.

I’ve posted a new version which is really much faster.


PhotoSynth viewer with pose estimator WebService

August 8th, 2011

I’ve spent 2 days implementing a WebService in C++. I’ve lost 1 day trying to use WebSocket (this wasn’t a good idea as Firefox and Google Chrome were not implementing the same version). I’ve also lost a lot of time with the beautiful drag and drop API… who is responsible for this mess? Anyway I’ve managed to create this cool demo:

The principle is very simple: the user drag and drop an unknown picture on the webpage. The picture is resized on the client side using canvas and then sent to the webservice using a classic POST Ajax call with the picture sent in jpeg base64. Then the pose estimator web service try to compute the pose and send an xml response. Then the pose is applied to the Three.js camera used in my PhotoSynth viewer.

I have cheated a little as I’ve used shots of a video with known intrinsic parameters. But I’ve two options to handle really unknown pictures:

  • If the jpeg as Exif I can use it to find the focal (using a CCD database and a JS Exif parser)
  • Otherwise I am planning to implement this in C++

I hope you enjoy this demo as much as I do and that Google and Microsoft are working in that direction too (especially Microsoft with Read/Write World…)