You can get the extension here. Make sure you are logged in on photosynth.net (and have joined the beta) while using the extension as some photosynth2 feature are still only available to beta users (hopefully this limitation will be removed soon).

Feature added by the extension:

The extension is changing the top menu to:

• Search: previous Explore page + new links to fullscreen grid based ps2 explore page.
• Explore: new fullscreen map based explore page.
• My Synths: previous My Photosynths page + new tab (my map, my ps2, my info).
• Preview: link to the new preview website + animated preview on the create page.

Search:

Animated preview when you hover a thumbnail of a PS2 + links to new explore page.

Clicking on “Explore most recent PS2″ or “Explore most viewed PS2″ will display a page like this:

The bottom row contains the latest synths of the current user (the one of the current synth being played). You can hover to see a preview and click to load a synth. Experimental feature: it’s using a new version of the viewer capable of quickly unload/load synths (don’t click too fast ).

Explore:

New fullscreen page showing synths (ps1, panorama and ps2) on a map. Sadly there is no proper ranking here so you need to zoom in a lot before being able to see synths. Also there is a known bug in google chrome which is preventing from seeing silverlight content opened in new tab.

My Synths:

The extension add 3 new tabs: My map, My PS2, My info

My map

This page is really complicated, adding a lot of feature… I will propably have another post explaining how to use it.

Basically you can map (=geotag) your own synths (ps1, panorama, ps2). You need to be logged in and click on ‘show untagged synths’ then you can search for a place on the map, move the orange pushpin or right click to reassign it location. Then once the orange pushpin is at the place where you’ve captured your synth, you can click on ‘assign pushpin location’ to assign the location to the corresponding synth. To select synths, you can press and hold ‘ctrl’ and then draw a rectangle, from that selection you can either remove their map position or assign them a text tag. Clicking on ‘movable pushpins’ will allow you to directly move your synth pushpins on the map. You can click on on ‘Find neighbors’ to find synths arround your synth (orange synth = community, green = yours). You can also play with the timeline to only display synths captured in the corresponding time interval. You can also change the url and switch w=0 to w=1 to enable the wikipedia option. This option will search for the closest wikipedia of your synth. Please consider donating to WikiLocation if you are using the w=1 option.

My PS2

This is a new page allowing you to quickly preview all your ps2 synths. They are grouped by topology (spin, panorama, wall, walk) and then sorted by captured date.

My info

Sadly the captured date information is not properly filled by the system (it’s using the upload date). You need to click on ‘fix capturedDate’ to set the captured date of all your ps2 (it might take a while: wait for the ‘done’ alert box).

Preview – Create

The extension is adding an animated preview which will help you to choose the proper topology:

Comments

This extension was created by me only and thus it doesn’t mean that this is representative of upcoming photosynth feature.

FYI this is not my first extension for the photosynth website, I’ve already created one which is adding a webgl fallback viewer if you don’t have silverlight for photosynth1 content.

Have fun exploring all feature introduced in this chrome extension!

There is still room for improvements: I’ve discovered lot of interesting stuff that I’ve to try in Lilli Thompson speech at the GDC. In particular I’d like to improve:

• memory usage: track V8 garbage collection
• thumbnails loading: could be speed-up by using a web workers too
• Three.js rendering: I still need to switch to r48 and use BufferGeometry
• bundle.out generation: still very slow for big synth

To accelerate bundle.out file generation I’ve found a solution working pretty well for very large ascii file (80mo+). I’m generating the ascii file by writing directly in binary in a Uint8Array (by chunk of 1mo). The resulting code is more complex but is as fast as regular string concatenation and in comparison as a memory footprint very limited. It’s not integrated in the extension yet as my test case is working well (ascii ply generation) but bundle.out is slow (I need to track V8 bailout/deopt to understand why it’s slow).

I’ve also improved a lot my binary parsing by using only native binary parsing (DataView). One thing about DataView: did they realize that if you forgot the endianness parameter it will use BigEndian? (littleEndian = x86, bigEndian = Motora 68k).

Thanks to both optimization the new version is 4x faster! (loading time: 24s > 6s)

solution 1

• cross-orign was forbidden in contentscript (fixed now since chrome 13)
• lack of TypedArray copy/transfert -> JS array replacement
• Loading time: 13600ms (24400ms with the previous version without optimized binary parsing)

solution 2

• Issue: the UI is frozen during parsing (lack of threading)
• Loading time: 6500ms

solution 3

• I’m using transferable objects to both send the xhr.response (ArrayBuffer) and receive the parsed Float32Array (vertex positions) and Uint8Array (vertex colors).
• Issue: I didn’t manage to spawn a worker from my extension directly (SECURITY_ERR: DOM Exception 18): I’ve been forced to inline the worker as a workaround.
• Loading time: 6000ms

solution 4

• Issue: because of this bug workers can not do cross-orign xhr.
• Loading time: N/A

I’ve also optimized the PLY file generation by generating binary file on the client side instead of ascii.

• Ascii: 74mo in 9000ms
• Binary: 17mo in 530ms (~17x faster!)

Sadly I didn’t manage to find a way to accelerate “bundle.out” ascii file generation yet (involving lot of string concatenation and numbers formatting).

I didn’t have updated the extension on the chrome web store yet as I’m working on using the new BufferGeometry of Three.js that should be more efficient. Building BufferGeometry directly from Float32Array (vertex positions array) and Uint8Array (vertex colors array) seems a really better solution IMO.

Mesh compression

I’ve also discover the webgl-loader mesh compression solution recently. I’ve compiled my own version of objcompress.cc and fixed the JSON export. Then I’ve created my own format allowing to pack in one file all the utf8 files (thus reducing the number of http request needed). I will post about it soon: stay tuned!

Firefox extension?

On a side note I’ve also been playing with the new mozilla add-on builder SDK… well IMO Firefox isn’t the best browser for extension anymore. I’ve been very disappointed by the restriction applied to code running in the contentscript (for example prototype.js is not working: I had to use a clone that I’ve built long time ago). And I’m also very concerned about their security strategy: they don’t have a cross-orign xhr permissions field in the manifest. Thus extensions can do cross-orign request to all domains by default! Nevertheless I’ve managed to port the binary parsing of the point cloud but Three.js is not working either .

I’m sure that some of you have an idea of what to do with a bundle.out file from a PhotoSynth .
BTW, this export wouldn’t have been possible without Peter Sibley help, thanks Peter! He has help me by giving me the missing information needed to do the bundle.out PhotoSynth export.

New features:

• Point cloud downloading is working now!
  • Previous downloadify solution wasn’t working due to a Chrome bug and was removed
  • New version is using Html5 BlobBuilder + FileSaver instead
  • But you need a Chrome version >15
• Picture thumbnails are displayed thanks to my sop bypass trick
• Windows Azure support (corresponding permission was added to the manifest)

Other features are coming:

• basic panorama support (using atlas.jpg only: no progressive download yet)
• download zip file of whole synth?
• bug fixes (the current version is still very buggy, especially the menu)
• other cool stuff are coming!

If you don’t have Chrome you can take a look at my previous video showing most of the new feature (except point cloud downloading). The previous version wasn’t available as it was using my previous trick to bypass sop restriction (too slow to be widely used).

The new version is available on the Google Web Store.

• Navigation system (like the Microsoft Silverlight one)
• Thumbnails display in the WebGL content [beta]

Adding the pictures wasn’t easy: it is not possible to draw a texture coming from another domain on a WebGL context for security reason. I have found a workaround for this issue (AFAIK unknown: see my post about it). I’ve also managed to “clone” the UI of the Silverlight viewer and to find out how to parse some navigation information from the JSON file provided with each Synth. But I wish that the PhotoSynth team gives me some information about the meaning of some other undocumented parameters…

I won’t release this new version as it is CPU hungry (because of the WebGL restriction workaround), but if the PhotoSynth team adds the missing CORS header this could solve the current WebGL restriction in a cleaner way .

A new feature is coming too (related to my new work at Acute3D).

You can download the new Google Chrome extension from the chrome web store: Three.js Google Chrome extension. The old one is still available but this is not a good idea to have both activated at the same time… Do not hesitate to use the donate button (located in the right side-panel) if you find this extension useful… thanks!

Warning: this extension may use a lot of ram (point cloud are store in memory as regular js array not TypedArray).

Possible improvement

• Add picture thumbnails in the 3d environment (may become impossible without Microsoft adding CORS to the jpeg due to WebGL security issue)
• Add a free flight camera (easy to add but harder without roll: I’ve added features to Three.js Quaternion class for this purpose, but still not ready)
• Make the download point cloud link work! This is not my fault (I hope) I think that this is a Chrome bug. The only workaround that I can think of (without being sure that it would work) for this issue would be that Microsoft host downloadify.swf on photosynth.net… please ask them to do that!
• Add a Meshlab like camera system (mouse handling + white sphere): if you know Three.js please give it a try!

Credits

Please take a look at the list of all dependencies used for this demo in my previous post describing this PhotoSynth viewer.