Sightsync

Photo-realistic Time-traveling in Mobile Augmented Reality

Summary

Overview

Sightsync is a retrospective Augmented Reality application, which enabled the school's alumni group to tour in virtually reconstructed campus in 1970s. The photo-realistic campus was crafted with photo & topological references and delivered with the help of Unity & Google Tango technology.

Development

We were very fortunate to have received references from the school's alumna group, including yearbooks which contained 500+ photographs taken in the campus in 1940s-1970s. We then took our time to manually identify each of their locations, so we can begin reconstructing the campus.

We used several modeling tools to reconstruct the campus: SketchUp to model the geometry, Maya to tweak & reformat and Quixel to create PBR materials.

Using Google Tango, we were able to scan & save the entire campus in a device's local storage, so that the AR anchoring could start & maintain on its own. As our users walked around in the campus, our app was able to precisely follow the current location and actively correct drifts without a user input.

Challenges

We had to end up limiting our app's "usable time" to nighttime, due to the fundamental issue in a large-scale outdoor anchoring: the daytime's constantly-changing location of the sun interfered with the anchoring algorithm, which relied on the color of specific points in the space.

We used photos from 1940s-1970s (top) to reconstruct the campus (bottom).

We collected and identify locations of 500+ photos on the top of a topology map from 1970s.

Team

Ryoichiro Oka
Lead Developer & Artist
BSc in Applied Computer Science at Woodbury University

Jesse Gilbert
Technical Adviser
Chair of Applied Computer Science at Woodbury University