QR Code in Sightsync

Background

Sightsync originally planned to utilize BLE for tracking and mapping, but had a couple of concerns:

• BLE's installation will be expensive with Sightsync's wide coverage of space,
• BLE's latency and accuracy will be insufficient in Sightsync's requirements.

There, Marty at the last ARcade meetup introduced me that QR codes (i.e. image recognition) can perform tracking and mapping as well.

Availability

I researched on it a bit to fortunately find out that it's a common method. Vuforia does recognize QR codes, hence it does mapping for free. The other libraries are also known to work in the same way according to the forum's conversation. There are several demonstrations as well.

Performance

Here is an ideal working example of QR code recognition and object mapping (not my work.) this video shows a QR code reader running on an Android device, which reads a QR code from camera, fetches a 3D object from the URL, and maps the object onto the original QR code. The object follows the QR code's location and rotation as it moves.

Notice how fast and stable the tracking and mapping in the video are performed. Sightsync needs this level of speed and stability.

Maintenance Cost

Reg in yesterday's conversation about this topic, pointed out a potential cost of maintaining the surface of those images for a stable recognition. It will be problematic in Sightsync's case because those images will be exposed to nature and children for a long time.

Problem in Sightsync

The most significant problem is that Sightsync cannot keep those images inside screen: users have to be looking around. The tracking and mapping fail to work when the device moves from the location where those images were lastly caught on screen. It will break Sightsync's requirement: the real and virtual sights have to be exactly synchronized.

The maintenance cost can be dealt with easily, however, the stability of sight synchronization has to be secured. I came up with three solutions to this problem:

1. Distribute an enough amount of objects in the physical space,
2. Run a traditional location tracking method simultaneously, or
3. Instruct users to not move their feet in action.

1. Placing images everywhere. Surely this will disturb users and the others. A bunch of QR codes pasted to each exhibition in a museum or on walls of buildings is anarchy. It could be avoided by registering not only QR codes but also general objects, however, it would make the stability sensitive to situations.

2. Collaborating with traditional location tracking. Machine power of mobile devices will be the biggest concern. Also, researching, purchasing or developing an algorithm to combine two tracking methods will take up a large resource.

3. Telling users to not move. This will restrict the range of AR experience that Sightsync can potentially provide to the users; for example, storytelling gimmicks will be less new or creative. Also, I have to come up with how actually to make the users pause their feet when the virtual world is attracting.
This, however, reduces many technical and design difficulties of Sightsync. The sight can be a pre-rendered 360° image or animation, which reduces bugs from AI's or renderer's runtime behaviors.
Most importantly, this still achieves Sightsync's goal: perfectly synchronize sights of the real and the virtual (as long as the design succeeds on pausing users.)

Summary

QR code does have a place in Sightsync: it is technically very achievable, capable of detecting and mapping, and most likely be more accurate than BLE. From negative perspectives, it essentially cannot track but only detect an initial location in Sightsync's usage, and also registered images have to be maintained during exhibition.

The decision that I have to make here is whether I give up a device tracking feature and obtain a nice and easy achievement instead, or keep pursuing the stability of sight synchronization with allowing users to move.

Sidenotes

Marty additionally explained how QR code can become a secondary business, for that you can display advertisements around the image and such, which is an interesting possibility.
Reg also told me that a QR code will have a power of "reminding" people of an app when they see one, which is another interesting topic to study.

---

Thank you a lot, Marty and Reg, for sharing your insights with me.