Augmented Reality – Detecting Planar Surfaces

Recently I was asked to do some research on whether or not it’s possible for an AR (Augmented Reality) app to do surface detection without a marker.

Basically an AR app needs markers to compare the distortion in order to surmize a plane surface. However, there is some exciting research coming around, see my response below:

Note:

First off I think it’s important to note that I haven’t seen an app that is able to recognize planar surfaces without depth mapping (which phones currently don’t do – thought that may change – http://finance.yahoo.com/news/etron-technology-unveils-latest-3d-102700778.html) or a marker of some sort.

There are a number of articles written on augmented reality which explain why a marker is needed:

http://www.cwjobs.co.uk/careers-advice/it-glossary/the-10-things-you-need-to-know-about-augmented-reality

The reason, depth mapping or a marker is so important for recognizing surface is that without some sort of reference, any image just looks like a bunch of pixels with color data. Pattern recognition helps this process because we can project what a pattern will look like at different angles and infer what plane the surface is on. Without a marker, the app simply doesn’t have a reference.

The future:

Research is being done to simulate depth mapping by changing perspective on the subject:

http://www.youtube.com/watch?v=GyFGmaOhL_4

Current state of things:

There are some apps that allow a user to take a photo of their environment first and then map the game to that rather than printing a marker:

http://www.augmented-reality-games.com/

Further there are some apps that allow a user to position 3D objects in an environment to look like they exist within the environment:

https://play.google.com/store/apps/details?id=com.ar.augment&hl=en

There are some games that include ar, which do well at simulating an environment without the use of codes or depth mapping:

https://itunes.apple.com/us/app/zombies-everywhere!-augmented/id530292213?mt=8

https://play.google.com/store/apps/details?id=com.picitup.iOnRoad&feature=search_result#?t=W251bGwsMSwxLDEsImNvbS5waWNpdHVwLmlPblJvYWQiXQ..

Considerations:

Detecting a users environment may actually make gameplay worse. Suppose a user is playing in an office and most of the enemies “get stuck” on a piece of furniture that prohibits them from playing properly.

The complexity of having shapes and characters move around custom objects is dramatic, now instead of simple movements of 3D objects, the characters may need to climb/jump/mount etc. More actions/movements of characters = more complex development.

Workarounds:

Geolocation game instead?

Is it a game like zombies, run where you physically run from zombies? Instead of ar, could you defend your tower by gathering supplies and running to different places where you take certain actions?

Do we need it?

Do we need to detect the user’s environment? Will the game suffer if a user can simply play anywhere, without the game detecting an environment?

Desk play?

Can we just ask the user to take a photo of their desk, and play some sort of tower defense game on top of their desk?

Recommendation:

The current state of AR environment games is limited (for the time being) to using markers in order to play a game on a surface or using free form techniques to place objects and characters in an environment that isn’t dependent on a user’s environment. More research is needed to get an accurate picture of AR surface detection techniques that do not depend on predesignated markers. However, whatever these techniques are, they do not exist in a “production” environment at this time.