webAR #3: What are the problems in using web AR?

The web AR engine is based on artificial intelligence that learns to look for markers in the image, such as graphics, a predetermined word or colour. When they are detected, the system reproduces the predetermined effect and adds it to the smartphone's camera view in real time.

How does this work in practice? We will explain using the example of a campaign we carried out together with the editorial team of the Gazeta.pl portal and the 180heartbeats JUNG v MATT agency. Our aim was to produce an errata for the famous textbook by Professor Wojciech Roszkowski, designed for the subject History and the Present, that would be attractive to young people.

All that is needed to check which parts of the text contain manipulation is a smartphone with a built-in camera and a web browser. The user does not need to install any application! 

As the camera scans the pages of the textbook, the artificial intelligence analyses their editorial layout: the positioning of images, titles or sub-headings. When it detects a piece of text questioned by the experts reviewing the publication, a graphic animation immediately appears on the phone's screen, informing the user that the content requires a correction or additional commentary. The student can read the correction or navigate to sources digging deeper into the topic. Read more about this campaign in our case study.

The smooth operation of web AR requires real-time processing of huge amounts of data. That is why we provided the campaign with a stable IT environment, reliable servers and fast data transfer. This is not the only challenge. Here is a list of limitations and issues in using webAR that you need to be aware of: 

• Browsers - not all browsers support camera access (e.g. Opera Mini and Firefox Mobile), but the most popular ones do.

• Consent - before launching the campaign, the browser will ask the user to agree to the use of the camera; if they don't agree, the question cannot be displayed again - you then need to go into the browser settings on your smartphone and manually enable camera access.

• Power saving mode - if the user has power saving mode on, the effect is much worse, animation playback can stutter.

• Accuracy of the effect - AR in the mobile app is of better quality because we can use ready-made mechanisms to speed up the calculations needed to display the effects; using the browser, we are limited in how much power the smartphone can use. 

• Older devices - smartphones 4-5 years old may struggle to display the effects smoothly; newer devices cope without problems. 

• Light - markers can look different in different light conditions, making it difficult for web AR engines to recognise them (this is an obstacle, but not insurmountable - see the case studies of Królewskie beer, where the markers were lettering and graphics printed on reflective labels applied to the bottles).