A prototype I developed for a leading English Language Training provider to illustrate the potential of developing an interactive website utilising the latest developments in web technologies such as WebAR (ar.js), WebVR and NLP Engines (Google's DialogFlow) to deliver personalised digital content tailored to the strengths and weakness of individual students closely tied to the physical textbook by scanning QR codes to access the relevant digital content throughout the book so mitigating the need for the user to download an app which is the current norm.
Each student is able to choose and customise their own avatar so making it feel more relevant to them reflecting their own gender and race, and which is used in both the AR and VR content. In the VR content I attempted to make it feel more immersive by adding your avatar’s body and shadow when you look down which is often lacking in other VR experiences so it feels more like you are floating or disembodied. I extended this even further by taking the height from your avatar so if you choose the child avatar then your viewpoint in the VR content is that of the child’s height in comparison with the other avatars.
By using a series of AR markers with one representing you as your avatar then you can move this marker towards another marker to initiate a conversation with that virtual character so allowing there to be a series of simple and distinct language based tasks to be completed potentially in sequence.
This idea was replicated in VR as well where it is possible to move around a virtual street scene and visit either a fast food restaurant or coffee kiosk to practice your conversational skills.
By using VR, it allows for a greater sense of immersion than you can experience in AR so may strength the learning process particularly in learning vocabulary as you are likely to engage more of your senses so similar to how memory champions are able to recall huge amounts of information by visualising this information in remembered physical spaces such as their local street or rooms in their house (Visualisation and Association) then VR might prove to be a more powerful learning tool in a similar way.
The current prototype targets only the Chrome browser on Android mobile devices as it uses the free in-built speech recognition available through this browser but could be extended to other devices including those running iOS by using a third-party cloud-based solution for speech recognition such as the one provided by Nuance. The prototype website is available to try here.