Where does the real world end?

Blurring the lines

By Antoinette Price

Everyday activities, such as shopping, watching sport on TV or even the ways we work and learn are going to change profoundly in the coming years, as more industries, including education, use augmented and virtual reality (AR/VR).

VR enters the classroom VR is being used more in schools, universities and for workplace training (Photo: touchstoneresearch.com)

Worldwide revenues for AR and VR are forecast to grow massively, from USD 5,2 billion in 2016 to more than USD 162 billion in 2020, according to market intelligence company, International Data Corporation.

As reality merges

During the 50th CES in Las Vegas this year, the AR/VR category was the largest yet seen. VR, for many, is associated with flight simulation and gaming, while AR has become well recognized in TV sport programmes (superimposing stadium and court lines over live shots to show whether the ball has landed ‘in’ or ‘out’), and more recently in smartphone apps, overlaying useful information on maps (about services or businesses), or in tourism (over artefacts in museums or historic destinations or monuments). 

Now the two technologies are coming together in what is known as ‘merged reality’ (MR), where the real and virtual worlds merge and objects from both can interact. Advances in VR technology were demonstrated at CES, in a prototype tether-free wireless headset which doesn’t require the hardware traditionally placed around a room in order to track movement and objects. Instead, the headset contains the processors, cameras and sensors that enable it to do this. It can also track the user’s movement in 3D space.

Where do Standards come in?

Behind the AR and VR scenes, software drives components such as displays, sensors, images, maps and tracking technology. It links to the hardware, which consists of processors, sensors and input devices (speech and gesture recognition systems). Displays include monitors and handheld devices such as smartphones and tablets. These contain optical sensors, accelerometers, gyroscopes, GPS and cameras for tracking movement. Eyeglasses, head-up display (HUD), computers, software and algorithms help AR systems integrate augmentations with the real world.

A number of IEC technical committees (TCs) and their subcommittees (SCs) produce International Standards which help ensure the reliability, safety, efficiency and interoperability of the components within this technology.

IEC and the International Organization for Standardization (ISO) have established a Joint Technical Committee, ISO/IEC JTC 1 to cover information technology. ISO/IEC JTC 1/SC 29 deals with the coding of audio, picture, multimedia and hypermedia information, and has published ISO/IEC 23000-13, an International Standard which focuses on the data formats used to provide an AR presentation which uses 2D/3D multimedia content.

IEC TC 47 ensures that sensors, which are vital to this technology, work reliably and efficiently, while IEC TC 100 produces Standards which contribute to the quality, performance and interoperability of audio, video and multimedia systems and equipment.

Is VR the future of education?

The medical world is embracing VR applications, many of which employ merged reality. This innovative technology and approach to training surgeons allows them to improve performance, increase surgery success rates and better explain health conditions and surgery procedures to their patients.

Meanwhile students will love their new anatomy classes, where books are exchanged for headsets, offering an entirely new and unforgettable way of learning.

Technology for training

A prominent UK dental school has developed VR to allow dental surgeries to teach students how much pressure to use when drilling.

A mirror and a haptic dental drill with feedback technology enable trainees to sense touch and force in a VR environment, as they operate on virtual 3D teeth shown on a screen. A real foot pedal controls the drill speed and settings, while eye glasses track head movement so that the 3D model on the screen moves relative to the head movement. Students practice procedures, watch their filmed performance and learn from mistakes before they get anywhere near a patient.

Distance learning with a difference

Increasingly, surgery is being filmed and streamed live to students and professionals around the world, using 360⁰ camera rigs and microphones atop the surgeon’s head.  Headset wearers follow the operation in real time, through the surgeon’s eyes. They have unprecedented views, can replay it as many times as required and experience being in an operating theatre while the surgeon explains what he is doing and answers questions.

Anyone anywhere can see how highly-skilled surgeons work and interact with their teams. With few opportunities to do this before performing a first surgery, VR technology is opening new doors for the next generation of surgeons.

The ENGAGE VR platform was designed to overcome the challenges of geography. It offers content developers the chance to create interactive virtual classrooms and trainings on diverse subject matters, anywhere in the world, for up to 30 individuals simultaneously. Instructors can observe and provide feedback, in real time, to students who interact with avatars. Oxford University academics demonstrated their life-saving LIFE product, which is based on ENGAGE, at CES. Through virtual distance learning, they helped train healthcare professionals in Africa in emergency care for infants, using an avatar baby.

Improving how we learn and diagnose

The Body VR platform created a buzz at CES for students and doctors alike. It allows developers to create content for higher education and simulations to help train radiologists, surgeons and physicians. Students put on headsets and move through the bloodstream, examining along the way how cell structures work, by interactively ‘touching’ and manipulating them. Doctors use it to view 3D volumetric renderings of imported CT or MRI scans, which they can manipulate using hand controls to see different layers and specific aspects.  

Teaching history by recreating it

History is another subject which lends itself well to AR and VR technology. Museums have already produced AR apps for smartphones to provide more information about their exhibits. VR offers headset wearers a first-hand experience amid historically accurate settings by transporting them back to a specific time and place.

Classes can go to the moon or relive the demise of the Titanic with VR, while tourists can see how the city they are visiting looked in ancient times, thanks to AR-equipped glasses which overlay the information before them.

You can read about other industries using VR applications in the May 2016 e-tech article Experiencing life like never before.

A few more stepping stones

The virtual world is a perfect place to train people for jobs where risk is involved, such as plant maintenance or surgery.  Users can practise safely for as long as they like. In the classroom, VR, AR and MR offer a unique and engaging way to understand and remember complex subject matter. Knowledge can be shared globally in innovative and affordable ways, as experts ‘enter’ a classroom on the other side of the world or are beamed to a remote location to train people in the field.

In the coming years, many such educational platforms will become available. However, the industry still has a few challenges to overcome. It must make VR headsets and AR glasses more affordable and appealing, as well as create far more diversified content if this medium is to be adopted widely.

Gallery
VR enters the operating theatres Real-time sensor fusion and 3D model update for minimally invasive surgery (Photo: co-me.ch)
VR enters the classroom VR is being used more in schools, universities and for workplace training (Photo: touchstoneresearch.com)
VR training for dentists VR dental training allows performances to be assessed (Photo: www.voxel-man.com)