Can technology help us improve upon reality?

  • Published
Screengrab of video showing man interacting with Mars surfaceImage source, Microsoft
Image caption,
Augmented reality tech is helping scientists walk virtually on Mars

Imagine walking on Mars and being able to examine rock formations from all angles, or collaborating on the same 3D hologram design with someone thousands of miles away.

Or imagine being able to diagnose and treat the diseases of people half way around the world while you remain in your clinic, or walking around a gallery and having your own holographic guide pointing things out to you on your smart glasses.

These are just some of the exciting examples of what "augmented reality" (AR) technology is beginning to offer us.

This clumsy phrase just means overlaying digital information onto the physical world via an electronic device, whether that's a mobile phone, tablet or smart glasses.

Media caption,

Accenture's augmented reality app makes use of Google's Project Tango technology

No wonder Microsoft and Magic Leap - backed by Google and Alibaba - prefer to call it "mixed reality".

In time, it will allow us to interact with a world that is enhanced by multimedia provided by very clever computers - reality, only better.

"In five years, people expect the screen to be replaced by artificial intelligence - a virtual private assistant interacting with us via AR glasses," says Esmeralda Swartz, vice-president of marketing, enterprise and cloud at communications firm, Ericsson.

Image source, Thinkstock
Image caption,
Smart glasses will enable us to interact digitally with the real world

Smart sensors and the internet of things will transform "the city of the future into a portal we interact with," says Ms Swartz."

And with AR, unlike the totally immersive virtual reality (VR), we can still see what's going on around us. We can move about, whereas VR users are restricted to a small area. This means AR has more immediate practical applications for business.

Heads up display

Google Glass blazed a trail for this kind of tech, even though the BBC's Technology correspondent concluded it was "a fascinating failure" - the monocular design being deemed too weird for most consumers.

Surprisingly, tech firm Vuzix is adopting a similar approach.

But a number of other companies are now producing stereoscopic smart glasses or head-mounted displays that have potential uses in business, education, health, defence, retail and science.

For example, Meta recently showed off its new, rather bulky, headset at the TED (Technology, Entertainment and Design) conference in Vancouver, Canada.

Image caption,
The BBC's Jane Wakefield tries on Meta's new augmented reality headset

And Osterhout Design Group (ODG), a tech firm with a history of creating night vision goggles and heads-up displays for the US military, describes its latest R7 glasses as "a tablet you wear on your head".

The experience created by the two stereoscopic displays is "like a 70-inch TV screen floating roughly 6ft (1.8m) in front of you," says ODG's chief operating officer, Pete Jameson. Users operate the glasses in conjunction with a wireless hand-held controller.

One of its users, Optech 4D, has created software that integrates with the glasses to help maintenance engineers in the oil and gas industry.

The glasses recognise the piece of equipment, bring up its service history on screen, and show the relevant maintenance procedures.

Image source, ODG
Image caption,
ODG's R7 smart glasses are being used in industry and healthcare environments

"The ability to have line of sight is very important so you don't have to look away from what you're doing," says Mr Jameson. And this is one of the key advantages of smart glasses in an industrial context.

Some of the more surprising applications of their glasses, which come with a list price of $2,750 (£1,950), include improving the eyesight of people with macular degeneration, adds Mr Jameson.

Hologram happy

Microsoft's HoloLens, the latest version of which will shortly be released to developers at a cost of about $3,000, has already attracted commercial partners, such as car manufacturer Volvo and design software company Autodesk.

Image source, Microsoft
Image caption,
Microsoft's HoloLens gesture-controlled headset can track head movements

Nasa is already using a couple of the holographic headsets on the International Space Station, giving astronauts the ability to access mission control information and instruction manuals without being stuck in front of a computer monitor.

And scientists are using video footage and measurements taken by Mars rovers - mobile robots on the planet's surface - to create a virtual world that they can inhabit, meet and work in while wearing HoloLens headsets.

Image source, Microsoft
Image caption,
Microsoft says its HoloLens headset will enable designers to collaborate

In the US, Case Western Reserve University and the Cleveland Clinic are using the headsets to teach anatomy to students. The teacher and students can view the same 3D models and add layers of detail or focus on specific organs.

Image source, Microsoft
Image caption,
Students can all see the same 3D holographic model the teacher is showing them

"We do believe the future is holographic," a Microsoft spokeswoman tells the BBC. "We're going to have experiences that extend and complement 2D experiences, but we also imagine a future without screens."

AR on mobile

Augmented reality on expensive headsets is one thing - and that may be the future - but what about applications on cheaper mobile devices?

We've seen how static images can come alive as 3D animations just by holding your phone's camera over adverts, courtesy of companies like Blippar.

And retailers such as Ikea have also been experimenting with AR - enabling you to see what that two-seater sofa would look like against that wall, or over there by the TV. And virtual fitting rooms are allowing us to try on clothes remotely.

Image source, Cimagine
Image caption,
Cimagine's augmented reality platform lets you see what objects would like like in your home

Cimagine, a tech company that has developed an AR platform for retailers, numbers Shopdirect.com and John Lewis among its clients.

Its software takes images of the retailer's digital product catalogue and superimposes them onto whatever your phone or tablet camera is looking at.

"Our software scans the room 60 times a second and keeps the virtual objects anchored to their positions," says chief executive and founder Yoni Nevo. "And our tracking and rendering engine makes products very lifelike - it's like they're actually there in the room."

As well as helping consumers visualise what products might look like in various places in their homes, Cimagine is helping companies sell products to other businesses.

For example, Coca-Cola uses the software to show restaurants and cinemas what its drinks vending machines might look like in different positions.

"Now they can close a deal after just one visit rather than three," says Mr Nevo.

Augmented, or mixed, reality "is going to be massive," concludes ODG's Pete Jameson.

"It's going to start out in the enterprise market then start to move in the consumer environment faster than we think."

Follow Matthew on Twitter: @matthew_wall