The HoloLens headset.
Microsoft
For the second time in as many months, I feel like I've taken a step into the world of science fiction—and for the second time in as many months, it's Microsoft who put me there.
After locking away all my recording instruments and switching to the almost prehistoric pen and paper, I had a tantalizingly brief experience of Microsoft's HoloLens system, a headset that creates a fusion of virtual images and the real world. While production HoloLens systems will be self-contained and cord-free, the developer units we used had a large compute unit worn on a neck strap and an umbilical cord for power. Production hardware will automatically measure the interpupillary distance and calibrate itself accordingly; the dev kits need this to be measured manually and punched in. The dev kits were also heavy, unwieldy, fragile, and didn't really fit on or around my glasses, making them uncomfortable to boot.
But even with this clumsy hardware, the experience was nothing short of magical.
Microsoft calls it holography. I'm not sure if it really is (Wired describes HoloLens' "light engine" as having a "grating," so perhaps it really is using interference patterns to reconstruct light fields rather than providing the same simple stereoscopic 3D found in VR systems), but this is a detail that only pedants will care about. (Though if it is true holography, it should solve the focus issue that many people find with existing 3D systems.)
However it works, HoloLens is an engaging and effective augmented reality system. With HoloLens I saw virtual objects—Minecraftcastles, Skype windows, even the surface of Mars—presented over, and spatially integrated with, the real world.
It looked for every bit like the holographic projection we saw depicted in Star Wars andTotal Recall. Except that's shortchanging Microsoft's work, because these virtual objects were in fact far more convincing than the washed out, translucent message R2D2 projected, and much better than Sharon Stone's virtual tennis coach. The images were bright, saturated, and reasonably opaque, giving the virtual objects a real feeling of solidity.
Minecraft comes alive
We ran through three interactive demos and watched a fourth. My favorite was Minecraft. I was in a room with a couple of tables and a picture on the wall. After putting on the headset and looking around the room so that the HoloLens could figure out where everything was, the world around me suddenly transformed. The table was no longer just a table. It had a big castle on it, with a river flowing beneath. The middle of the table was no longer there; I could peer through the hole to see the river below. As I moved around the room, I could examine the castle from all angles.
My attention was then turned to a second table which had something of a zombie infestation. Fortunately, the foolish zombies were clustered around a block of TNT. Detonating the TNT blasted through the table, revealing a lava pit below. The zombies toppled through the hole and fell to their deaths.
The picture frame on the wall now housed more TNT. Triggering it revealed a large cavern beyond—and out flew a bunch of blocky bats.
Through it all, the 3D effect was thoroughly convincing. The system felt very low latency; as I moved my head and walked around, the objects retained their positioning in the real world, with the castle, for example, never becoming detached from or wobbling around on the table. While Minecraft of course falls some way short of having photorealistic graphics, the melding of real and physical nonetheless felt convincing.
Enlarge / Minecraft leaking into reality.
Microsoft
If Microsoft can get the price of HoloLens right, it could become the must-have Minecraftaccessory at Christmastime. Microsoft's decision to buy Minecraft's developer all of a sudden makes sense.
Using Skype
A Skype demo was intriguing. I, with the headset, was talking to a person using regular Skype on a Surface Pro 3. The person was helping me wire up a light switch (and it seems that, yes, it was a real light switch with real electricity running through it). My assistant saw the world through my eyes; I saw him on a floating Skype pane that I could pin in place, where his head would remain. Using his Surface Pro 3 pen, the assistant drew diagrams showing me how to wire the switch up, pointing out which tool I should use for each task.
Enlarge / We weren't allowed to take any pictures of the HoloLens experience, and helpfully, Microsoft's image demonstrating the Skype experience shows someone changing a plumbing fixture rather than wiring up a light switch. The effect is much the same, mind you.
While I'm not so sure about the specific situation of using Skype to perform simple home repairs, the ability to overlay directions and guidance onto the real world feels like something that could be valuable in all manner of technical fields. The ability to see things through someone else's eyes is also intriguing. It almost brings to mind the film Strange Days, and I'm sure we can all remember what the S.Q.U.I.D.s were used for there; I imagine that HoloLens will find similar use cases.
Mission to Mars
Our third and final demo took me to the surface of Mars. I walked around a 3D world constructed from data captured by the Curiosity rover. (NASA intends to use HoloLens to explore data from Curiosity and collaboratively make decisions on how the rover should spend its time.) The experience reinforced just how immersive this kind of augmented reality can be; the Martian imagery obliterated most of the room I was in, except for a computer workstation. I joked that I was surprised to see a computer desk on the Martian surface, because that's what I was seeing.
The Mars demo extended the HoloLens experience in a few ways. In Minecraft and Skype, the interactivity was provided by a mix of voice command and hand gesture—a sort of finger wag serving as a mouse click—with the cursor driven by the direction I was looking. On Mars, I could use a mouse cursor to perform finer manipulations.
On Mars I was also joined by a second person, who appeared before me as a sort of golden apparition. This other person was using HoloLens, too, and so I could see a gaze line emanating from the face, showing me exactly what was being looked at. The apparition talked to me about some of the rocks and how they indicated that we were likely standing in what was once a lake bed.
Finally, the one non-interactive demo showed off HoloStudio, Microsoft's 3D modeling application for HoloLens. We saw a koala in a space suit and a monster truck assembled in front of us, again using a mix of voice and gesture control. Microsoft described this as "print preview for 3D printing;" by projecting the objects in 3D space and integrating them with the real world, artists and designers can get a sense of just how their objects will look before sending them to the 3D printer.
The future
Is this the future? In a few years' time will we do away with computer screens and walk around with magical 3D glasses? I don't know about that. Traditional screens are better for shared viewing, don't have battery life concerns, don't require headgear, and so on and so forth.
But HoloLens certainly felt like part of the future. It excites me. I imagine a world of gaming that interacts with the environment around me. A world where designers and engineers can manipulate virtual 3D objects JARVIS-style, simply by using their hands. A world where I can sit at my desktop PC and have a monitor that's near infinitely large and totally private. I don't know exactly what people will use HoloLens for, but it feels very much like the killer app is out there, just waiting for someone to develop.
I felt it to be a much more compelling future than VR, too. The VR systems I've tried make me feel too detached from the outside world, to a degree that I find disorienting. Also unlike VR systems, HoloLens didn't make me want to instantly vomit. It felt very natural.
I would like to know why Microsoft didn't go this route (as others are doing) and wonder if it would be easy to offer a blackout mode that didobscure the view of the outside world for a VR experience—such a thing may not be to my tastes, but others seem to enjoy them.
I also find myself anticipating future iterations of the technology; I can imagine a HoloLens with pupil tracking, for example, so the cursor could follow my eye movements rather than my head movements.
Microsoft wowed me with HoloLens. I'm already impatient for production hardware, and I can't wait to see what people do with this.
Samsung's prototype system combines a smartphone app with brainwave sensors to detect the early signs of stroke.Samsung Tomorrow
