Mixed reality at Microsoft’s Build Conference (and E3)

This week Microsoft is holding its annual Build developer conference. Yesterday was the Windows Keynote and as has been true for the past few years, there was a segment on Hololens, Microsoft’s augmented reality headset.

What was different about this year is that Microsoft has broadened its horizons some. While in the past it seemed like they were embracing augmented reality and to a certain extent downplaying virtual reality, this year they’re embracing both. In fact they’re backing away from either of these terms and instead talking about mixed reality, a term that encompasses both VR and AR.

As part of the discussion they announced new motion controllers for use with mixed reality, and at the same time they announced that Acer (and eventually other hardware developers) will release a mixed reality bundle this holiday. It’ll include a visor and a pair of the new motion controllers for $399. That’s really cheap.

Microsoft continues to embrace the “inside-out” tracking they first showed on Hololens. Rather than having to use an external camera (as with Playstation VR) or set up sensors (like the Vive uses), Hololens and the Acer visor both put the cameras on the visor itself. By tracking stationary external objects (presumably the corner of a room, or a window or door) the system can extrapolate how the visor is moving, and it can also track the motion controllers.

The upside of this is easy set up and a self-contained experience. Hololens is an untethered experience that you can presumably put on anywhere and use. The Acer visor is tethered but at least doesn’t require any other set up other than plugging it in.

The downside is that the system can only track things you’re looking at. If you think about, say, a tennis game, the system wouldn’t be able to track your hand directly when you reached behind yourself for a serve. (As far as I can tell the visor doesn’t have any backwards facing sensors.) That said, the system probably has a rough idea of where your hand is based on the last place it was “seen” and gyroscope (or some other kind of) data from the controllers.

This year the presenters studiously avoided talking about gaming and mixed reality, instead urging us to look forward to learning more at E3 next month. I assume Microsoft’s E3 press event is going to talk a lot about mixed reality.

If Microsoft is smart, the same visor will work on your Windows PC and your Xbox. That seems like an obvious thing but remember they tried to sell Kinect for Windows as a separate SKU from Kinect for Xbox. I don’t think they’ll make the same mistake again, though.

The $400 price point for the visor and controllers is potentially disruptive (more so if the same visor can be moved from the Xbox to a PC easily). The Playstation VR bundle, which includes the visor, controllers and the camera you need to make is all work, is $500 and it’s currently the cheapest full VR experience (stuff like the Gear VR that uses a cell phone is cheaper, of course). The Acer is $100 less and presumably can be used for AR and VR. The form factor is that of a VR visor but the forward facing cameras mean is should be possible to project the real world into the visor in order to make it “virtually transparent.”

Of course how well this works will depend on the quality of the cameras and the displays inside the visor. If nothing else I hope we get a ‘peekaboo’ feature where we can tap a button and see the world around us without taking the visor off. Helpful for checking to see if that thump you just heard was the dog knocking something over or aliens coming into your house to abduct you. (Hey just because I’m paranoid it doesn’t mean the aliens AREN’T after me.)

If you’re a developer you can pre-order a devkit that includes just the Acer visor for $299. It ships in August, which seems pretty late if MS expects support for the holiday season.

My expectation for E3 is that Microsoft will announce Project Scorpio bundled with the Acer Mixed Reality set for $799. I’m basing that on my expectation that Project Scorpio alone will be $499, so the bundle will save you $100 off of buying the two items separately, and I’m sure it’ll include some kind of software. Either a game or a demo disk or something. Of course the Mixed Reality bundle (and Project Scorpio) will both be available separately as well.

What I’m not sure of is whether the Acer visor will require Scorpio or if it’ll support the original Xbox One in some scaled down capacity.

I’ve also read some hopeful speculation that Microsoft might surprise us and launch Scorpio earlier than expected…potentially in August. The idea is that they’d want to get the new console out in time for all the big fall releases like Destiny 2. If the Acer dev kit isn’t shipping until August, and the retail version not until (presumably) the November timeframe, I think it makes an August Scorpio launch less likely. I think MS is going to want to introduce these two products at once. They seem pretty serious about succeeding in mixed reality.

I guess we’ll learn more next month!