With the December 16 launch of Rogue One: A Star Wars Story, Disney-owned Lucasfilm continues to build out the Star Wars universe. A key part of developing future Star Wars stories is Lucasfilm’s R&D lab, dubbed ILMxLab. This team of visual effects wizards, sound designers and video game developers is housed in the Letterman Digital Arts building in San Francisco, where virtual reality experiences like the HTC Vive Trials on Tatooine and 360-degree video experiences like the new Rogue One X-Wing video. ILMxLab even worked with Electronic Arts’ Dice to develop the Star Wars Battlefront Rogue One: X-Wing Mission game expansion for PlayStation VR.
Rob Bredow, chief technology officer at Lucasfilm, oversees ILMxLab and has been a driving force in mapping out how virtual reality, augmented reality and mixed reality will impact Star Wars and other franchises. We caught up with Bredow at the VIEW VR Conference in Turin, Italy recently to talk about the future of entertainment in this exclusive interview.
What do you see Google’s Daydream View opening up for the masses on the mobile side of virtual reality?
Anything that gets you to a tetherless environment and gets you more people who are able to experience VR is a fantastic thing for virtual reality, especially when you’re looking at raising the bar of what can be done in that tetherless environment. A lot of our focus to date has all been on very high-end headsets because the truth is, most of our focus at Lucasfilm is: where are things headed in a few years? The reason we gave Trials on Tatooine out to the world was because it was an interesting experiment, but that’s just a small seed that we’re planting along the way. We’re really interested in where things are going to be a few years from now, and Daydream is a good step in that direction, for sure. But we’re looking three years ahead to try to make sure we’re building all the technology, building our storytelling, and understanding what kinds of things people love, so that we can be there when the audience is ready for those kinds of experiences.
Where are we going to be in three years?
Anybody who tells you they know is probably selling something, right? I really don’t know. VR has had resurgences in the past. This is not the first time this has happened, but this one is bigger and it has some very big players with Facebook, Google, Sony and Microsoft all making big investments. It does seem like the time has come, whether it takes two or three years or whether it takes ten years is still to be determined. People’s first reaction when they put this generation of headsets on is that “Wow” moment, and we’re still in the wow moment as an industry. There will be a phase of like, “Well, this is kind of expensive.” It could see a bit of correction from that and that could be two months or it could be a few years, but we’re going to still stay on that trajectory where VR is going to be useful for lots of things.
Is VR the most exciting technology you’ve explored?
It’s equally as exciting as the early days of computer graphics when every few months there was a whole new technique that made it possible for us to do something completely new again. It really feels very similar to those days where we had some idea of how to make these kinds of images, but it was impractical to do it the right way so we had to do a lot of cheats and cut a lot of corners. Today, walking into VR, all of us have many more years of history and many more years of techniques to rely on, but a lot of those techniques just don’t work in VR for one reason or another, so now there’s a whole new set of challenges ahead of us.
How does video game design and thinking impact virtual reality storytelling?
It’s a perfect question because it’s interesting the way they complement each other. Giving people instructions without ever having to tell them what to do is just a lovely thing that transfers great into VR. You already know how to use your hands. You already know how to interact with things because you’re working in a virtual version of the real world, and the more simple tasks we can give people to do reinforces it as positive, and then we build on that for the next thing which is just classic good game design. We find that transfers really well, and that’s great for storytelling too because you can get people to follow along and you don’t have to worry about a fail through for every case if you’ve trained the user properly. So for the Trials on Tatooine experiment, when we first started doing it, I’d sit there sweating bullets watching people go through it and they’d miss that TIE Fighter coming in. Now we have to rethink how we’re going to train the user to look in the right direction, or change the composition in the storytelling so that they get the kind of interactions we’re looking for.
In VR, sound seems like it takes a bigger focal point. How does the Skywalker Sound fit into ILMxLab?
Yeah, we love working with the Skywalker team. They’re so amazingly talented. Not only did they bring their entire library and all the filmmaking experience, but they’re embedded here at ILMxLab. So, very early in the Trials on Tatooine experience—when we just had a Millennium Falcon model and nothing else—we’re mocking things up, we’re doing it with stand-in sound for everything, and you’re exactly right. We like to joke that audio is 51 percent of the experience and it really is. It’s so important, and there are the obvious things like giving them a TIE Fighter cue when they’re flying in. But that’s barely scratching the surface of what you can do with audio cues to give people a sense of presence; a sense of being immersed in the environment. Things like putting a fun sound system together so when the subwoofers go you feel like the Millennium Falcon is landing right next to you is pretty amazing, and we wouldn’t have discovered that without their involvement.
