You're used to C-3PO being diplomatic and prissy, wringing his droid hands over any and every little setback. You're used to him sticking close to R2-D2. What you're probably not used to, though, is him doing all of those things in your living room. But thanks to a new partnership between secretive VR start-up Magic Leap and Lucasfilm’s ILMxLAB, that's just what you might get.
The partnership, which Magic Leap founder Rony Abovitz announced today at WIRED Business Conference in New York City, pairs the mixed-reality company with what might be Lucasfilm's most forward-looking division: xLab formed last year in order to create experiences for immersive platforms like virtual/augmented/mixed reality. And together, they'll be developing Star Wars-related content for Magic Leap's much-discussed (and still mostly unrevealed) technology.
Neither company will officially acknowledge any specific experiences, or even when Magic Leap's technology will be available to consumers. But a video that Abovitz premiered at the conference explores one possible early application. Where the other Magic Leap teases have concentrated on desktop replacement or spectacle, this one just happens to feature two iconic characters from the world's best-known fictional universe. And yes, they're talking to you. Not only are they in your world, but you're clearly in theirs.
And you've all got problems. "Might I have a word with you please?" C-3PO asks, shuffling in your direction. "I regret to report that due to unforeseen circumstances, we have not yet reached the desired arrangement with Jabba the Hutt regarding Captain Solo's debt." Meanwhile, Artoo comes around the side of a table in the room, and projects a hologram on the tabletop. He's not just hovering on your display, either; the table occludes the lower half of his body.
"An army of stormtroopers is searching for us," Threepio says. "It can only be a matter of time before we are blasted into spare parts!" On the table, said army of stormtroopers fans out, illustrating his point. Then, closer to you, a tiny holographic Millennium Falcon rises from the tabletop takes off, pursued by TIE Fighters—and disappears into hyperspeed. C-3PO just looks at you, a little sadly: "How did we get into this mess?"
These aren't actors hired to come visit you in droid costumes; they're computer-generated phantasms that appear to be solid entities. It's a little bit Holochess, a little bit choose-your-own-adventure, and completely unlike anything you've ever seen.
That's fitting, considering the singular nature of the relationship. Lucasfilm and ILM have worked with partners before, especially technical ones. Similarly, Magic Leap has worked closely with New Zealand effects house Weta for years. But this collaboration isn't just two companies working together: it's two groups of mad geniuses working together to take Star Wars off of the screen and onto the street.
To do that, they'll be creating what Abovitz calls a "semi-secret" joint lab, based at ILM's headquarters in San Francisco's Presidio, that will house researchers and programmers from both companies, and even members of Lucasfilm's Story Group. "I've spent 24 years at ILM, but I cannot recall anything that looks or feels like this," says Vicki Dobbs Beck, xLab's executive in charge. "I've never seen anything where we'll be innovating on both a creative and technological level in such close proximity."
While xLab launched a year ago, Lucasfilm and Magic Leap had been circling around the idea of a relationship for much longer. xLab co-founder John Gaeta came to ILM in 2013 to be a creative director focusing on emerging platforms---or, as he puts it, "listening amongst my friends and colleagues about rumors, always looking for people pursuing new forms of media." The next-big-thing grapevine being what it is, it wasn't long before Gaeta started hearing about Magic Leap.
He reached out to founder Rony Abovitz through social media, and they started to talk. Abovitz introduced Gaeta to Graeme Devine, a longtime game developer who had recently come to Magic Leap, and in fall 2014, the pair invited Gaeta down to Florida to see their technology.
What Gaeta found wasn't pretty---the company was working with a huge, unwieldy prototype they called The Beast---but it was also unprecedented. "I've seen some pretty mad-scientist labs with convoluted constructions, so that part didn't faze me at all," Gaeta says. "But even with the first few things they showed me, I experienced something I'd never in my life experienced: I was able to look at a computer-generated construct, focus along the length of it at any point, and it would abide by my natural eye focus."
That's the kind of thing that makes a visual effects pro into a believer. Gaeta was sold, but he needed to get his colleagues on board. "Even in the house that Lucas built, there was a slow awakening," he says. "With all of the other big things happening here, the setup of a new era of Star Wars, it took some time." But by January 2015, a full complement of ILM folks headed to Magic Leap. Then came Lucasfilm president Kathleen Kennedy and general manager Lynwen Brennan.
The protracted courtship made sense; while Magic Leap has gotten more than a billion dollars in funding, the company's longevity was by no means guaranteed. "They're one of the coolest properties ever built; we're a grungy little startup," Abovitz says. "I think they were watching us before they got involved: 'Are you guys going to be around in six months? Can you actually keep delivering?' It was important for them to check us out over time."
But as the conversation continued, the two companies began to see themselves in each other. "From the beginning, they impressed us with their approach," says xLab CTO Rob Bredow. "Their whole pitch about 'adding magic' was something that was very consistent with what John and I were looking to do with xLab, which is to put you in the middle of the spectacular worlds we create."
Bredow calls that "experiential storytelling," like what happens when you walk down Main Street at Disneyland. "You're not being delivered a narrative in the same way as in a movie theater," Bredow says. "The kind of experiences we can deliver in immersive entertainment will be somewhere between the two."
Armed with the assets and assent of ILM, the Magic Leap team began to explore those experiences. "We built some really crazy moments," Abovitz says. "One-to-one scale ships and characters, things where you suddenly feel like you're there. I could be looking at somebody, and there's also a ship taking off and a droid running around---like you walked onto the set, but it's all real." The ultimate mission is to take the vast persistent Star Wars universe, that sprawling paracosm that the movies are just a slice of---and to make that persistence come alive. "We want to be a medium how to tell stories---and ultimately, maybe the medium," Abowitz says.
That's a ways in the future. The companies are still spinning up the joint lab, communicating every day. But the race is on---both for experiences related to the Star Wars stories we already know, and for things that are still top-secret. "We've seen the roadmap of what's coming, which is kind of crazy if you think about it," says Magic Leap's chief business officer, Rachna Bhasin. "So we've been imagining evergreen experiences from iconic characters as well as how we can expand the story, how characters and moments can happen in a way that they couldn't on the screen."
"The collaboration with Magic Leap is exactly the kind of relationship we want to foster—inspirational, cutting-edge, and innovative," Lucasfilm president Kathleen Kennedy said in an email to WIRED. "With the world-class storytelling and tech talent in our joint lab, the promise of compelling mixed reality entertainment is one step closer."
And a few steps after that? Gaeta has some ideas, and they all involve Star Wars coming to you. Maybe X-wings will fly over your neighborhood at random times. Maybe cantina creatures will crawl out from behind your couch. Maybe, as AI gets better, you'll even be able to talk things over, share a glass of blue milk. For Abovitz, a lifelong science-fiction fan, it can't come too soon. "My life has probably been overly influenced by Star Wars," he says. "I started a robotics company, I think, because of the droids." C-3PO helped him back then, and judging from what Magic Leap and ILMxLab are working on, Abovitz might soon be able to return the favor.
