When Google launched its Pixel 2 flagship smartphone last year, it included something of a surprise: A co-processor called Pixel Visual Core, the company’s first homegrown, consumer-facing piece of silicon. And while that feels like a momentous foray, the co-processor has lain dormant for months. Monday, Pixel Visual Core goes to work.
As it turns out—and as Google had nodded at previously—the hidden chip inside every Pixel serves a narrow but critical purpose. It will use its eight custom cores, its ability to crunch 3 trillion operations per second, all in the service of making your photos look better. Specifically, the photos you take through third-party apps like Instagram, WhatsApp, and Snapchat.
Those are the three partners at the Pixel Visual Core switch-flipping; since it’s open to all developers, more will presumably follow. They’ll all gain the powers to produce Google's HDR+ images, photos that rely on a series of post-processing tricks to make images shot with the Pixel appear more balanced and lifelike. Photos taken with the Pixel Camera app have already benefited from HDR+ powers since launch—that's one reason Pixel 2 earned the highest marks yet given to a smartphone by industry-standard photo-rater DxOMark. But Pixel Visual Core will extend the feature to the streams, feeds, and snaps of Pixel owners as well, after an update that will roll out early this week.
To understand why Google would devote its first homemade smartphone processor to a relatively narrow function—not just photography, but HDR+ specifically—it helps to understand the importance of HDR+ to the Pixel’s photo prowess. For starters, it’s not the HDR you’re used to.
“HDR+ actually works shockingly differently,” says Isaac Reynolds, project manager for Pixel Camera. Where HDR essentially tries to combine three or so simultaneous exposures for the best result, HDR+ takes up to 10 identical underexposed shots. “We take them all and chop them into little bits, and line them on top of one another, and average the image together,” says Reynolds, who ticks off the reduction in noise and color quality as just two of the benefits.
That’s not just hype, or at least not entirely. HDR+ really does have tangible benefits—especially in Google’s implementation.
“HDR+ technology is a very good technology for noise and data preservation. This removes the noise in the picture,” says Hervé Macudzinski, manager of DxOMark.com. “That enables Google to provide a nice picture with low level noise high level detail.”
You can see an example of what that means in the below before-and-after shots, with the usual caveat that Google provided them, and your own experience may vary.
The various benefits of HDR+ are also more or less pronounced depending on the conditions of the shot you’re taking. It helps especially bringing clarity to low-light images, or to give an assist if you for some reason take a portrait with the sun at someone’s back.
Google’s not the only company capable of this particular trick, but its execution clearly stands apart.
“The HDR+ is very impressive because they did something very efficient,” says Macudzinski. “If you want to do that, it’s going to be optimized and very powerful.”
Pixel Visual Core will also power two related photographic enhancements; RAISR, a technique to sharpen zoomed-in shots, and Zero Shutter Lag, which is exactly what it sounds like.
Until now, these optimizations have been off limits for third-party developers. Photos taken within the Instagram app, for instance, look a bit muddled compared to those taken with the Pixel’s native camera app. Which is where Pixel Visual Core comes in.
The primary benefit of the Pixel Visual Core, now that it’s on? You still won’t even notice it, says Ofer Shacham, the chip’s engineering manager.
“If we look at HDR+ as a key benchmark for us, it gives us the ability to run five times faster than anything else in existence, while consuming about 1/10th of the energy of the battery. We can put it under the hood,” says Shacham. “We basically hide it. That’s what enables every developer to use it, while not consuming energy from the battery, and even better, reducing the energy consumption from the battery while those applications take pictures.”
That also hints at why Google decided to go it alone with Pixel Visual Core, rather than rely on the powerful Snapdragon 835 processor that handles the bulk of the Pixel 2’s computational needs. The Pixel Visual Core offers not just customization, but flexibility.
“Google in a sense is a software and algorithm company,” says Shacham. “We want something that allows us to rapidly innovate, rapidly change the algorithm, rapidly improve it.”
To that end, the Pixel Visual Core is also programmable. That means while it works primarily in service of HDR+ today, it could go toward making other applications zip in the future, a possibility that Shacham acknowledges, while declining to go into detail on what sorts of use cases Google envisions.
More broadly, though, the Pixel Visual Core represents Google’s first foray into an increasingly common trend of smartphone manufacturers rolling their own silicon, giving itself tighter control over its product and weaning itself off of chip giant Qualcomm.
“I think it’s significant in that, first off, Google is an advertising company, who is also an operating system provider, and they are going more deeply vertical in what they’re doing by adding semiconductor features to enhance the experience,” says Patrick Moorhead, president of Moor Insights & Strategies. “Any time somebody in software gets into hardware, interesting things happen—as in interesting really good, or interesting really bad.”
It would also make sense, Moorhead says, for Google to extend its processor plans beyond Pixel Visual Core. Microsoft uses a custom system-on-a-chip for the Xbox. Apple’s A series SoC has contributed greatly to the iPhone’s dominance. And with Google having poached a key Apple chip designer last summer, it seems unlikely that an HDR+ coprocessor is the end of the line.
For now, though, Pixel 2 owners can look forward to adding an HDR+ veneer to their social media pics—while waiting Google’s broader ambitions to come more fully into focus.
Read our Pixel 2 review to find out why it's the best Android smartphone you can buy
And if you don't have a Pixel 2, don't worry; there's plenty you can do to customize your Android phone to suit your exact needs
With the Pixel Visual Core, Google became just the latest smartphone maker to roll their own silicon
