Holy crap, Google’s Pixel 6 launch event was much bigger than expected. Not only did Google debut two affordable flagships with killer new features, but it gave us a closer look at its Tensor chip, a new processor that’s got Qualcomm sharing passive-aggressive memes on Twitter. It also launched Pixel Pass, a Google equivalent to Apple One, and kicked off the real Android 12 rollout.

Read This Article on Review Geek ›

Read 8 remaining paragraphs | Comments

Read 7 remaining paragraphs | Comments

It makes sense that phone manufacturers are paying extra attention to how faces show up in photos, and the new Pixel 6, announced by Google today, introduces a suite of new AI-powered tools to make humans show up better than ever. The two highlights are Face Unblur — which helps reduce blur on moving faces — and Real Tone. The latter is some AI-powered post-processing magic – powered by Google’s brand new Tensor chip –  aiming to make faces with all skin tones show up as well as possible.

Whether you’re taking selfies or someone-elsies, the vast majority of photos taken with a smartphone are of human beings. Traditionally, it has been extremely hard to get the exposure to look good for photos where multiple faces appear in the photo — especially if the faces all have different skin tones. The new Pixel 6 brings a layer of computational photography to the mix to ensure that everyone who appears in the photo looks as good as they can. The Pixel team worked with a diverse set of expert image-makers and photographers to tune the white balance, exposure and algorithms. They claim that this ensures that the photos work for everyone, of every skin tone.

Google highlights that it sees Real Tone as a mission and an improvement on its camera systems, rather than a conclusive solution to the challenges facing photographers. The company has invested substantial resources into ensuring that all people — and particularly people of color — are better represented in the way cameras capture their faces.

“My mother is a dark-skinned Black woman, my father is a white German. My whole life there’s been a question: How do we get one picture where everyone looks good,” said Florian Koenigsberger, Advanced Photography product marketing manager for the Android team, in a briefing interview ahead of the release of the new phones. “The new camera is a step along the journey. Google’s diversity numbers are not a mystery to the world, and we knew we definitely had some shortcomings in terms of lived experience and who could speak authentically to this.”

The camera team worked with photographers, colorists, cinematographers, direc1tors of photography and directors to get a deeper understanding of the challenges in lighting and capturing a diverse set of skin tones — and in particular people with darker skin tones. Among others, the team leaned on the experience from a broad spectrum of professionals, including Insecure’s director of photography Ava Berkofsky, photographer Joshua Kissi, and cinematographer Kira Kelly.

“We focused on bringing this really diverse set of perspectives, not just in terms of ethnicity and skin tones, but also a variety of practices,” said Koenigsberger. “The colorists were actually some of the most interesting people to talk to because they think of this as a science that happens in the process of creating images.”

The Google product team worked with these imaging experts to give them cameras and challenged them to shoot extremely challenging imaging situations, including mixed light sources, back-lighting, interiors, multiple skin tones in one image, etc.

“We had to learn where things fall apart, especially for these communities, and from there we can figure out what direction we can take from there,” Koenigsberger explains. “The imaging professionals were very frank, and they were directly in the room with our engineers. I helped facilitate these conversations, and it was fascinating to see not just the technical learnings, but also the cultural learning that happened in this space. I am talking about ashiness, darker skin tones, textures. The nuances for mid-tones can vary.”

The process starts with the camera’s facial detection algorithms. Once the camera knows it is looking at a face, it can start figuring out how to render the image in a way that works well. In testing across devices, Google’s team found that the Pixel 6 consistently performed better than those from competing manufacturers, and even the older-generation Pixel phones.

It isn’t immediately clear how the feature works in practice, and whether it does global edits (i.e. applies the same filter across the entire image), or whether the AI edits individual faces as part of its editing pass. We are hoping to take a deeper look at this specific aspect of the camera functionality to see how it works in practice very soon.

The camera team highlights that the work done in this space means that the training sets for creating the camera algorithms are more diverse by a factor of 25. The Real Tone feature is a core part of the camera algorithms, and it cannot be turned off or disabled.

The former Head of the Facebook app, who reported directly to CEO Mark Zuckerberg, Fidji Simo, defended the social network at the start of an interview at the WSJ Tech Live event this afternoon. The exec was there to discuss her new role as Instacart CEO and her vision for the future of food delivery, but was asked to comment on the recent Facebook whistleblower’s testimony and the attention it has since raised.

Simo said she understood the scrutiny given Facebook’s impact on people’s lives. But she’s also worried that Facebook will never be able to do enough to appease its critics at this point, despite the complexity of the issues Facebook is grappling with as one of the world’s largest social networks.

“They are spending billions of dollars in keeping people safe. They are doing the most in-depth research of any company I know to understand their impact,” she argued, still very much on Facebook’s side, despite her recent departure. “And I think my worry is that people want ‘yes’ or ‘no’ answers to this question, but really these questions require a lot of nuance,” she added.

While the whistleblower, Frances Haugen, suggested that Facebook’s decision to prioritize user engagement through its algorithms was ultimately putting profits over people, Simo cautioned the choices weren’t quite as binary as have been described to date. She explained that making changes based on the research Facebook had invested in wasn’t just a matter of turning a dial and “all of a sudden, magically problems disappear — because Facebook is fundamentally a reflection of humanity,” she said.

Instead, Simo said that the real issues at Facebook were around how every change Facebook makes can have significant societal applications at this point. It has to work to determine how it can improve upon the potentially problematic areas of its business without incidentally affecting other things along the way.

“When we discuss trade-offs, it’s usually trade-offs between two types of societal impacts,” she noted.

As an example, Simo used what would seem like a fairly straightforward adjustment to make: determine which posts make Facebook users angry then show people less of those.

As Haugen had testified, Facebook’s algorithms have been designed to reward engagement. That means posts with “likes” and other interactions spread more widely and are distributed higher up in people’s News Feeds. But she also said engagement doesn’t just come from likes and positive reactions. Engagement-based algorithms will ultimately prioritize clickbait and posts that make people angry. This, in turn, can help to boost the spread of posts eliciting stronger reactions, like misinformation or even toxic and violent content.

Simo, however, said it’s not as simple as it sounds to just dial down the anger across Facebook, as doing so would lead to another type of societal impact.

“You start digging in and you realize that the biggest societal movements were created out of anger,” she said. That led the company to question how it could make a change that could impact people’s activism.

(This isn’t quite how that situation unfolded, according to a report by The WSJ. Instead, when the algorithm was tweaked to prioritize personal posts over professionally produced content, publishers and political parties adjusted their posts towards outrage and sensationalism. And Zuckerberg resisted some of the proposed fixes to this problem, the report said.)

“That’s just a random example,” Simo said of the “anger” problem. “But literally, on every issue, there is always a trade-off that is another type of societal impact. And I can tell you for having been in these rooms for many, many years. It’s really never about like, oh, are we doing the right thing for society, versus the right thing for Facebook and for profits’…the debate was really between some kinds of societal impact and another kind — which is a very hard debate to have as a private company.”

This, she added, was why Facebook wanted regulations.

“It’s not surprising that Facebook has been calling for regulation in this space for a very long time because they never want to be in a position of being the ones deciding which implications, which ramifications, which trade-offs they need to make between one type of societal impact and another type of societal impact. The governments are better positioned to do that,” she said.

Given the increasing amount of evidence coming out that Facebook itself understood, through its own internal research, that there were areas of its business that negatively impact society, Simo didn’t chalk up her departure from the social network to anything that was going on with Facebook itself.

Instead, she said she just wasn’t learning as much after ten years with the company, and Instacart presented her with a great opportunity where she could learn “a different set of things,” she said.