Amazon's Panos Panay Says Ambient AI Will Reduce App Reliance

Panay described today's smartphone interaction as "point, shoot, return," saying users must remember what they want their tools to do. He said the next phase begins with ambient devices that use sensors and AI to understand what needs to be done, rather than requiring users to direct every action. "I think where we're headed is a world [that] starts with ambient devices," he said. According to Panay, background operation lets people stay more present, get more done, spend more time with others and worry less about missing something.

The Bee lineup includes a band that can be worn on the wrist or detached as a pendant, plus a credit-card-sized device that attaches magnetically to the back of a smartphone. The devices record audio and take notes about a user's day, then find to-dos, reminders and insights that may be needed later. To address privacy, Bee devices do not store raw audio from conversations; they process audio in real time and delete it when finished. Transcripts are encrypted with keys created and controlled by the device, meaning no one else, including Amazon employees, can access those files unless the user intentionally shares them.

Panay said privacy was central to how ambient devices should work. "You have to give customers that control," he said. "Every product is different, but for Bee specifically, as an example in the ambient space like that, you have to design with privacy in mind. And that's what we do. That's critical."

Panay, who joined Amazon after 19 years at Microsoft, where he oversaw Surface development, said he expects app interaction to become less important as more tasks are handled by AI. "Do I think we have a future where there's a bit of a balance — a whole generation that doesn't say 'give me the app' for that, but says it should just understand what I need? Yeah, I do. I think that transformation is coming. I think that is the next generation," he said. He added that the smartphone will not necessarily disappear and said he is not a proponent of one form factor replacing another. Instead, he expects a new crop of sensor-equipped devices to emerge and an intelligence layer to connect them, provided the technology remains useful.

In a separate onstage conversation at the same conference, Panay told startup founders that failure is an inevitable and required part of building a successful product. "You have to be willing to fail, and actually, not just willing, you actually have to go fail," he said. "If you haven't failed, you probably haven't pushed the boundary. It's literally about harvesting the lessons learned when you fail. You have to be willing to face it, and as you face it, learn from it."

He pointed to the launch of Microsoft's Surface RT tablet as a major failure during his time at the company, one that led to a nearly $1 billion write-down. Panay recalled being on the front cover of a newspaper with the word "failure." He said admitting the failure was easy, but the real challenge was understanding the decisions that led to it, including whether software was incomplete, inventory was wrong or the retail balance was off. "Because the failure wasn't that moment. It was decisions that led up to it two years before that," he said.

Panay said founders should not make excuses or abandon their vision. "You take that failure, but you don't let go of your vision," he said, adding that Surface has done well. He said learning along the way allows founders to adjust course, often by about 30 degrees rather than 180 degrees, and that some people hold on too long because they have not accepted the moment of failure.