Qualcomm’s ‘Holy Grail’: Generative AI Is Coming to Phones Soon

A Snapdragon 8 Gen 2, Qualcomm’s premium mobile chipset from 2022, in front of a rig to test chips.

David Lumb / CNET

Generative AI like ChatGPT and Midjourney have dazzled imaginations and disrupted industries, but their debut has mostly been limited to browser windows on desktop computers. Next year, you’ll be able to make use of generative AI on the go once premium phones launch with Qualcomm’s top-tier chips inside.

Phones have used AI for years to touch up photos and improve autocorrect, but generative AI tools could bring the next level of enhancements to the mobile experience. Qualcomm is building generative AI into its next generation of premium chips, which are set to debut at its annual Qualcomm Summit in Hawaii in late October. 

Summit attendees will get to experience firsthand what generative AI will bring to phones, but Qualcomm senior vice president of product management Ziad Asghar described to CNET why users should get excited for on-device AI. For one, having access to a user’s data — driving patterns, restaurant searches, photos and more — all in one place will make solutions generated by AI in your phone much more customized and helpful than general responses from cloud-based generative AI. 

“I think that’s going to be the holy grail,” Asghar said. “That’s the true promise that makes us really excited about where this technology can go.”

There are other advantages to having generative AI on-device. Most importantly, queries and personal data searched are kept private and not relayed through a distant server. Using local AI is also faster than waiting for cloud computation, and it can work while traveling on airplanes or in other areas that lack cell service. 

But an on-device solution also makes business and efficiency sense. As machine learning models have gotten more complex (from hundreds of thousands of parameters to billions, Asghar said), it’s more expensive to run servers answering queries, as Qualcomm explained in a white paper published last month. Back in April, OpenAI was estimated to spend around $700,000 per day getting ChatGPT to answer prompts, and that cost prediction was based on the older GPT-3 model, not the newer GPT-4 that is more complex and likely to be costlier to maintain at scale. Instead of needing an entire server farm, Qualcomm’s solution is to have a device’s existing silicon brain do all the thinking needed — at no extra cost.

“Running AI on your phone is effectively free — you paid for the computing power up front,” Techsponential analyst Avi Greengart told CNET over email. 

Greengart saw Qualcomm’s on-device generative AI in action when the chipmaker had it on display at Mobile World Congress in February, using a Snapdragon 8 Gen 2-powered Android phone to run the image generating software Stable Diffusion. Despite being an early demo, he found it “tremendously exciting.” 

A Snapdragon 8 Gen 2 chipset.

David Lumb/CNET

What on-device generative AI provides users

Qualcomm has ideas for what people could do with phone-based generative AI, improving everything from productivity tasks to watching entertainment to creating content. 

As the Stable Diffusion demo showcased, on-device generative AI could allow people to tweak images on command, like asking it to change the background to put you in front of the Venice canals, Asghar said. Or they could have it generate a completely new image — but that’s just the beginning, as text and visual large learning models could work in succession to flow from an idea to a ready output.

Using multiple models, Asghar said, a user could have their speech translated by automatic speech recognition into text that is then fed into an image generator. Take that a step further and have your phone render a person’s face, which uses generative AI to make realistic mouth movements and text-to-speech to speak back to you, and boom, you’ve got a generative AI-powered virtual assistant you can have full conversations with. 

This specific example could be powered in part by third-party AI, like Facebook parent company Meta’s recently launched large language model Llama 2 in partnership with Microsoft as well as Qualcomm.  

“[Llama 2] will allow customers, partners and developers to build use cases, such as intelligent virtual assistants, productivity applications, content creation tools, entertainment and more,” Qualcomm said in a press release at the time. “These new on-device AI experiences, powered by Snapdragon, can work in areas with no connectivity or even in airplane mode.”

Inside Qualcomm HQ’s Appointment-Only Museum Filled With Retro Phones

+21 more See all photos

Qualcomm won’t limit these features to phones. At its upcoming summit, the company plans to announce generative AI solutions for PC and auto too. That personal assistant could help you with your to-do lists, schedule meetings and shoot off emails. If you’re stuck outside the office and need to give a presentation, Asghar said, the AI could generate a new background so it doesn’t look like you’re sitting in your car and bring up a slide deck (or even help present it).

“For those of us who grew up watching Knight Rider, well, KITT is now going to be real,” Asghar said, referring to the TV show’s iconic smart car.

Regardless of the platform, the core generative AI solution will exist on-device. It could help with office busywork, like automatically generating notes from a call and creating a five-slide deck summarizing its key points (“This is like Clippy, but on steroids, right?” Asghar said). Or it could fabricate digital worlds from scratch in AR and VR.

Beyond fantasy worlds, generative AI could help blind people navigate the real world. Asghar described a situation where image-to-3D-image-to-text-to-speech model handoffs could use the phone’s camera to recognize when a user is at an intersection and inform them when to stop, as well as how many cars are coming from which directions.

On the education front — perhaps using a webcam or a phone’s camera — generative AI could gauge how well students are absorbing a teaching lesson, perhaps by tracking their expressions and body language. And then the generative AI could tailor the material to each student’s strengths and weaknesses, Asghar theorized.  

These are all Qualcomm’s predictions, but third parties will have to decide how best to harness the technology to improve their own products and services. For phones, generative AI could have a real impact once it’s integrated with mobile apps for more customized gaming experiences, social media and content creation, Techsponential’s Greengart said.

It’s hard to tell what that means for users until app makers have generative AI tech on hand to tinker and integrate into their apps. It’s easier to extrapolate what it could do based on how AI helps people right now. Roger Entner, analyst for Recon Analytics, predicts that generative AI will help fix flaws in suboptimal photos, generate filters for social media, and refine autocorrect — problems that exist right now. 

“Generative AI here creates a quality of use improvement that soon we will take for granted,” Entner told CNET over email.

A Snapdragon 8 Gen 2 encased in a red puck in front of a rig used to test chips in production.

David Lumb / CNET

Generative AI is coming to premium phones first

Current generative AI solutions rely on big server farms to answer queries at scale, but Qualcomm is confident that its on-device silicon can handle single-user needs. In Asghar’s labs, the company’s chips handled AI models with 7 billion parameters (aspects that evaluate data and change the tone or accuracy of its output), which is far below the 175 billion parameters of OpenAI’s GPT-3 model that powers ChatGPT, but should suit mobile searches.

“We will actually be able to show that running on the device at the [Hawaii] summit,” Asghar said.

The demo device will likely pack Qualcomm’s next top-tier chip, presumably the Snapdragon 8 Gen 3 that will end up in next year’s premium Android phones. The demo device running Stable Diffusion at MWC 2023 used the Snapdragon 8 Gen 2 announced at last year’s Snapdragon Summit in Hawaii.

In an era of phones barely lasting through the day before needing to recharge, there’s also concern over whether summoning the generative AI genie throughout the day will drain your battery even faster. We’ll have to wait for real-world tests to see how phones implement and optimize the technology, but Asghar pointed out that the MWC 2023 demo was running queries for attendees all day and didn’t exhaust the battery or even warm to the touch. He believes Qualcomm’s silicon is uniquely capable, with generative AI running mostly on a Snapdragon chipset’s Hexagon processor and neural processing unit, with “very good power consumption.”

“I think there is going to be concern for those who do not have dedicated pieces of hardware to do this processing,” Asghar said.

Asghar believes that next year’s premium Android phones powered with Qualcomm’s silicon will be able to use generative AI. But it will take some time for that to trickle down to cheaper phones. Much like how on current phones AI assistance for cleaning up images, audio and video is best at the top of the lineup and gets less effective for cheaper phones, generative AI capabilities will be lesser (but still present) the further down you go in Qualcomm’s chip catalog.

“Maybe you can do a 10-plus billion parameter model in the premium, and the tier below that might be lesser than that, if you’re below that then it might be lesser than that,” Asghar said. “So it will be a graceful degradation of those experiences, but they will extend into the other products as well.”

As with 5G, Qualcomm may be first to a new technology with generative AI, but it won’t be the last. Apple has quietly been improving its on-device AI, with senior vice president of software Craig Federighi noting in a post-Worldwide Developers Conference chat that they swapped in a more powerful transformer language model to improve autocorrect. Apple has even reportedly been testing its own “Apple GPT” chatbot internally. The tech giant is said to be developing its own framework to create large language models in order to compete in the AI space, which has heated up since OpenAI released ChatGPT to the public late in 2022.

Watch this: Comparing Bing Chat, Bard Chat and ChatGPT 11:37

Apple’s AI could enter the race against Google’s Bard AI and Microsoft’s Bing AI, both of which have had limited releases this year for public testing. Those follow the more traditional “intelligent chatbot” model of generative AI enhancing software, but it’s possible they’ll arrive on phones through apps or be accessed through a web browser. Both Google and Microsoft are already integrating generative AI into their productivity platforms, so users will likely see their efforts first in mobile versions of Google Docs or Microsoft Office.

But for most phone owners, Qualcomm’s chip-based generative AI could be the first impactful use of a new technology. We’ll have to wait for the Snapdragon Summit to see how much our mobile experience may be changing as soon as next year.