Apple in talks with startup on shrinking AI models for iPhone

Apple is in talks with a Silicon Valley startup that says it can shrink powerful artificial intelligence (AI) models small enough to run directly on an iPhone. PrismML, a company backed by Khosla Ventures and spun out of the California Institute of Technology, has publicly released compressed versions of Alibaba's open-source Qwen model. The startup claims it reduced the model from approximately 54 GB to less than 4 GB, enabling all 27 billion of its parameters to run on an iPhone 15 or newer.

PrismML CEO Babak Hassibi told CNBC that Apple and other companies are evaluating the startup's models for speed, energy efficiency, and performance. These discussions are in early stages but are progressing well. The technology could significantly enhance Siri's speed and privacy by processing more AI tasks on-device rather than routing requests to the cloud. Hassibi acknowledged a slight performance trade-off, noting a few percentage points of reduced overall performance.

If PrismML's claims hold up in real-world applications, this technology could reshape demand for memory and datacenter compute. Carolina Milanesi, principal analyst at Creative Strategies, noted that smaller models could enable Apple to move more demanding features onto the iPhone, including advanced computational photography, video generation, and health or fitness tools that handle sensitive personal data.

The startup compresses AI models by simplifying their internal information storage, drastically reducing the memory required. PrismML states its compressed models use 10 to 15 times less memory, provide responses 6 to 8 times faster, and consume 3 to 6 times less energy than conventional versions.