ModelBest and OpenBMB Release MiniCPM-V 4.7 for Phones and Tablets

ModelBest and the OpenBMB community have released MiniCPM-V 4.7, a new open-source multimodal model intended for on-device operation on mobile devices like phones and tablets. The model brings advanced capabilities for understanding images, documents, and videos to hardware with around 3GB of memory.

The model is available in 35 billion parameter (35B-A3B) and 1 billion parameter (1B) versions. The 35B-A3B version is described as the first open-source Mixture-of-Experts (MoE) multimodal model built for smooth on-device performance without compromising quality. It activates approximately 3 billion parameters per token and loads expert modules as needed, bringing large-model multimodal capability to iOS and Android devices.

Capabilities include general visual understanding, STEM problem-solving, OCR (Optical Character Recognition), document and chart interpretation, and long-video understanding. MiniCPM-V 4.7 aims to improve AI efficiency and performance on edge devices by reducing inference time. The 1B version is designed for lighter-weight devices, supporting basic vision, OCR, and multi-image understanding.

ModelBest's Lookahead Expert Scheduling (LES) technology predicts future expert needs and pre-loads them, significantly enhancing performance. Combined with optimizations like layer-wise quantization, this can increase generation speed up to five times compared to previous iterations. The model also supports various levels of visual token compression to balance detail and speed.

MiniCPM-V 4.7 is released as open source for developers and has support from hardware manufacturers and technology companies, including AMD, Apple, Arm, and NVIDIA, as well as various chip vendors via the FlagOS software stack. Model weights, inference code, and fine-tuning code are publicly available.