Xiaomi to Update MiMo-V3 Architecture with HySparse2 Technology
Xiaomi is updating its MiMo-V3 model architecture with HySparse2 technology. The new design aims to reduce the costs associated with long-context inference and improve retrieval for agentic workloads.
Xiaomi is implementing a significant architectural update for its MiMo-V3 artificial intelligence model. The core of this upgrade is the HySparse2 technology, engineered to substantially lower the expenses related to processing extensive data contexts and enhance data retrieval efficiency in agent-based applications.
Fuli Luo, who leads the Xiaomi MiMo project, confirmed the changes. HySparse2 is specifically developed to cut down the inference costs for long contexts. The company states that the technology can reduce prefill computation by up to 5.02 times and the KV cache by 4.5 times when processing contexts exceeding one million tokens.
These advancements could have broad implications for AI development, potentially enabling more efficient and cost-effective analysis of lengthy documents, conversation histories, or other large datasets. The improvements aim to make sophisticated AI applications more accessible.
Xiaomi also reported performance gains on several benchmarks, noting higher MRCRv2 and RULER-v2 scores. Concurrently, AgentPPL and LongPPL results decreased, indicating the model's improved capacity for understanding and generating coherent text over extended inputs.