Thinking Machines unveils smaller, high-performance AI model

Thinking Machines, a well-funded startup led by former OpenAI CTO Mira Murati, has released Inkling-Small, its latest open-source AI model. Launched just two weeks after its predecessor, Inkling, the new model promises near-equivalent performance at approximately one-quarter the size, offering substantial benefits in compute efficiency and cost for businesses.

The new 276-billion-parameter model, released under an Apache 2.0 license, achieves performance metrics close to the original 975-billion-parameter Inkling. It ranks only one point behind its larger sibling on the Artificial Intelligence Analysis Index. Capable of processing text, image, and audio inputs and producing text outputs, Inkling-Small supports a context window of up to one million tokens, making it a competitive option for organizations with more constrained computational resources.

Inkling-Small employs a Mixture-of-Experts (MoE) architecture. This design allows the model to utilize only 12 billion active parameters per token out of its total 276 billion parameters. This efficiency significantly reduces its computational requirements and inference costs while preserving much of the larger model's coding, reasoning, and multimodal capabilities.

The model is available with full weights on Hugging Face, with fine-tuning support via its Tinker API. While Inkling-Small is not suitable for laptops, it is considerably easier to deploy than the original Inkling. This makes it a practical choice for enterprises possessing some GPU capacity. Users can adjust compute needs to balance quality, latency, and cost based on task difficulty.