LMU professor: AI development at inflection point, smaller models are key
Professor Björn Ommer at LMU Munich highlights an inflection point in AI development. He criticizes the current "scaling race," where AI model size is used as a performance metric. Ommer argues that smaller, more efficient models offer more sustainable solutions.
Munich – Artificial intelligence (AI) development is at a critical inflection point, where simply increasing model size no longer automatically enhances intelligence. Professor Björn Ommer from Ludwig-Maximilians-Universität München (LMU) strongly criticizes the current "scaling race," where large language models (LLMs) constantly aim to increase their size and advertised performance.
"Apparently, the 'large' in large language models has become a measure of quality," Ommer stated in an interview. "People believe AI simply becomes more powerful by growing bigger. But AI development is currently at an inflection point: models are no longer getting more intelligent simply by being scaled up with more computing power and even more training data."
Ommer, known for contributions including the Stable Diffusion image generation model, emphasizes that the current race is unsustainable both economically and ecologically. Large models require vast amounts of data, computing power, and energy, limiting their use to only major tech corporations. This "scaling race" is not environmentally sound, according to Ommer.
Furthermore, Ommer raises concerns regarding data privacy. Terms of service from large providers may allow user input data to be used for training their models. This can be a barrier, particularly for organizations handling sensitive data, such as in the medical sector.
"This is also why we did not want to participate in the scaling race with our algorithms," Ommer said. He advocates for smaller, independent solutions tailored to specific local needs, allowing for better data control and security.