Researchers at Prefrontal laboratory introduce Latent Sparse Attention method
Oxford and Princeton-linked researchers at the stealth AI lab Prefrontal have developed a new method to accelerate data processing in large-scale AI models.
Professor James Whittington and Dr. Ruairidh Battleday, co-founders of the British AI research laboratory Prefrontal, have published new findings regarding Latent Sparse Attention (LSA). The research proposes a method intended to reduce the computational bottlenecks associated with 'attention' mechanisms, which currently limit how efficiently AI models process large context windows.
According to the researchers, LSA allows AI systems to categorize information based on semantic meaning rather than sequence position. Testing indicated the method could achieve up to 72.4x faster processing in attention layers compared to conventional full attention mechanisms while maintaining similar accuracy levels. This development targets the rising computational costs associated with scaling context windows to millions of tokens.
This paper follows the team's previous work on DiG-bench, a framework that evaluated the ability of AI models to infer rules through experimentation. The benchmark highlighted significant gaps in the capability of current models to perform independent scientific discovery compared to human participants.
Prefrontal continues to operate in stealth, focusing on human-inspired AI architecture. The team aims to improve how machines handle large datasets to facilitate scientific research and discovery. Future efforts from the laboratory are expected to build on these findings as they seek to integrate these methods into broader AI applications.