Shrunken chatbots: AI models suddenly fit in 4 GB of RAM

A remarkable advance has emerged in the world of AI technology: large voice models such as 27B LLMs can now run on consumer hardware, including mobile phones, thanks to the technique of quantization. Bonsai, a company, has demonstrated how this is possible, meaning that AI models can become significantly more efficient and powerful without relying on powerful servers. This development has far-reaching implications for the application of AI in mobile and private environments as it significantly lowers the barrier to the use of large AI models.

Quantization reduces the size of the models without affecting their performance. This makes it possible to run AI models on devices with limited memory and processing power. This is especially relevant because many users cannot access powerful servers or cloud infrastructures. Instead, they can now use AI models directly on their devices, which greatly simplifies the use of AI in practice.

In addition to increasing efficiency in AI model processing, security in the digital world is also being re-evaluated. For example, Microsoft removed SMS-based authentication from Entra ID to increase security. This shows that companies and organizations are increasingly relying on technological innovation to improve their safety standards. At the same time, AI scaremongering is also viewed critically. An article discusses the limits of artificial intelligence and warns of excessive panic over AI. It emphasizes that LLMs are not AGIs and the danger of AI tyranny is underestimated. These debates are important for developing a realistic perspective on the role of AI in society.

The combination of more efficient AI model processing and increased security is also becoming relevant in the corporate world. This makes the use of AI in practice not only easier, but also safer. Companies can now integrate AI technologies into their processes without relying on external infrastructure. This enables greater flexibility and faster implementation of AI solutions.

## What this means for users AI model processing in xynne is strengthened by such efficiency gains, enabling more powerful but resource-efficient AI solutions.