Microsoft launches Mu, an on-device small language model for Windows 11

Microsoft Unveils Innovative Mu Language Model in Windows 11

In an exciting development, last year, Microsoft introduced a groundbreaking feature with its Copilot+ PCs, equipped with a dedicated Neural Processing Unit (NPU). This NPU is capable of performing over 40 trillion operations per second (TOPS), significantly enhancing the device’s computational capabilities. Accompanying this innovation is a proprietary technology called Phi-Silica, which serves as an on-device responsive language model (SLM).This system not only enhances Microsoft’s first-party applications but also extends functionalities to various third-party applications.

Introducing the Mu Model

Recently, Microsoft further advanced its AI ambitions by unveiling Mu, a compact on-device language model integrated within Windows 11. Designed to empower an AI-driven agent inside the Settings app, Mu aims to understand user queries articulated in natural language. This innovation is seamlessly integrated with the existing search box, streamlining user interactions. Currently, Windows Insiders in the Dev Channel who own Copilot+ PCs have access to this new feature.

Performance Features of the Mu Language Model

Much like the previously discussed Phi-Silica, Mu is optimized to function effectively on NPUs, boasting a remarkable processing speed exceeding 100 tokens per second while operating locally. According to Microsoft’s official announcement, here are some key characteristics of the Mu language model:

Mu is classified as a 330 million parameter encoder-decoder language model.
This encoder-decoder framework offers approximately 47% reduced first-token latency and a decoding speed that is 4.7 times faster compared to decoder-only models of similar specifications.
To optimize efficiency, Mu implements weight sharing across several components, thereby lowering the total parameter count.
The training of Mu was executed using NVIDIA A100 GPUs within Azure Machine Learning, ensuring high-performance outcomes.
Notably, Mu exhibits performance levels comparable to the fine-tuned Phi-3.5-mini model, despite being roughly one-tenth of its size.
This model is particularly effective for handling multi-word queries, while the search functionality within the Settings app continues to offer lexical and semantic results for shorter or incomplete inputs.

How to Experience the New AI Agent

Enthusiastic users interested in exploring this advanced AI-powered agent can download Windows 11 Build 26120.3964 (KB5058496) or any subsequent version to access the innovative features in the Windows Settings app.

Source & Images