Running AI Applications Locally on Low RAM PCs: A Step-by-Step Guide

This year can certainly be coined as the “AI-powered era.” From our smartphones and PCs to the workplaces, the influence of artificial intelligence has become omnipresent. Nonetheless, due to specific hardware prerequisites, not all devices are equipped to support AI functionalities fully.

If you own an older PC that falls short of the latest AI requirements, don’t fret. There are still several options to engage with these advanced technologies on less powerful systems. In this comprehensive guide, we will explore whether it’s feasible to operate large language models (LLMs) on low-end PCs, and if so, which models are suitable for older machines.

Is It Possible to Run AI Locally on PCs with Limited RAM and Outdated Hardware?

Large Language Models (LLMs) must be installed locally on your device before they can be executed. Consequently, they typically demand robust hardware and ample RAM for optimal performance. However, some LLMs can function without stringent hardware demands.

What Are the Minimum Specifications Needed to Run LLMs?

There isn’t a one-size-fits-all answer to this question. The hardware requirements vary based on the specific LLM being utilized. While certain models may need a minimum of 8GB of RAM, others might require as much as 16GB. Generally, LLMs are best utilized on PCs with at least an eight-core CPU and a strong in-built or dedicated graphics processing unit (GPU).

How Can I Utilize AI on My Older PC?

If your PC doesn’t meet the minimum requirements for prevalent LLMs but you still wish to run AI tools, you have options. Online AI chatbots can be a great solution. These digital assistants can perform nearly all the functions of locally installed LLMs, providing you with robust capabilities regardless of your device’s specifications.

However, local chatbots do have their benefits, including functionality without needing a constant internet connection and better access to data stored on your device. In contrast, online solutions depend on consistent internet access.

Top LLMs for Low-End Windows PCs

Up to this point, we’ve identified two lightweight LLMs that can operate efficiently on older devices: DistilBERT and ALBERT. It’s important to note that while these models excel in various tasks, they may not be suitable for highly complex problems.

Developed by HuggingFace, DistilBERT is marketed as a “smaller, faster, and cheaper” alternative to BERT. Its compact size allows it to operate using minimal memory, making it an exceptionally efficient lightweight LLM.

ALBERT is another option that functions similarly to DistilBERT but employs different design principles. This model is suitable for high-end PCs as well but works adequately for less demanding tasks.

If your PC has 8GB of RAM, the GPT Neo 125M version is also an option. This open-source model offers the flexibility for customization based on user preferences and strikes a balance between performance and system requirements, often matching or surpassing GPT-2’s capabilities.

Steps to Install LLM Chatbots

Each LLM comes with its unique installation procedures, so no universal method exists. Before proceeding with the download and installation, it’s essential to verify your hardware specifications and identify a compatible LLM. Afterward, download a tool like Docker, which will enable you to run applications within isolated environments.

Next, visit the official website of your chosen LLM, such as HuggingFace or GitHub, and follow the provided installation instructions meticulously. Additionally, be prepared for potential software updates that may be necessary to ensure seamless operation.

Source