I turned PowerToys Run into an AI chatbot, and while it feels like a proof of concept, it’s still very impressive

by

AI permeates virtually every aspect of our lives today, and it’s hard not to notice its presence. While some may view it with skepticism, AI offers numerous practical applications that can help simplify our daily routines.

Chatbots, which are quite prevalent among artificial intelligence technologies, such as ChatGPT and Copilot, function best when they have access to a cloud-based network for optimal performance.

While this isn’t the sole method for applying AI Large Language Models, there are numerous user-friendly tools available these days that make it possible to implement such technology on personal devices closer to home.

Using a third-party plug-in in conjunction with PowerToys Run, an excellent substitute for the Windows 11 Start Menu, sets up a local AI chatbot, which can be easily accessed via a straightforward keyboard shortcut.

Setting this up requires a bit of initial effort, and it may not be lightning-quick at present, yet its proof of concept is undeniably impressive. After experimenting with it myself, I managed to develop my personal PowerToys Run AI chatbot, which I find quite compelling.

PowerToys + Ollama = local AI chatbot anywhere on your Windows 11 PC

In simpler terms, when it comes to PowerToys, its role is primarily a user interface. It employs an external add-on to communicate with Ollama, which is the software that actually executes the Language Learning Model (LLM).

To get started with this, it’s essential that you set up both components first. This isn’t too challenging, and the Ollama GitHub repository serves as a useful launchpad. However, keep in mind that an AMD or NVIDIA GPU offering optimal performance is required for best results.

In essence, Ollama functions as a command-line interface (CLI) tool, which implies you’ll be utilizing it within a terminal environment. However, it’s possible to incorporate a graphical user interface (GUI) if desired, but for our current purposes, this additional feature isn’t required.

After installation, you might find yourself wanting to get the latest version of Llama, specifically 3.1. By default, the plugin is configured for this. To do so, open your terminal window and type in PowerShell commands as follows:

ollama pull llama3.1

If you’re looking for a quick way to test out the plugin without delving into the latest version, Llama 3.1 is your go-to option for downloading. However, you can adjust the Language Model used with the plugin within the Settings of the PowerToys app. Just choose whatever model suits your needs best.

To make use of this tool, it’s essential that you have the PowerToys-Run-LocalLLmm plugin installed. You can acquire it by downloading it from its GitHub repository and then extracting it to the folder on your computer where PowerToys Run plugins are kept.

For example, on my PC, it followed this path:

LOCALAPPDATA%\Microsoft\PowerToys\PowerToys Run\Plugins

After that, just make sure you quit and restart PowerToys to make it take effect.

Easy to use, and a great example of how to integrate AI into using a Windows 11 PC

To initiate your preferred Live Local Machine (LLM) function within PowerToys Run, just type “llm” followed by your query. For brief queries, the response will come quickly. However, if it’s a more detailed query, you may need to be patient for the results.

From what I gather, it seems that the reason for this is that it outputs the whole response at a time. However, if you use Ollama in the terminal, it will display the response word by word. Consequently, the execution times may not vary significantly, but you won’t notice any activity due to the way it presents the results.

In simpler terms, you might feel like your brain is working slowly, and it seems to be true, but there’s a reason behind it. Your computer hardware also plays a significant role in this process. For instance, I have an RTX 5080 graphics card that can handle Ollama smoothly. With less powerful hardware, you may experience longer wait times without a doubt.

One potential solution to address this issue straight away is that the system may automatically truncate long responses from your plugin. Since the plugin does not trigger scrolling on its own, it’s essential to adjust the size of the PowerToys Run window manually to accommodate longer outputs.

To expand the display area for your search results, adjust the setting named “Number of Results Visible Before Scrolling” in the primary PowerToys application by increasing its value. Please note that despite this change, the plugin itself won’t start scrolling, but it will present you with a larger result box instead.

In this post, the images are arranged with a spacing of 10. Increasing the value makes more room visible. If parts of the image are still truncated, you can copy the entire image by clicking the ‘copy’ button, and then paste it into a text editor for better viewing.

In essence, this configuration aligns perfectly with my daily needs for Windows 11. Why can’t Copilot be integrated directly into the system, much like how Microsoft integrates it into other applications? Given its widespread use across various platforms, it seems fitting.

The primary advantage I find with PowerToys Run is that I utilize it more than any other tool on Windows 11. Unlike other tools that might interrupt my workflow by requiring AI assistance, PowerToys integrates it seamlessly. Plus, since it operates locally instead of relying on a cloud service, I strongly favor its use.

Read More

2025-05-12 17:06