A daring developer reincarnates Clippy from the Microsoft Graveyard to pop up like an AI genie with insightful details about cryptic screenshots on Windows PCs — like Google’s Circle to Search tool

What you need to know

  • An avid tech enthusiast and developer recently shared an interesting project featuring Microsoft’s discontinued office assistant, Clippy.
  • The project dubbed AI-Snip leverages Clippy’s smarts, explaining obscure details captured in screenshots.
  • The developer also claims the project can be used for translation or custom prompts.

As a tech enthusiast who has spent countless hours tinkering with various software applications over the years, I must say that the resurrection of Microsoft’s beloved office assistant, Clippy, in the form of AI-Snip, has brought a smile to my face. It’s reminiscent of those nostalgic days when Clippy would pop up at the most unexpected moments, offering unsolicited advice or help that was often more confusing than enlightening.


Just over a week ago, Salesforce CEO Marc Benioff criticized Microsoft for allegedly underperforming in the AI industry by likening their new tool, Copilot, to the outdated Microsoft Clippy. Now, it appears that Microsoft’s virtual assistant may have risen from the dead, as an enthusiastic developer has showcased a project called AI-Snip on Reddit.

According to Developer Yannik Keller:

This tool is incredibly handy for daily tasks. Whenever I encounter something I don’t comprehend, I simply take a screenshot and have it explained to me. Beyond that, it also offers features like translation and personalized prompts.

Demo of GPT-4o as an Image to Text model that makes MS Clippy explain the screenshots you take. from r/OpenAI

The project has generated excitement on Reddit, as numerous users reminisce about the virtual office assistant by sharing fond memories. One user expressed their excitement with, “My heart leaps for joy… oh wait, that’s too exaggerated, let’s calm it down… stop, darn you again, Clippyyyyy,” indicating a strong emotional response. Another user commended the idea, saying, “It’s an excellent concept.” They also expressed surprise that Microsoft isn’t marketing something enjoyable like this.

The project resembles Google’s Circle to Search function found on Android devices. This feature enables users to search for anything directly on their Android phone without having to switch apps, by either circling, highlighting, scribbling, or tapping the specific area that catches their attention. As you focus on a particular region, information related to it will be displayed in real-time.

Microsoft Edge has an equivalent tool called Circle to Copilot, functioning much like Google’s Circle to Search. However, an upcoming feature named Click-to-Do may prove even more potent in its capabilities.

As per the developer, the most straightforward method to operate AI-snip on your Windows system is by downloading the binary from the releases page. For advanced users who are proficient in Python, there’s also an option to clone the repository and follow the detailed instructions provided within the AI-Snip GitHub repository for a manual setup.

On Linux as well, you can operate AI-Snip, but keep in mind that the developer mentioned a limitation: “With each execution of this program, you will only be able to perform one snip.” This is due to the fact that Linux lacks a consistent system tray similar to Windows.

Here are available hotkeys when leveraging AI-Snip on your PC:

  • D: Toggles clippy (always starts on)
  • C: Toggles if AI response is copied to clipboard (always starts off). This will also disable streaming, resulting in slower response times.
  • E: Go into translation into english mode. Will also set clipboard on.
  • L: Go into Latex mode. Toggles clippy off and clipboard on.
  • T: Free prompt mode. Write your own prompt for the AI to answer.
  • Q or Esc: Cancel your ongoing snip or close clippy (you may have to re-select the clippy window if you de-selected it).

According to Keller, AI-Snip operates in an explanatory setting. This implies that whenever you clip a section, Clippy will automatically appear with additional details and background information about the clipped content for better understanding.

Read More

2024-10-29 23:39