What you need to know
- Reddit has blocked Bing from crawling the social media site.
- The change took place on July 1, 2024, after which Microsoft’s AI tools had to stop crawling Reddit to index content.
- It appears that all search engines apart from Google are now unable to scrape Reddit for data, which means those search engines cannot show results from Reddit’s website.
As a tech enthusiast with years of experience using various search engines to find information on the internet, I’ve always relied on Bing for its comprehensive search results, especially when it comes to Reddit. The social media platform is a goldmine of information and discussions on virtually any topic under the sun, and Bing made it easy to access that content through its search engine.
Starting this week, searching for Reddit content on Bing will yield no results. If you attempt to restrict your search results to only include Reddit or include “Reddit” in your query with Bing, Microsoft’s search engine, you won’t see any outcomes from the platform. The same is true when using DuckDuckGo or other non-Google search engines.
Reddit has implemented a robots.txt directive, prohibiting websites from collecting data through web scraping. For those interested, you may access the particular page detailing this restriction to review the associated code.
Microsoft confirmed the change in a statement to Search Engine Land:
“Respecting the robots.txt protocol, Bing ceased crawling Reddit starting July 1, due to their newly implemented restriction in this file that forbids any crawling activity on their site.”
In spite of a change in Google’s system, searches on Reddit can still elicit relevant results when conducted through Google. According to a Reddit representative who spoke with The Verge, the absence of Bing from Reddit’s search functionality does not stem from its alliance with Google.
“Our ongoing negotiations with Google concerning our partnership are unconnected to this issue. In fact, we’ve been in talks with various search engines. Unfortunately, not all have been willing or capable of providing binding assurances regarding their handling of Reddit content, including its application towards AI technologies.”
Based on my extensive experience in tech journalism, I can tell you that partnerships between tech giants and popular online platforms are quite common. In this case, it was reported that Google and Reddit entered into a deal allegedly worth $60 million. This arrangement grants Google real-time access to Reddit’s content. However, when asked about Bing being blocked on Reddit, the spokesperson for the company did not deny the existence of the partnership but did not confirm whether it was the reason behind this issue.
A blessing in disguise?
Search engines like Bing have been useful for discovering topics discussed on Reddit. Previously, I managed to locate relevant content by incorporating “Reddit” into my queries or explicitly limiting results to Reddit. Search engines are typically more proficient at retrieving information compared to Reddit’s internal search engine.
Exploring Reddit can be beneficial for discovering particular threads or information. However, Bing’s inability to access Reddit data might be advantageous when it comes to generating AI-driven responses. For instance, Google’s mishaps, such as recommending people eat glue or suggesting a depressed person jump off a bridge, occurred due to its integration with Reddit content for search results. These suggestions were far from healthy or appropriate.
Thank you to Liam (@djsonicwarrior on X) for the tip about Bing being blocked by Reddit.
Read More
- GOAT PREDICTION. GOAT cryptocurrency
- LDO PREDICTION. LDO cryptocurrency
- JASMY PREDICTION. JASMY cryptocurrency
- DOP PREDICTION. DOP cryptocurrency
- CYBER PREDICTION. CYBER cryptocurrency
- NOT PREDICTION. NOT cryptocurrency
- SCR PREDICTION. SCR cryptocurrency
- BRN PREDICTION. BRN cryptocurrency
- METIS PREDICTION. METIS cryptocurrency
- DEXE PREDICTION. DEXE cryptocurrency
2024-07-25 22:09